Cleared Cmd Notification

Binford, Charles cbinford at
Tue Dec 7 21:10:14 PST 1999

* From the T10 Reflector (t10 at, posted by:
* "Binford, Charles" <charles.binford at>
This email is a follow-up to T10/99-311r0.  I received some direct email
asking a few probing questions.  After answering, I decided it might be good
send this thread to the wider audience (sorry for the duplication for those
on both T10 & T11).

I've moved my summary to the top for those who don't want to read my
rambling, detailed responses.

To summarize, right now I'm leaning toward the following:
*** Goals:
- Provide some notification to the "other" initiators that their I/Os to a
device have been cleared by a task management function(e.g. Target Reset).

- Provide a mechanism to quickly cleanup aborted I/Os without having to
resort to ABTs/RRQ on each outstanding I/O to catch all of those "ambiguous"
exchanges (The ABTs/RRQ is currently required by PLDA).

*** Solution:
- Send a new SCSI Status, 'Cmd Cleared', for any I/O which is canceled by
any task management function (excluding Abort Tag).

- Cmd Cleared Status sent on all I/Os cleared by a task management function
regardless of initiator (including the "initiating" initiator)

- No ordering requirements or restrictions between the Cmd Cleared Status on
any/all affected I/Os vs. either Task Mgmt Response or any UA resulting from
the Task Mgmt function.

- Other causes of I/Os being abnormally terminated (e.g. PLOGI, LOGO, TPRLO,
etc.) are handled as they are today.  (I'd prefer if the target sent an
explicit PRLO to all affected initiators if it receives TPRLO).

Now, here is the original questions from Brian Doore (forwarded to me by Jim
Coomes)  and my response.
From: Brian Doore
Date: 11/10/99 06:10 PM

To:   Jim Coomes at SEAGATE

Subject:  Re: Cleared Cmd Notification  (Document link: Jim Coomes)

It's a nice idea but I think there are many implementation difficulties.
are a few "improvements" to make implementation easier:

1.  I don't like sending a response to every command.  I'd rather use
Commands as a model, where we just return this for one command and quietly
the others away.  This has the advantage that if we don't have sequence
initiative for a command, we can probably find another command where we do
use that to send the status.

*********** cb response ****
One of the objectives is to eliminate the "ambiguous exchange"s FCP requires
to be ABTS'd.  In PLDA we couldn't come up with a rule to say what an
ambiguous exchange was so we said ABTS ALL open exchanges after a Target
Reset (for example).  This aspect if not the primary goal (which it to
notify the other initiators), but I consider it a fairly important secondary
goal.  During the discussions at the WG it was considered to just find an
I/O and return the Power Up UA (29/00) on it - no host change necessary.
However that approach (and your suggested approach) leaves the commands in
flight up in the air.

2.  If a target is unable to deliver the status for any reason (loop
initialization in progress, can't win arbitration in 100 ms, etc) it may
give up
trying to send it.  This would mean that if a Reset LIP occurs on one port,
target would only deliver this status to hosts on the alternate port.

In the case of a reset, should the responses be sent before or after the
occurs?  After is probably better, so the host won't try to retry the
while the target is resetting.  But that means we have to keep the command
queue, or at least a list of command identifiers per-host, alive through the
reset; and at this point we've already logged out the hosts if it's a LIP

****** cb response  *****
This area needs some more thinking, but my initial reaction is to say this
new status applies only to Task Management functions.  LIP Reset is not a
task management function so it doesn't apply - just reset as you do today.
This doesn't hurt my original goal of notifying the other initiator because
the LIP is seen by all on the local loop and all interested initiators on
remote loops via RSCN.

In the case of third-party process logout (TPRLO), we would have to defer
actually doing the logouts until we had sent the responses (can't send a
response to someone you're not logged in with).  What is now a simple loop
in a
single thread becomes a tangle of callbacks and/or context switches.  And
we have to defer sending the TPRLO response until responses were sent to
host?  (Probably).  Very messy.

*********** cb response ******
I think a good target implementation of TPRLO would be to send an explicit
PRLO to each host affected (we don't do this yet either.....).  So, the
question is on Login/Logout (or PRLI/PRLO) is the secondary goal of cleaning
up those "ambiguous" exchanges needed?  If I stick with my 'task management
functions only' rule then this new status would not apply to FC layer
login/logouts.  If I send an explicit PRLO on TPRLO the primary goal is not
applicable - any other flavors of logouts/logins are single host.  So, back
to the ambiguous exchange question.  PRLI seems to be the only ELS which
could result in questions - the ACC to PRLI being in flight simultaneous
with a new command.  All other ELSs which cancel I/Os will need to be
followed by PRLI before new commands may be accepted.  So, why is the host
sending any command after sending a PRLI but before receiving the ACC?
Shouldn't happen in well behaved host driver and in-order deliver.  What
about Out Of Order?  I'd like to ignore it :-).

We would also need to defer sending Clear Task Set or Target Reset responses
until all the 0x40 responses were sent.

************* cb response *****
Seems it would be cleaner if the rule was a target should return all I/Os
with the new status before giving Task Mgmt Response.  This gives the host a
clean spot to know when to cleanup (via ABTS) any straggler I/Os).  However,
this doesn't apply to parallel SCSI because there the "Task Mgmt Response"
is going bus free, so by definition you have I/Os coming back with 'Command
Cleared' status AFTER the "Task Mgmt Response".  To make the SCSI model
consistent it seems one should not have an additional rule for FCP.

One question which came up during the WG discussion was whether or not the
Cmd Cleared statuses needed to be returned before the Power Up UA.  One
feeling is drivers which enable the Cmd Cleared status feature via mode
select will no longer assume canceled I/Os upon receipt of Power Up UA -
rather they would rely upon the new Cmd Cleared status to trigger the I/O
retry.  With this attitude there is no ordering restrictions between the Cmd
Cleared Statuses and either the Task Mgmt Response or the Power Up UA (if

Finally, I've been going round and round myself on whether or not the Cmd
Cleared statuses should be returned to the initiator which sent the Task
Mgmt Function, or just the other initiators.  The key problem being fixed
only needs Cmd Cleared status to the others - the "initiating initiator"
knows what he did.  However, when considering the secondary goal (eliminate
the ambiguous exchanges) I lean toward using the Cmd Cleared status for all
I/Os regardless of initiator.  I also feel the all initiators approach
simplifies the logic for both the initiator and target.  On the target side,
no need to decide which initiator gets it, if the I/O is cleared, send the
new status.  On the host side, you can assume all I/Os will have some status
or need an ABTS upon timeout.

* For T10 Reflector information, send a message with
* 'info t10' (no quotes) in the message body to majordomo at

More information about the T10 mailing list