TASK SET FULL Definition(s) in FC-TAPE and SAM-2

George M. Ericson GEricson at BAnet.net
Fri Mar 19 21:07:51 PST 1999

* From the T10 Reflector (t10 at symbios.com), posted by:
* "George M. Ericson" <GEricson at BAnet.net>

Excellent summary of the real world.

In our FC experience at CLARiiON, the multi-initiator case you site is a
real problem with nasty consequences for the device driver that does not
provide some hysteresis around queue full and busy.  My recommendation fo=
device driver writers is to delay for a short time (1-5  seconds should d=
before retrying a busy.  (Note that for FC systems, immediate retries to =
really busy device will generate on the order of 15,000 to 30,000 interru=
per second, more than enough to bring many servers to their knees.)  For
queue full, treating the indication as an indication of the available que=
size is a fine short term policy decision.  However, not good at all for
medium/long term policy.  Within a relatively short time, (probably 10's =
seconds), the device driver should ratchet the assumed queue size back up.
Unless told by some other mechanism, the device driver should not assume =
hard upper bound for queue size based on a queue full indication.

George Ericson
CLARiiON Advanced Storage Division, Data General
GEricson at CLARiiON.com

-----Original Message-----
From: owner-t10 at Symbios.COM [mailto:owner-t10 at Symbios.COM]On Behalf Of
Gerard Roudier
Sent: Wednesday, March 17, 1999 4:15 PM
To: gop at us.ibm.com
Cc: t10 at Symbios.COM
Subject: Re: TASK SET FULL Definition(s) in FC-TAPE and SAM-2

* From the T10 Reflector (t10 at symbios.com), posted by:
* Gerard Roudier <groudier at club-internet.fr>

On Wed, 17 Mar 1999 gop at us.ibm.com wrote:

> * From the T10 Reflector (t10 at symbios.com), posted by:
> * gop at us.ibm.com
> *
> Joe,
> The is a significant difference on how an initiator is supposed to resp=
> to a BUSY vs a TASK SET FULL status.
> A BUSY only tells the initiator that the command could not be received =
> the target and the initiator should try again later. There is no
> as to how much later from the target.
> A TASK SET FULL also tells the initiator that the command could not be
> received by the target and that in initiator should not try again until=
> receives a command complete indication from a currently outstanding
> command. So in the case of a TASK SET FULL there is an indication from =
> target as to when the initiator has a chance of having the command
> accepted.

The TASK SET FULL only tells the initiator that the device decided that i=
has no resources enough to accept the command. It only informs the
initiator on the fact that the cause has been stated to be a lack or
resources by the device. It is initiator responsibility not to behave
stupidly in such a situation, and in real life it is not that easy to
implement good heuristics for TASK SET FULL conditions that deal
gracefully with all existing device firmware implementations.

Indeed, it the device has outstanding commands, the behaviour you suggest
in reasonnable, but if it hasn't any, initiator still must recover from
the situation. This has happened with some early drives, at least, likely
when write caching was enabled, because commands are completed as soon as
data is copied into the cache and the device can base the lack of
resources condition on cache usage, or just share the cache memory for
data and command queues. This may also happen in multi-initiator
environment, regardless the way the target handles its resources. Such
behaviours are quite compliant with the specifications.

The behaviour of device firmwares regarding TASK SET FULL condition may b=
basically of the following patterns:

1) Devices that implement a fixed command queue depth. We can find
   middle-range hard disks that support a fixed value of 63 or 64
   commands, for example.
   Such devices return TASK SET FULL if you try to queue more commands.
   No need to say, that it is easy to handle from the initiator.

2) Devices that returns TASK SET FULL on some complex condition on
   the amount of resource used. For that devices, the device may return
   TASK SET FULL with a variable number of outstanding commands,
   especially when write caching is enabled. These devices require to
   be a bit more clever for the handling of TASK SET FULL conditions
   for performances not to be affected. Numerous high-end hard disks
   behave so.

And, btw, if several initiators are queuing commands to the same target,
obviously, a TASK SET FULL can be returned by this target to an initiator
that haven't any command currently queued to this target.

> This difference is important because the way most initiators respond to
> BUSY is to immediately resend the command which has the effect of clogg=
> up the SCSI bus with needless activity. Where as TASK SET FULL causes t=
> initiators to hold back resending the command thus reducing traffic on =
> SCSI bus.

A device that has outstanding commands from a initiator and returns BUSY
status when it is queued with a new command by this initiator is probably
rare. Anyway, requeuing the command immediately when a device returns BUS=
is probably not the way to go, whatever outstanding commands from the
initiator exist or not. My opinion is that the BUSY status is not a
condition that must be assumed to last a very short time in average and s=
requeuing it immediately is probably close to the worst thing to do. But
since BUSY condition does not happen so often and is generally a not to
long transient condition, such a behaviour from initiators is probably
harmless in real life. Perhaps, BUSY should be handled the same way as
TASK SET FULL if it happens while the target has outstanding commands fro=
the initiator and the requeue of the command should be delayed of a short
delay (tens of milliseconds) if no outstanding commands currently exist
|from that initiator.

> Bye for now,
> George Penokie
> Dept PPV  114-2 N212
> E-Mail:    gop at us.ibm.com
> Internal:  553-5208
> External: 507-253-5208   FAX: 507-253-2880



* For T10 Reflector information, send a message with
* 'info t10' (no quotes) in the message body to majordomo at symbios.com

* For T10 Reflector information, send a message with
* 'info t10' (no quotes) in the message body to majordomo at symbios.com

More information about the T10 mailing list