Date: Aug 14, 1991 X3T9.2/91-127 Rev 0 To: X3T9.2 Committee (SCSI) From: George Penokie (IBM) Subject: Tagged Queuing Implementation Example Proposal: Command Queuing An Initiator may use either Tagged or Untagged Queuing but cannot use both at the same time. Both Tagged and Untagged Queuing can coexist on the same Target at the same time, provided they are associated with different Initiators. The following commands shall not be queued. - Request Sense command - Inquiry command - Commands linked to previous commands - Commands for which disconnection is not allowed - Commands in which a SCSI bus error occurred between selection and first disconnection following the receipt of the Command Descriptor Block. - Commands for an invalid Logical Unit Number. - Commands which cause an incorrect initiator connection. In the above listed situations, the command queue is bypassed, the received command becomes an active I/O process on receipt of that command, and any queued I/O processes are preserved on the command queue. Queued commands are removed from the command queue and executed when the addressed Logical Unit determines it is able to start an active I/O process. Commands may be removed from the command queue without executing for any of the following reasons: -All I/O processes for a given initiator are terminated by an ABORT message. -All I/O processes for a given initiator are terminated by an incorrect initiator connection. -All I/O processes from all initiators are terminated by a SCSI BUS RESET or a BUS DEVICE RESET message. -All I/O processes from all initiators are terminated by a CLEAR QUEUE message. -An I/O process for a given initiator is terminated be an ABORT TAG message. -If the QErr bit of the Control Mode page is set to 1 then all I/O processes for all initiators are terminated after the contingent allegiance or extended contingent allegiance condition has been cleared. Once a command is removed from the command queue either for one of the above reasons or because it has began execution it shall only be placed back on the command queue by an initiator reissuing the command. Untagged Queuing Untagged queuing allows a target to accept a command from an initiator for a logical unit or target routine while I/O processes from other initiators are being executed. Only one command for each I_T_x nexus shall be accepted at a time. An I/O process may be initiated any time the BUS FREE phase exists even if an I/O process from a different initiator is active. If the disconnect privilege is not granted in the IDENTIFY message of the current I/O process, the target may either suspend all other I/O processes or it may return BUSY status to the current I/O process. The I_T_x nexus sufficiently specifies the relationship so that the target can reconnect to the initiator to restore the pointers for I/O process as long as only one I/O process per I_T_x nexus is issued. It is the responsibility of the initiator to assure that only one such I/O process is issued at any time. If the initiator violates this rule an incorrect initiator connection shall occur. There is no guarantee that the I/O processes are executed in the order they were received in a multiple initiator environment when Untagged Queuing is being used. Reserve and Release commands may be used to temporarily prevent other initiators from establishing an I_T_x nexus if any restrictions apply to the order of command processing in a multiple initiator environment. The Queue Algorithm Modifier parameter value of zero (restricted re-ordering) in the Control Mode page, only applies to commands from the same initiator when tagged queuing is enabled. Tagged Queuing Tagged queuing allows a target to accept multiple I/O processes from the same or different initiators until the logical unit's command queue if full. If an I/O process is received and the command queue is full, the target shall terminate the I/O Process with QUEUE FULL status. Tagged queuing may be enabled and disabled using the DQue bit in Mode Select page 0Ah. The command queue is setup by the target for each supported logical unit and target routine. Initiators may add or delete I/O processes to the queue. When adding an I/O process, the initiator may specify fixed order of execution, allow the target to define the order of execution, or specify that the I/O process is to be executed next. A target with one or more I/O Processes outstanding from a given initiator may reconnect to that initiator to continue any of the I/O Processes. This may occur regardless of any options which limit the order of command execution (i.e., reordering algorithm, HEAD OF QUEUE, ORDERED, or SIMPLE queue tags). Linked commands shall be considered a single I/O process, and are all assigned the queue tag which was established in the initial connection. Tagged queuing may also be temporarily disabled internal to the SCSI device during certain initialization periods or to control internal resource utilization. During these periods the target shall return QUEUE FULL status. A device that does not support tagged queuing for any reason (e.g., not implemented, disabled by the DQue bit in the control mode page, or it has been switched to an operating definition that does not include tagged queuing) shall respond to any queue tag message with a MESSAGE REJECT message. The target shall continue the I/O process as if it was an untagged I/O process. Tagged Queuing Messages The queue tag messages allow the initiator to establish an unique I_T_L_Q nexus to identify each I/O process. The I_T_L_Q nexus allows the target to reconnect to a specific I/O process and allows the initiator to restore the set of pointers for that I/O process. If only SIMPLE QUEUE TAG messages are used, the target may execute the commands in any order that is deemed desirable and has no restrictions on intermixing commands from different initiators on a multi-initiator system within the constraints of the queue management algorithm specified in the control mode page. If ORDERED QUEUE TAG messages are used, the target shall execute the commands in the order received with respect to other commands received with ORDERED QUEUE TAG messages regardless of which initiator sends the commands. All commands received with a SIMPLE QUEUE TAG message prior to a command received with an ORDERED QUEUE TAG message, regardless of initiator, shall be executed before that command with the ORDERED QUEUE TAG message. All commands received with an SIMPLE QUEUE TAG message after a command received with an ORDERED QUEUE TAG message, regardless of initiator, shall be executed after that command with the ORDERED QUEUE TAG message. A command received with a HEAD OF QUEUE TAG message is placed first on the command queue, to be executed after all active I/O processes have completed execution. A command received with a HEAD OF QUEUE TAG message shall not suspend an I/O process for which the target has begun execution. Consecutive commands received with HEAD OF QUEUE TAG messages are executed in a last-in-first-out order. A command received with a HEAD OF QUEUE TAG message shall not suspend a series of linked commands for which the target has begun execution. The ABORT, ABORT TAG, BUS DEVICE RESET, and CLEAR QUEUE messages may be used to clear all or part of the command queue. Tagged Queuing Message Error Handling If the disconnect privilege is not granted in the IDENTIFY message for a tagged I/O process the target shall return BUSY status. An I/O process received without a queue tag message while there are any tagged I/O commands in the command queue from the same initiator is an incorrect initiator connection unless there is a contingent allegiance or extended contingent allegiance condition. Tagged Queuing Command Exceptions When a Format command is placed on the queue the target shall respond to all new valid tagged commands, regardless of initiator, with a QUEUE FULL status and all new valid untagged commands, regardless of initiator, with a BUSY status. Inquiry and Request Sense shall execute as defined below even when a Format command is in the queue. Note: A valid command is one which has no error conditions which would have prevented it from being placed on the queue if there wasn't a Format command in the queue. (e.g. incorrect initiator connection, illegal request, etc.) Note: There is a possibility some commands could be placed on the queue after the Format command is received, before the target can prevent this. These commands may be left on the Queue until the Format is completed and then executed normally. When the Format command begins executing the target shall respond to all new valid commands with CHECK CONDITION status, regardless of the Immed bits value. And, unless an error has occurred, the target shall return a sense key of NOT READY and an additional sense code of LOGICAL UNIT NOT READY FORMAT IN PROGRESS, with the sense key specific bytes set for progress indication. The Inquiry command when sent with a tagged message, regardless of which tagged message is sent, it shall execute as if the tagged message was a HEAD OF QUEUE TAG message. When a Reserve command is sent the initiator shall wait until GOOD status is returned before placing any further commands on the queue. If the initiator does not follow this rule unexpected results may occur. When a tagged Request Sense command is sent and there is no contingent allegiance or extended contingent allegiance condition, regardless of which tagged message is sent, the command shall execute as if the tagged message was a HEAD OF QUEUE TAG message. See Tagged Queuing Error Conditions for how queueing targets handle the Request Sense command during a contingent allegiance or extended contingent allegiance condition. When a Start/Stop Unit command is placed on the queue the target shall respond to all new valid tagged commands, regardless of initiator, with a QUEUE FULL status and all new valid untagged commands, regardless of initiator, with a BUSY status. Inquiry and Request Sense shall execute as defined above even after the Start/Stop Unit command is in the queue. Any additional Start/Stop Unit commands shall be placed on the queue and executed in turn. Note: A valid command is one which has no error conditions which would have prevented it from being placed on the queue if there wasn't a Start/Stop Unit command in the queue. (e.g. incorrect initiator connection, illegal request, etc.) Note: There is a possibility some commands could be placed on the queue after the Start/Stop Unit command is received, before the target can prevent this. These commands may be left on the Queue until the Format is completed and then executed normally. Tagged Queuing Error Conditions When a tagged or untagged Request Sense command is sent and there is a a contingent allegiance or extended contingent allegiance condition the target shall return the sense data to the initiator and clear the contingent allegiance. The extended contingent allegiance shall be handled as described in the extended contingent allegiance section. A tagged or untagged Inquiry command shall not clear the contingent allegiance. Any other tagged or untagged command for the I_T_x_y nexus shall cause the contingent allegiance to be cleared. The command that clears the contingent allegiance shall be executed unless execution would cause an incorrect initiator connection. See below for more details. There are two error recovery options allowed. The error recovery option to be used is specified in the control mode page queue error management (QErr) bit. First Post-recovery Option The first post-recovery option (QErr bit set to 0) is to continue execution of commands in the queue after the contingent allegiance or extended contingent allegiance condition has cleared. If the contingent allegiance condition is cleared by an untagged command, other than a Request Sense command, and there are tagged commands in the queue for the initiator which had the contingent allegiance condition then an incorrect initiator connection occurs. If the contingent allegiance condition is cleared by a tagged command, other than a Request Sense command, the target shall place that command on the queue and continue execution. While a contingent allegiance or extended contingent allegiance condition exists only the I/O processes in the queue for that initiator which has the contingent allegiance or extended contingent allegiance condition shall be suspended and the target may accept I/O processes to the queue from other initiators. If commands are combined by the queuing algorithm such that the error affects more than one command, then an contingent allegiance condition shall be generated for all affected commands. Second Post-recovery Option The second post-recovery option (QErr bit set to 1) aborts all I/O processes after the contingent allegiance or extended contingent allegiance condition has been cleared. A unit attention condition shall be generated for all other initiators and the additional sense code shall be set to COMMANDS CLEARED BY ANOTHER INITIATOR. If the contingent allegiance condition is cleared by an untagged or tagged command, other than a Request Sense command, the target shall place that command on the queue and continue execution. Deferred Errors Deferred errors are normally related to a command that has already completed. As such, there is not attempt to return the queue tag value assigned to the original command.