X3T9.2/87-59 REV 4 Memo to: ANSI X3T9.2 Memo from: Committee on Command Queueing James McGrath Jeff Stai Quantum Western Digital 1804 McCarthy Blvd 2445 McCabe Way Milpitas, CA 95035 Irvine, CA 92714 Robert N. Snively I. Dal Allan Adaptec ENDL 580 Cottonwood 14426 Black Walnut Ct. Milpitas, CA 95035 Saratoga, CA 95070 Paul Nitza Emulex 3545 Harbor Blvd Costa Mesa, CA 92626 Date: June 16, 1987 Revised: August 13, 1987 Revised: October 1, 1987 Final draft of proposal Subject: Command Queueing Proposal for SCSI-2 (Rev 4.0) The SCSI architecturally supports very complex multi-tasking environments. The architecture already supports the queueing of commands from different initiators against a single SCSI device and LU. This is referred to in this proposal as untagged queueing. With minor additions, the SCSI-2 is also capable of supporting the queueing of commands from the same initiator against a single SCSI device and LU. This is referred to in this proposal as tagged queueing. Note that tagged queueing is a superset of untagged queueing. This document defines the extensions that are required to provide both forms of queueing. It is a final draft of the working paper of the committee on Command Queueing, including input from two working group sessions. Revision 4 of this document, dated October 1, 1987, includes the following changes with respect to revision 2, dated June 16, 1987: a) Consecutive Head of Queue tagged operations are managed in a LIFO manner. b) The tagging message immediately follows the ID message. c) The queueing structure locks up until an error has been serviced by a REQUEST SENSE or other command. d) A mode of operation providing data integrity protection is provided. e) A Queue Control page is provided in the MODE SELECT command. f) Modifications are made in include the changes required to support Extended Contingent Allegiance and Autosense. g) Modifications are made to correct the definition of the synchronous data transfer negotiation protocol. h) Modifications are made to correct the definition of soft RESET. INTRODUCTION This section describes the concept intended to be implemented by the attached text revisions. Untagged queueing is managed the same way as in the SCSI X3.131-1986. All commands are untagged and no overlapped commands for a given LU from a given initiator are allowed. Tagged queueing is a far more powerful concept. An identified command may be queued according to the specified rules from any initiator to any LU. Conceptually, each command transmitted to a target is transmitted either with an unordered queue tag, with an ordered queue tag, with a head of queue tag, or without a queue tag. If the command is transmitted to the target without a tag, it indicates that the command must be executed in the order received from the transmitting initiator. No time relationship between the execution of untagged commands from other initiators is guaranteed except through use of the reservation instructions. Only one untagged command can be pending from a particular initiator for each LU. If the command is transmitted to the target with an unordered queue tag, the command can be executed in any order on the selected LU with respect to other unordered tagged commands from any initiator. The order of actual command execution is under control of the target. The ordering is selected to optimize various parameters, including response time, throughput, and data integrity. If the command is transmitted to the target with an ordered queue tag, the command must be placed in the queue after all unordered queue tagged commands previously received from any initiator and before any unordered queue tagged commands received later. Ordered queue tagged commands are executed in the strict order received, regardless of initiator. In a stream of tagged commands, the order of execution of an untagged command with respect to various types of tagged commands is not defined. Head of queue tagged commands are placed in the queue for execution before all other commands. Any commands that may have been taken from the queue and for which execution has been started will not be preempted by head of queue tagged commands. Sequential head of queue tagged commands are executed in LIFO order. The use of the UNORDERED QUEUE TAG implies that the command or link of commands may be queued by the target SCSI device and executed against the LU in a target selected order. The transmission of a command with the ORDERED QUEUE TAG message implies that that tagged ordered command or link of commands must be executed in the strict order received by that LU regardless of the initiator. The use of the ORDERED QUEUE TAG implies that the command or link of commands must be queued by the target SCSI device and executed against the LU in the strict order received. All previous UNORDERED QUEUE TAG labeled commands or untagged commands must be executed before the command labeled by the ORDERED QUEUE TAG. All subsequent UNORDERED QUEUE TAG labeled commands or untagged commands must be executed after the command labeled by the ORDERED QUEUE TAG, regardless of initiator. The use of the HEAD OF QUEUE TAG implies that the command must be queued by the target SCSI device and placed first in the queue for execution immediately after termination of any command that may be active. No active command is interrupted or terminated by a HEAD OF QUEUE TAG tagged command. Consecutive head of queue tagged commands are executed in LIFO order. Only one command with a given tag value can be active against an LU from an initiator at a time. The number of queue tag bits assures that an unused tag can easily be found. If the command queue is full, an indication must be given to an initiator that no more commands can be enqueued until one or more of the already queued commands is complete. Execution of the command is halted and must be reinitiated by the initiator. The QUEUE FULL status is presented to the first command transferred that cannot be queued. The command will neither be queued nor executed. The initiator must reinitiate the command at a later time. If the command queue is full and an untagged command is received, BUSY status is presented and the command is not executed or queued. The initiator must reinitiate the command at a later time. The command queue tagging protocol is chosen to be compatible with SCSI X3.131-1986 as well as functional with the SCSI-2 standard. The command queue tagging protocol is further chosen to allow disconnection any time after the IDENTIFY message and the QUEUE TAG message have been transmitted. Error handling is controlled by the state of the newly defined Queue Control page MODE SELECT parameters. The recovery parameters allow one of two options to be selected. The first is to continue execution of commands in their normal queued sequence after the error information has been presented to the host via REQUEST SENSE, after the error information has been cleared from the target by the transmission of another command, or after Extended Contingent Allegiance (ECA), if any, has been cleared. During the Contingent Allegiance or Extended Contingent Allegiance state, the queue for that LUN is frozen and any command transmitted to the target by another initiator will receive BUSY status. As soon as the REQUEST SENSE or clearing command has been executed or the ECA state has been terminated, normal queueing operation resumes. If commands are combined by the queuing algorithm such that more than one command's execution is affected by the error, then a CHECK CONDITION shall be posted for all affected commands. The second recovery action clears the entire remaining queue if an error occurs during the execution of any command. When CHECK CONDITION status is presented the queue is frozen until either a REQUEST SENSE has been executed, another command has been transmitted from the affected initiator, or the ECA state has been terminated. During the Contingent Allegiance or ECA state, any command transmitted to the target by another initiator will receive BUSY status. No new queued commands will begin execution. As soon as the Contingent Allegiance or ECA state has been terminated, the entire queue for all initiators to that LUN will be cleared. A Unit Attention condition is posted for all initiators which had commands currently queued or in execution against that LU. Extended Contingent Allegiance (ECA) provides the ability for an initiator to make adjustments in the content of the queue if that is required to enhance error recovery. Other recovery action steps can also be taken with the assurance that no activities dispatched from the queue or received from other initiators will interfere with the recovery actions. Deferred errors are normally related to a command that has long since completed. In that case, there is no attempt to point back to the queue tag assigned to the original failing command, since the original queue tag may well have been reassigned by the time the deferred error is identified. System level recovery, including check-pointing, periodic synchronization, and journaling are required to properly guarantee system integrity in the presence of deferred errors. The tagging message for a queued command is transmitted immediately after the IDENTIFY message and before the transfer of the command bytes. The tagging messages, if used, must preceed any other outbound control messages, including Synchronous Data Transfer negotiation, Wide Transfer negotiation, and any others. Data integrity is ultimately a system responsibility. However, the system can be assisted by standardized limitations on the reordering of commands permitted the target. Two modes of operation are defined for a target. The default mode performs absolute verification that the contents of data on the storage medium will be exactly the same after completion of a set of queued commands in the optimized order as would the medium after completion of the same commands executed in the received order. This mode requires the target to prevent queue re-ordering that allows data to be read before it has been written. A second mode is defined that makes no guarantees with respect to the order of execution of the queued commands. Command ordering will optimize the order in a target dependent manner, making use only of the ordering process commanded by the queue tag messages. In this case, the host operating system must guarantee that no command sequence is generated that will threaten data integrity. The target will be free to maximize performance in all other ways. The proposal is added to the end of section 6 of the SCSI-2 document. The section will include both a description of untagged command queueing and tagged command queueing. In addition to the inclusion of section 6, section 5 will pick up new messages in table 5-4 and in section 5.5. Also, section 6.3 and table 6-7 will be altered to reflect the new status byte code. MODE SELECT and MODE SENSE are also modified. The modifications are given for the DAS and Sequential Access device types, but must also be included for other device types as pages are defined for their MODE SELECT and MODE SENSE commands. To add the CLEAR QUEUE message and the extra two byte messages required to implement this proposal, Table 5-2 is replaced with the following table: Table 5-2: Message Codes ============================================================================== Code Support Description Direction Init Targ ------------------------------------------------------------------------------ 00h M M COMMAND COMPLETE In 01h O O EXTENDED MESSAGE (MULTI-BYTE) In Out 02h M O SAVE DATA POINTER In 03h M O RESTORE POINTERS In 04h M O DISCONNECT In 04h O O DISCONNECT Out 05h M M INITIATOR DETECTED ERROR Out 06h O M ABORT Out 07h M M MESSAGE REJECT In Out 08h M M NO OPERATION Out 09h M M MESSAGE PARITY ERROR Out 0Ah O O LINKED COMMAND COMPLETE In 0Bh O O LINKED COMMAND COMPLETE (WITH FLAG) In 0Ch O M BUS DEVICE RESET Out 0Dh O O AUTOSENSE DATA FOLLOWS Out 0Eh O O CLEAR QUEUE Out 0Fh Reserved Code 10h O O INITIATE RECOVERY In Out 11h O O RELEASE RECOVERY Out 12h - 1Fh Reserved Codes 20h O O UNORDERED QUEUE TAG (TWO BYTES) In Out 21h O O HEAD OF QUEUE TAG (TWO BYTES) In Out 22h O O ORDERED QUEUE TAG (TWO BYTES) In Out 23h O O ABORT TAG (TWO BYTES) Out 24h - 7Fh Reserved Codes 80h - FFh M O IDENTIFY In 80h - FFh M M IDENTIFY Out ============================================================================== Key: M = Mandatory support, O = Optional support. In = Target to initiator, Out = Initiator to target. Replace paragraph four ("ABORT 06h. This ... for the initiator.") of page 5-17 in section 5.5.2 with the following text. Note that the ECA text proposal is included in this paragraph. ABORT 06h. This message is sent from the initiator to the target to clear the present operation and all queued operations for this initiator and logical unit. If a logical unit has been identified, all pending data and status and all commands queued for this initiator and logical unit shall be cleared, and the target shall go to the BUS FREE phase. Pending data, status, and queued operations for other initiators shall not be cleared. If a logical unit has not been identified, the target shall go to the BUS FREE phase. No status or pending message shall be sent for the operation and the queue shall not be affected. It is not an error to issue this message to a logical unit that is not currently performing an operation for the initiator or has no operations queued from this initiator. Transmission of this message shall terminate any ECA state that may exist between the selected LU and the selecting initiator. Insert the following text between paragraph seven ("... the BUS FREE phase.") and paragraph eight ("IDENTIFY 80h to FFh...") of page 5-18 in section 5.5.2, renumbering tables 5-3 through 5-8 to be 5-7 through 5-12: CLEAR QUEUE 0Eh. The CLEAR QUEUE message shall be implemented if tagged queueing is implemented. The message may be sent by the initiator. When this message is received the target will perform an action equivalent to receiving a series of ABORT TAG messages for all commands currently queued against the identified LU from all initiators. All commands in the queue are cleared from the queue. All executing commands are immediately halted. The media may have been modified by partially executed commands. All pending status and/or data for that LUN for all commands is cleared. No status or ending message will be sent for any of the outstanding commands. If the target is currently connected for the command specified in the tag value, the target will go to BUS FREE phase. A Unit Attention condition is raised for all other initiators with commands that either had been executing or were queued for that LU. The Unit Attention condition is received by each initiator upon transmission of the next command to that LU from that initiator. Note that previously established conditions, including MODE SELECT parameters, RESERVE/RELEASE conditions, and ECA state are not changed by the CLEAR QUEUE command. UNORDERED QUEUE TAG 20h. Table 5-3: UNORDERED QUEUE TAG ================================================================= Byte | Value | Description ----------------------------------------------------------------- 0 20h UNORDERED QUEUE TAG 1 x Tag value ================================================================= The UNORDERED QUEUE TAG message (Table 5-3) shall be implemented if tagged queuing is implemented. It is transmitted by both the target and the initiator. It is transmitted immediately after the IDENTIFY message to provide a tag for that particular command or link of commands during the entire period that the command is queued or being executed. The tag consists of the quadruple of the Initiator, Target, LUN, and Tag Value. A command tagged by the UNORDERED QUEUE TAG message will be enqueued for execution by the target in a target unique manner. The UNORDERED QUEUE TAG message out phase is requested by the initiator immediately after the IDENTIFY message by leaving ATN active after the time that ACK drops in response to the REQ for the IDENTIFY message. Target devices supporting tagged queueing will not disconnect until after both the IDENTIFY message out and the UNORDERED QUEUE TAG message, if any, out have been received. The UNORDERED QUEUE TAG message in is provided immediately after the reselection IDENTIFY message in has been presented to complete the identification process of the operation being conti- nued. If no type of tagging message in is provided, it is as- sumed that the reconnection is being made on behalf of an active untagged command. If a SCSI device receives an UNORDERED QUEUE TAG message and does not support tagged queueing, the device will respond with a MESSAGE REJECT message and complete the command as if it were an untagged command. If a target receives a message with a Tag Value that is currently being used for that Initiator/LU, then it will respond as if an untagged overlapped command had occurred. Both the original command and the overlapping command will be aborted, according to the rules for the ABORT TAG message, regardless of whether the original command is enqueued for future execution or is actively being executed. (See section 6.5.1) If an initiator receives a message with a Tag Value that is not currently being used for that Target/LU combination, then it will respond with an ABORT message, clearing all activity for that initiator and LUN. HEAD OF QUEUE TAG 21h. Table 5-4: HEAD OF QUEUE TAG ================================================================= Byte | Value | Description ----------------------------------------------------------------- 0 21h HEAD OF QUEUE TAG 1 x Tag Value ================================================================= The HEAD OF QUEUE TAG message (Table 5-4) shall be implemented if tagged queuing is implemented. It is sent by both the target and the initiator. It is transmitted immediately after the IDENTIFY message to provide a tag for that particular command or link of commands as it is being queued or executed by the initiator and the LU. The HEAD OF QUEUE TAG message requests that the tagged command be placed first in the queue for execution. The HEAD OF QUEUE tagged command will not preempt commands already dispatched from the queue, but will be executed next after a currently executing command. Subsequent HEAD OF QUEUE tagged commands will be inserted at the head of the queue for execution in LIFO order, except where a previous HEAD OF QUEUE tagged command has already been dispatched from the queue for execution. The Tag value is used in conjunction with the Initiator, Target, and LU number to generate a quadruple that uniquely identifies the associated command. The HEAD OF QUEUE TAG message out phase is requested by the initiator immediately after the IDENTIFY message by leaving ATN active after the time that ACK drops in response to the REQ for the IDENTIFY message. Target devices supporting tagged queueing will not disconnect until after both the IDENTIFY and the HEAD OF QUEUE TAG messages out have been received. The HEAD OF QUEUE TAG message in is provided immediately after the reselection IDENTIFY message in has been presented to complete the identification process of the operation being continued. If no type of tagging message in is provided, it is assumed that the reconnection is being made on behalf of an active untagged command. If a SCSI device receives a HEAD OF QUEUE TAG message and does not support it, the device will respond with a MESSAGE REJECT message and complete the command as if it were an untagged command. If a target receives a message with a Tag Value that is currently being used for that Initiator/LU, then it will respond as if an untagged overlapped command had occurred. Both the original command and the overlapping command will be aborted, according to the rules for the ABORT TAG message, regardless of whether the original command is enqueued for future execution or is actively being executed. (See section 6.5.1) If an initiator receives a message with a Tag Value that is not currently being used for that Target/LU combination, then it will respond with an ABORT message, clearing all activity related to that initiator and LUN. ORDERED QUEUE TAG 22h Table 5-5: ORDERED QUEUE TAG ================================================================= Byte | Value | Description ----------------------------------------------------------------- 0 22h ORDERED QUEUE TAG 1 x Tag value ================================================================= The ORDERED QUEUE TAG message (Table 5-5) shall be implemented if tagged queueing is implemented. The message is sent by both the target and the initiator. It is transmitted immediately after the IDENTIFY message to provide a logical identifier for that particular command or link of commands as it is being executed by the initiator and the LU. The ORDERED QUEUE TAG message requests that the labeled command be placed in the queue for execution in the strict order received. All earlier commands of any tag type except HEAD OF QUEUE tagged commands are executed before the ORDERED QUEUE TAG labeled command while all commands transmitted later must be executed after the ORDERED QUEUE TAG labeled command. The Tag value is used in conjunction with the Initiator, Target, and LU number to generate a quadruple that uniquely identifies the associated command. The ORDERED QUEUE TAG message out phase is requested by the initiator immediately after the IDENTIFY message by leaving ATN active after the time that ACK drops in response to the REQ for the IDENTIFY message. Target devices supporting tagged queueing will not disconnect until after both the IDENTIFY and the ORDERED QUEUE TAG messages out have been received. The ORDERED QUEUE TAG message in is provided immediately after the reselection IDENTIFY message in has been presented to complete the identification process of the operation being continued. If no type of tagging message in is provided, it is assumed that the reconnection is being made on behalf of an active untagged command. If a SCSI device receives a ORDERED QUEUE TAG message and does not support it, the device will respond with a MESSAGE REJECT message and complete the command as if it were an untagged command. If a target receives a message with a Tag Value that is currently being used for that Initiator/LU, then it will respond as if an untagged overlapped command had occurred. Both the original command and the overlapping command will be aborted, according to the rules for the ABORT TAG message, regardless of whether the original command is enqueued for future execution or is actively being executed. (See section 6.5.1) If an initiator receives a message with a Tag Value that is not currently being used for that Target/LU combination, then it will respond with an ABORT message, clearing all activity for that LUN and initiator. ABORT TAG 23h. Table 5-6: ABORT TAG ================================================================= Byte | Value | Description ----------------------------------------------------------------- 0 23h ABORT TAG 1 x Tag value ================================================================= The ABORT TAG message (Table 5-6) shall be implemented if tagged queueing is implemented. The message may be sent by the initiator. When this message is received, the target will dequeue the command identified by the tag value. If the target has already started execution of the command, the execution will be halted. The media contents may have been modified before the execution was halted. In either case, any pending status and/or data for that command is cleared. No status or ending message will be sent for the identified command. Pending status, data, and commands for other queued or executing operations will not be affected. If the target is currently connected for the command specified in the tag value, the target will go to BUS FREE phase. Execution of other commands queued against the specified LU will continue in the normal manner. Replace Table 6-6 on page 6-8 in section 6.3 with the following: Table 6-6: Status Byte ============================================================================== Bit| 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0 | Byte | | | | | | | | | ============================================================================== 0 | Reserved | Status Byte Code |Reserved| ============================================================================== Replace Table 6-7 on page 6-9 in section 6.3 with the following: Table 6-7: Status Byte Code Bit Values ============================================================================== Bits of Status Byte ----------------------------- 7 6 5 4 3 2 1 0 Status(es) Represented ------------------------------------------------------------------------------ R R 0 0 0 0 0 R GOOD R R 0 0 0 0 1 R CHECK CONDITION R R 0 0 0 1 0 R CONDITION MET/GOOD R R 0 0 1 0 0 R BUSY R R 0 1 0 0 0 R INTERMEDIATE/GOOD R R 0 1 0 1 0 R INTERMEDIATE/CONDITION MET/GOOD R R 0 1 1 0 0 R RESERVATION CONFLICT R R 1 0 1 0 0 R QUEUE FULL All other status codes are reserved. ============================================================================== Key: R - Reserved bit (= 0) Insert after paragraph one ("... at a later time.") on page 6-10 the following: QUEUE FULL. The QUEUE FULL status shall be implemented if and only if tagged queueing is implemented. It is transmitted when an UNORDERED QUEUE TAG, ORDERED QUEUE TAG, or HEAD OF QUEUE TAG message is received from the initiator and when no commands can be accepted from that initiator for that LUN. This will usually be due to the lack of available memory in the target to hold appropriate queue and target state information. The QUEUE FULL status will also be transmitted if an untagged command is attempted but all available memory has been filled by queued command activity. The command is not executed. The command must be started again by the initiator at a later time. The following text should be added to the definition of MODE SELECT for all device types that support page format MODE SELECT. The text should be added after the last line of page 8-45 for Direct-Access Devices, immediately before table 9-13 on page 9-19 for Sequential-Access Devices, and after the last line of page 15-15 for Optical Memory Devices. When other devices support paged mode select, they too should have this page supported. Table ?-?? Queue Control ====================================================================== Bit| 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0 | Byte | | | | | | | | | ====================================================================== 0 |Resrvd |Resrvd | Page Code (0Ah) | _____|_______|_______|_______________________________________________| 1 | Parameter Length (04h) | _____|_______________________________________________________________| 2 | Reserved | _____|_______________________________________________________________| 3 | Queue Algorithm Modifier | Reserved | QERR | EQ | _____|_______________________________|________________|______|_______| 4 | (MSB) Maximum Queue Depth | _____|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _| 5 | Maximum Queue Depth (LSB) | | | ====================================================================== This page allows the tagged queueing protocols to be modified and allows the enabling and disabling of the tagged queueing function. The Queue Algorithm Modifer bits allow the initiator to restrict and modify the algorithm allowed for enqueueing UNORDERED QUEUE TAG labeled commands. The following values are defined. Value Definition 0h Guaranteed Data Integrity 1h Unrestricted Re-ordering Allowed 2h - 7h Reserved 8h - Fh Vendor Unique A value of 0h in the Queue Algorithm Modifier bits requires the target to order the actual execution sequence of the queued commands such that data integrity is guaranteed at any time. This requires that, if the transmission of new commands was halted at any time, the final value of all data observable in the target must have exactly the same value it would have if the commands had been executed in the same received sequence without queueing. The guaranteed data integrity value of the Queue Algorithm Modifier bits is the default value. A value of 1h in the Queue Algorithm Modifier bits allows the target to order the actual execution sequence of the queued commands in any manner it selects. Any data integrity exposures related to command sequence order are explicitly handled by the host operating systems through the selection of the appropriate commands and the tagging messages. Values are made available for vendor unique modifications of the Queueing Algorithm. Values are reserved for future standard queueing algorithms. The Queue Error Management (QERR) control bit indicates how the queue will be treated after the presentation of CHECK CONDITION status and after the resulting Contingent Allegiance or ECA state has terminated. If the QERR bit is set to zero, those commands still waiting on the queue will begin being dispatched in a normal manner as soon as the Contingent Allegiance or ECA state has terminated. This is the default state of the control bit. If the QERR bit is set to one, those commands waiting on the queue will be aborted as soon as the Contingent Allegiance or ECA state has terminated. A UNIT ATTENTION error will be posted for each initiator, other than the one detecting the original error, which had commands waiting on the queue. The Enable Queueing (EQ) control bit enables or disables queueing in the target. If the EQ bit is set to zero, queueing will not be executed. Any information contained in the queue for that LU and initiator will be lost. If the EQ bit is set to one, queueing will be enabled. When queueing is enabled, all queue control tags and protocols are active in the target. It is assumed that initiators will only enable queueing if they also are fully capable of supporting the queue control tags and protocols. The default value of the EQ bit is zero, queueing disabled. The Maximum Queue Depth field indicates the number of queued commands that the LU can accept before a QUEUE FULL status must be presented. It is the maximum number of queue locations available for the selected LU from the selecting initiator. The sum of all Maximum Queue Depth fields from all LU's to all initiators may be significantly greater than the actual total queue depth available, since the same queue elements may be visible to more than one LU and to more than one initiator. A value of zero means that there is no limit to the queue depth if queueing is enabled. The default value is vendor unique. The tables of Page Codes on pages 8-22, 8-48, 9-12, 15-13, and 15-17 must be altered to include a reference to the above page, coded 0Ah. The tables of Sense Keys and Error Codes must have the UNIT ATTENTION code installed that will indicate to an initiator that another initiator has damaged its queued commands. The recovery procedure for such damage involves aborting all known queued commands active against the LU and restarting them. The UNIT ATTENTION condition will have the following description. Sense Key: 06h UNIT ATTENTION Error Code: 2Fh (suggested) Queue Damaged Additional Sense Code: none The description of "Soft" RESET must be clarified to allow the continuation of queueing activity across soft reset activities. This requires the addition of the following text immediately after item "(3) Preserve any......commands, etc)" in section 5.2.2.2 on page 5-11. (4) Preserve all the information required to continue normal dispatching of commands queued prior to the RESET. The rules for handling "Soft" RESET must be modified as follows to correctly treat the case where identification of the command is incomplete until the queue tag messages have been transferred. The text on pages 5-11 and 5-12 "(1) An initiator....for a successfully received COMMAND COMPLETE message." shall be replaced with the following text: (1) An initiator shall not consider a command to be fully identified until the IDENTIFY message is sent to the target and the target responds by changing to any other information transfer phase and requests that at least one byte be transferred. This requires that the target have completed the service of all MESSAGE OUT operations associated with the IDENTIFY message, including queueing tags. (2) A target shall consider a command to be fully identified when it successfully receives the IDENTIFY message and all MESSAGE OUT operations associated with the IDENTIFY message. The existence of such operations is made known by the continued presence of the ATTENTION signal. (3) If an initiator selects a logical unit for which there already is an active command with the same queue tag (if any) for the same initiator, the target shall clear the original command and perform the new command. (4) If a target reselects an initiator to continue a command for which the initator has no record, the initiator shall abort that command by sending the ABORT or ABORT TAG message, depending on whether the reselecting command is a tagged command. (5) An initiator shall consider a command to be completed when it negates ACK for a successfully received COMMAND COMPELTE message. Section 6.? will describe the queueing capability of the SCSI with the text provided below. Section 6.?.3 demonstrating examples of queueing may be appropriately moved to an appendix. 6.? Command Queueing Command queueing allows a target to accept more than one command at a time for execution by a particular LU. Untagged command queueing allows an LU to simultaneously accept commands from more than one initiator at a time. Tagged command queueing allows an LU to simultaneously accept multiple commands from each initiator to each LU. By using command queueing, a target avoids the overhead of presenting BUSY status to a command when it is actually busy processing other commands. In addition, command transmission overheads are decreased because command transmission can be overlapped with mechanical delays in the target LU. Within the specified limitations, the commands may be dequeued, initiated, and completed in an order established by the target. 6.?.1 Untagged Command Queueing Untagged command queueing allows an LU to accept a new command from an initiator while the LU is actually processing commands for another initiator. Only one command may be active for each LU from each initiator at any time. A new command may be accepted for the LU at any time that the SCSI bus is free whether or not another command from a different initiator is being executed by the LU. The target may force the command to disconnect before or after the Command Descriptor Block has been received. It is preferable for the target to accept the Command Descriptor Block so that processing may begin immediately upon termination of the active command and other previously scheduled queued commands. The command is labeled implicitly by the known initiator and target addresses and the LUN. As long as only one command is active from each initiator, the LU can always reconnect to the correct initiator and pointer set with that triple set of information. It is the responsiblity of the initiator to assure that no more than one command is active at any time. Section 6.5.1 describes the actions taken if more than one command is activated from an initiator to the same LU. Untagged command queueing can be supported by SCSI-2 devices or by any SCSI X3.131-1986 device meeting conformance level 2. It is assumed that the initiator will support the required number of active subchannel functions. Each subchannel is effectively a storage area for the pointers associated with each ongoing untagged queued command. 6.?.2 Tagged Command Queueing Tagged command queueing allows a Logical Unit to continue accepting commands until its command queue is full, regardless of how many commands may already be active from each initiator. Three extended messages, UNORDERED QUEUE TAG, ORDERED QUEUE TAG, and HEAD OF QUEUE TAG are defined to allow the initiator to uniquely label each command or set of linked commands with a distinctive QUEUE TAG logical identifier immediately after the IDENTIFY message is transferred. This allows the initiator to explicitly expand the implicit labeling of the commands so that a reconnection can always be properly identified by the quadruple of information including the initiator address, the target address, the LUN, and the Tag Value provided in the queueing message. Each initiator must assure that all its outstanding quadruples of labels are unique. The three extended messages each specify a different relationship between the labeled command and other previously enqueued commands. For a particular initiator, multiple commands labeled by any of the three tagging messages, UNORDERED QUEUE TAG, ORDERED QUEUE TAG, or HEAD OF QUEUE TAG as well as no more than one untagged command may be executed to the same LU. If only commands labeled by UNORDERED QUEUE TAGS are being transmitted, the commands may be executed in an order selected by the target device. Commands from other initiators are also executed in an order selected in the same manner. The command ordering is done by the target to meet the performance and functional goals desired for that target and LU. Commands with ORDERED QUEUE TAGS must be executed in the exact order received with respect to other ORDERED QUEUE TAG labeled commands and with respect to commands without logical identifiers. In addition, all UNORDERED QUEUE TAG labeled commands, regardless of initiator, received before a particular ORDERED QUEUE TAG labeled command must be executed before that ORDERED QUEUE TAG labeled command. All UNORDERED QUEUE TAG labeled commands, regardless of initiator, received after a particular ORDERED QUEUE TAG labeled command must be executed after that ORDERED QUEUE TAG command. All HEAD OF QUEUE TAG labeled commands are placed first in the queue for execution immediately after any command that may have previously been dequeued and initiated. Consecutive HEAD OF QUEUE TAG labeled commands are executed in LIFO order. The specification provides two modes that define specific rules on the relative order of execution of commands other than ORDERED QUEUE TAG and HEAD OF QUEUE TAG labeled commands. The two modes define specific and different queue management algorithms to be executed by the target. Commands that do not have tag values provided are managed by the target according to the rules for untagged queueing. Only one such command may be active at a time for each initiator/target/LUN triple. Links of commands are tagged by a single queue tag. That queue tag applies to the entire chain. The first command of the link chain received by the target is queued in the normal manner. Subsequent commands in the link chain are executed immediately following the execution of the first command in the link chain. In this manner, commands within a link chain are executed in strict order, with no possibility of interleaving with other commands to the same LU. Implementors note: Using linked commands may not provide a substantial throughput improvement in busy systems. The strict sequential ordering of commands required by linking may partially defeat the target's attempt to increase throughput through appropriate queue reordering algorithms. The RESERVE and RELEASE commands are normally executed as ORDERED QUEUE TAG labeled commands. Use of HEAD OF QUEUE TAG labeled commands for these functions could result in subsequent reservation conflicts with previously issued commands. Caution must be used in programming these commands to avoid constructing queues with deadlocks. An independent communications path may be required to prevent such deadlocks. The TEST UNIT READY and INQUIRY commands are often executed as HEAD OF QUEUE TAG labeled commands, since the information they have to offer is often either instantaneously available or has no effect on the state of the LU. Error handling is controlled by the state of the Queue Control MODE SELECT parameters. The recovery parameters allow one of two recovery action options to be selected. The first is to continue execution of commands in their normal queued sequence after the Contingent Allegiance or Extended Contingent Allegiance (ECA) state has been terminated. During the Contingent Allegiance or ECA state, any command transmitted to the target by another initiator will receive BUSY status. No commands will be dispatched from the queue to the LU during the Contingent Allegiance or ECA state. The commands executed during the Contingent Allegiance or ECA state shall be untagged commands. The commands may modify the state of the queue by removing all or selected members of the queue as part of the recovery procedure. As soon as the Contingent Allegiance or ECA state is terminated, normal execution begins again with execution of commands from the queue or any other appropriate source. The second recovery action clears the entire remaining queue after the Contingent Allegiance or ECA state has been terminated. A Unit Attention condition is posted for all initiators which have commands that had currently been queued or in execution against that LU. When the Contingent Allegiance or ECA state is first established, the queue is frozen until the state is terminated. During this time, any command transmitted to the target by another initiator will receive BUSY status. If commands are combined by the queuing algorithm such that more than one command's execution is affected by the error, then a CHECK CONDITION shall be posted for all affected commands. Deferred errors are normally related to a command that has long since completed. As such, there is no attempt to point back to the queue tag assigned to the original failing command. Devices not supporting any tagging messages, either because they do not support command queueing, because command queueing has been disabled by the Queue Control Page of the MODE SELECT command, or because they meet X3.131-1986, respond with a MESSAGE REJECT message to reject the tagging messages. The command is expected to continue from that time on in the normal manner without making use of the tagging information or the Tag Value and without performing tagged queueing. Untagged queueing will still operate normally. Tagged command queueing may also be switched off internally by the device during certain initialization periods or to control internal resource utilization. If the period of suspension is short, the QUEUE FULL status may be used to temporarily reject commands. If the period is expected to be longer, the MESSAGE REJECT message will allow the commands to continue to be executed but without queueing. The ABORT TAG message is a mechanism designed to remove selected operations from the queue. The message is presented to the target immediately after the IDENTIFY message to complete the Initiator/Target/LUN/Tag Value quadruple required to completely identify the command to be deleted from the queue. After completing the acceptance of the ABORT TAG message, the target shall go to BUS FREE. Execution of an ABORT TAG message for which there is no corresponding command in process or enqueued in the target is not an error. If the target is able to identify the command referenced by the quadruple of information, then the command will be removed from the queue. If the command is already dequeued and being executed, the command execution is halted. Any pending status or accumulated data is abandoned and not presented to the initiator. The command may already have started to modify the data in the target. It is the responsibility of the initiator software to perform appropriate system level recovery procedures. The CLEAR QUEUE message is a mechanism designed to remove all commands to the selected LU from the queue and from execution, regardless of either the initiator or the Tag Value of the command. Any untagged command for that LUN is also removed from the queue or from execution. After completing the acceptance of the CLEAR QUEUE message, the target shall go to BUS FREE. Execution of a CLEAR QUEUE message when no commands are in process or enqueued in the target is not an error. Pending status or accumulated data is abandoned and not presented to the initiator. One or more of the commands may already have started to modify the data in the target. A Unit Attention condition is raised for all other initiators with commands which had been either executing or had been queued against that LUN. It is the responsibility of the initiator software to perform appropriate system level recovery procedures. 6.?.3 Example of Command Queueing An example of command queueing requires the consideration of the execution of a number of commands. After each command, the state of the queue kept in the target must be shown to indicate the function actually performed by the queueing. 6.?.3.1 Typical Tag Sequences for a Queued Command A tagged queued command requires the following sequences for normal execution. The initiator first arbitrates for the bus, and after successfully obtaining the bus, selects the appropriate target SCSI device. The ATN line is asserted during the selection sequence to indicate that a message out sequence is required by the initiator. The first message byte transferred is a normal IDENTIFY message out. The ATN line continues to be asserted during the message out sequence to indicate that the initiator requires a second message out cycle. The second message byte transferred is the appropriate queue tag message, in this case an UNORDERED QUEUE TAG message out. The third and last message out byte is transmitted containing the Tag Value. It is transferred with ATN deasserted to indicate that no more message out bytes are required. The target then obtains the Command Descriptor Block. Assuming the command requires disconnection, the target transmits a DISCONNECT message to the initiator and then enters the BUS FREE phase. The target places the transmitted command, identified by the Initiator/Target/LUN/Tag Value quadruple, at the appropriate place in the queue for execution. When the target dequeues the command for execution, a physical latency period may be required. At the end of this period, when the target is prepared to transmit the appropriate data, the target begins an arbitration and enters a RESELECTION phase. After a successful reselection, the target transmits the IDENTIFY message to indicate to the initiator the LU for which the reselection is taking place. After transmitting the IDENTIFY message, the target transmits the two bytes of the UNORDERED QUEUE TAG with the Tag Value originally transmitted by the initiator. The initiator uses the Initiator/Target/LUN/Tag Value quadruple to identify the correct set of pointers and control blocks associated with the command and to establish the necessary preconditions for data transfer. The target begins data transfer. With appropriate synchronous or asynchronous data transfer protocols, the required data fields are transmitted to or from the initiator by the target. When the data transfer is successfully completed, the target terminates the operation with GOOD status and with a COMMAND COMPLETE message, indicating that all transfer for the command is completed and that the initiator can post a completion indication for the command to the appropriate host software. The system level completion indication identifies the command through the proper mapping of the Initiator/Target/LUN/Tag Value quadruple to the system level operation identifier. 6.?.3.2 Example of Tagged Queueing Execution An example of the execution of five queued commands is described in sufficient detail to demonstrate how tagged queueing operates. All tagged commands are from one initiator to a single LU of a single target. The five commands are defined in table 6-8. The target is a direct executed, it is assumed that the actuator is in position to access logical block 10000. TABLE 6-8 TAGGED QUEUEING EXAMPLE: COMMANDS TO BE EXECUTED COMMAND TAG TAG VALUE FIRST BLOCK BLOCK COUNT READ UNORDERED 01 10000 1000 READ UNORDERED 02 100 1 READ ORDERED 03 1000 1000 READ UNORDERED 04 10000 1 READ UNORDERED 05 2000 1000 The optimum order would require that those blocks close to the actuator be the first blocks accessed, followed by those increasingly far from the actuator. However, the command with Tag Value 3 is identified by an ORDERED QUEUE TAG, so that all unordered tagged commands transferred previously must be executed before, while all unordered tagged commands transferred after the ordered command must be executed after the ordered queue tagged command. The actual order in which the operations were dequeued and executed would then be as shown in Table 6-9. TABLE 6-9 TAGGED QUEUEING EXAMPLE: ACTUAL ORDER OF COMMAND EXECUTION COMMAND TAG TAG VALUE FIRST BLOCK BLOCK COUNT READ UNORDERED 01 10000 1000 READ UNORDERED 02 100 1 READ ORDERED 03 1000 1000 READ UNORDERED 05 2000 1000 READ UNORDERED 04 10000 1 Commands tagged as 01 and 02 are executed in the order received since the actuator is already in position to execute command 01. Command 02 must be executed before command 04 or 05 because the ordered command tagged as 03 was transmitted after commands 01 and 02 but before commands 04 and 05. Command 03 is then executed after command 02. The commands tagged as 04 and 05 are executed after the ordered command 03. Command 05 is executed before command 04 because the actuator is in position to access block 2000 after executing command 03. Command 04 is then executed. As an example of the operation of the head of queue command tagging, consider that a new command, identified by a HEAD OF QUEUE TAG message with a Tag Value of 08, is transmitted to the target while the ordered command with the Tag Value of 03 is being executed. The command tagged 03 continues operation, but the new head of queue command is placed in the queue for execution before all subsequent commands. In this case, the queue for execution after the ordered command 03 was executed would appear as shown in Table 6-10. TABLE 6-10 TAGGED QUEUEING: ORDER MODIFIED BY HEAD OF QUEUE TAG COMMAND TAG TAG VALUE FIRST BLOCK BLOCK COUNT READ ORDERED 03 IN EXECUTION READ HEAD OF Q 08 0 8 READ UNORDERED 05 2000 1000 READ UNORDERED 04 10000 1 The extraction of maximum performance gains using tagged queueing requires careful implementation of the queueing and dequeuing algorithms in the target. In addition, all operating system software in the initiators must be well disciplined to allow a maximum number of ordered commands to be executed with a minimum of synchronizing ordered commands. RESERVE/RELEASE, SET LIMITS, and appropriate software locking conventions must be used to guarantee the proper relationship between transaction execution and the data stored on the peripheral devices. These conventions are not defined by this standard. The above text should completely reflect the suggested modifications to the function for SCSI-2 devices. The text assumes the existence of the ECA and Contingent Allegiance functions recommended by the working group in September. The committee offers this proposal for your final review and acceptance.