X3T9.2/91-015 Date: Fri, 1 Feb 91 16:18:41 -0500 From: "John A. Gallant UEG" To: distribution: ;, see end of body Subject: Changes for the Latest CAM document, (LONG - much included text) Hello Dal Alan, John Lohmeyer and the CAM email group. Sorry for the delay, I have collected a number of small changes and additions for the CAM document. As I now understand it the process for this is to send mail to both of you and the X3T9.2 committee. Please do not dispair over the initial size of this mail, there is a lot of included text from the CAM document. My modifications are indicated with a change bar "|" in column 0. I will deal with the easy ones first, these I believe fall under the simple edit category: 1) Alignment of bytes/shorts/longs with in CCBs. There are a few CCBs that are not properly aligned. The following tables should be properly arranged. The alignment issue has a lot of meaning in the RISC CPU area. TABLE 8-3 GET DEVICE TYPE CCB +----+---+ |Size|Dir| Get Device Type +----+---+--------------------------------------+ | | | - - - - - OSD - - - - - | | 4 | O | Address of this CCB | | 2 | O | CAM Control Block Length | | 1 | O | Function Code | | 1 | I | CAM Status | | 1 | | fill | | | | - - - - Common - - - - - | | 1 | O | Path ID | | 1 | O | Target ID | | 1 | O | LUN | | 4 | O | CAM Flags (OSD) | | | 4 | O | Inquiry Data Pointer | | | 1 | I | Peripheral Device Type of Target/LUN | +----+---+--------------------------------------+ [ I swapped the pointer and type fields. I was able to remove the reserved pad. ] TABLE 8-4 PATH INQUIRY CCB +----+---+ |Size|Dir| Path Inquiry +----+---+--------------------------------------+ | | | - - - - - OSD - - - - - | | 4 | O | Address of this CCB | | 2 | O | CAM Control Block Length | | 1 | O | Function Code | | 1 | I | CAM Status | | 1 | | fill | | | | - - - - Common - - - - - | | 1 | O | Path ID | | 1 | O | Target ID | | 1 | O | LUN | | 4 | O | CAM Flags (OSD) | | 4 | I | Features Supported | | 1 | | Version Number | | | | 00-07h Prior to Rev 1.7 | | | | 08h Implementation Version 1.7 | | | | | 09-FFh Rev No e.g. 23h = 2.3 | | 1 | | SCSI Capabilities | | | | 7 Modify Data Pointers | | | | 6 Wide Bus 32 | | | | 5 Wide Bus 16 | | | | 4 Synchronous Transfers | | | | 3 Linked Commands | | | | 2 reserved | | | | 1 Tagged Queuing | | | | 0 Soft Reset | | 1 | | Target Mode Support | | | | 7 Processor Mode | | | | 6 Phase Cognizant Mode | | | | 5-0 reserved | | 1 | | Miscellaneous | | | | 7 0=Scanned Low to High | | | | 1=Scanned High to Low | | | | 6 0=Removables included in scan | | | | 6 1=Removables not included | | | | 5-0 reserved | | 16 | I | Vendor Unique HBA capabilities | | 4 | I | Size of Private Data Area | | 4 | I | Asynchronous Event capabilities | | | | 31-24 Vendor Unique | | | | 23- 8 reserved | | | | 7 New Devices found during rescan| | | | 6 SIM module De-Registered | | | | 5 SIM module Registered | | | | 4 Sent Bus Device Reset to Target| | | | 3 SCSI AEN | | | | 2 reserved | | | | 1 Unsolicited Reselection | | | | 0 Unsolicited SCSI Bus Reset | | 1 | I | Highest Path ID Assigned | | 1 | I | SCSI Device ID (of Initiator) | | | 1 | | reserved | | | 1 | | reserved | | 16 | I | Vendor ID of SIM-supplier | | 16 | I | Vendor ID of HBA-supplier | | 4 | O | OSD Usage | +----+---+--------------------------------------+ [ I had to add two reserved bytes prior to the ID fields. Made the Rev field be 23/2.3 instead of 21/2.1. Dal is this the correct # ?] TABLE 8-9 ABORT XPT REQUEST CCB +----+---+ |Size|Dir| Abort XPT Request +----+---+--------------------------------------+ | | | - - - - - OSD - - - - - | | 4 | O | Address of this CCB | | 2 | O | CAM Control Block Length | | 1 | O | Function Code | | 1 | I | CAM Status | | 1 | | fill | | | | - - - - Common - - - - - | | 1 | O | Path ID | | 1 | O | Target ID | | 1 | O | LUN | | 4 | O | CAM Flags (OSD) | | | 4 | O | CCB to be Aborted Pointer | +----+---+--------------------------------------+ [ I removed the 2 byte reserved field. ] TABLE 8-12 TERMINATE I/O PROCESS REQUEST CCB +----+---+ |Size|Dir| Terminate I/O Process Request +----+---+--------------------------------------+ | | | - - - - - OSD - - - - - | | 4 | O | Address of this CCB | | 2 | O | CAM Control Block Length | | 1 | O | Function Code | | 1 | I | CAM Status | | 1 | | fill | | | | - - - - Common - - - - - | | 1 | O | Path ID | | 1 | O | Target ID | | 1 | O | LUN | | 4 | O | CAM Flags (OSD) | | | 4 | O | CCB to be Aborted Pointer | +----+---+--------------------------------------+ [ I removed the 2 byte reserved field. ] TABLE 10-1 ENABLE LUN CCB +----+---+ |Size|Dir| Enable LUN CCB +----+---+--------------------------------------+ | | | - - - - - OSD - - - - - | | 4 | O | Address of this CCB | | 2 | O | CAM Control Block Length | | 1 | O | Function Code | | 1 | I | CAM Status | | 1 | | fill | | | | - - - - Common - - - - - | | 1 | O | Path ID | | 1 | O | Target ID | | 1 | O | LUN | | 4 | O | CAM Flags (OSD) | | 2 | O | Group 6 Vendor Unique CDB Length | | 2 | O | Group 7 Vendor Unique CDB Length | | | 4 | O | Pointer to Target CCB List | | | 2 | O | Number of Target CCBs | +----+---+--------------------------------------+ [ I swapped the CCB list pointer and the number of CCBs fields. ] 2) Misc changes in the existing CCBs: There are a couple of changes that I would like to have done. TABLE 8-6 SET ASYNC CALLBACK CCB +----+---+ |Size|Dir| Set Async Callback +----+---+--------------------------------------+ | | | - - - - - OSD - - - - - | | 4 | O | Address of this CCB | | 2 | O | CAM Control Block Length | | 1 | O | Function Code | | 1 | I | CAM Status | | 1 | | fill | | | | - - - - Common - - - - - | | 1 | O | Path ID | | 1 | O | Target ID | | 1 | O | LUN | | 4 | O | CAM Flags (OSD) | | | 4 | O | Asynchronous Event Enables | | | | 31-24 Vendor Unique | | | | 23- 8 reserved | | | | 7 New Devices found during rescan| | | | 6 SIM module De-Registered | | | | 5 SIM module Registered | | | | 4 Sent Bus Device Reset to Target| | | | 3 SCSI AEN | | | | 2 reserved | | | | 1 Unsolicited Reselection | | | | 0 Unsolicited SCSI Bus Reset | | | 4 | O | Asynchronous Callback Pointer | | | 4 | O | Peripheral Driver Buffer Pointer | | | 1 | O | Size of Allocated Peripheral Buffer | +----+---+--------------------------------------+ [ The "I"/in fields needed to be corrected to "O"/out fields. ] TABLE 9-1 SCSI I/O REQUEST CCB +----+---+ |Size|Dir| SCSI I/O Request +----+---+--------------------------------------+ | | | - - - - - OSD - - - - - | | 4 | O | Address of this CCB | | 2 | O | CAM Control Block Length | | 1 | O | Function Code | | 1 | I | CAM Status | | 1 | | fill | | | | - - - - Common - - - - - | | 1 | O | Path ID | | 1 | O | Target ID | | 1 | O | LUN | | 4 | O | CAM Flags (OSD) | | | 4 | O | Next CCB Pointer | | 4 | | reserved (OSD) | | 4 | O | Callback on Completion | | | 4 | O | Peripheral Driver Pointer | | 4 | O | SG List/Data Buffer Pointer | | 4 | O | Data Transfer Length | | 4 | O | Sense Info Buffer Pointer | | 1 | O | Sense Info Buffer Length | | 1 | O | CDB Length | | 2 | O | Number of Scatter/Gather entries | | 4 | | reserved (OSD) | | 1 | I | SCSI Status | | 3 | | reserved (OSD) | | 4 | I | Residual Length | | 12 | O | CDB | | | | - - - - - OSD - - - - - | | 4 | O | Timeout Value | | 4 | O | Message Buffer Pointer (Target-only)| | 2 | O | Message Buffer Length (Target-only)| | 2 | O | VU Flags | | 1 | O | Tag Queue Action | | 3 | | reserved (OSD) | | n | O | Private Data | +----+---+--------------------------------------+ [ I have moved the Peripheral Driver Pointer field down to follow the Callback on Completion field. This places these "bits" from the pointer field down below some "flags" fields in the other CCBs. This is a big help to the Peripheral Device drivers that reuse their allocated CCBs. ] 3) Misc changes and additions to the CAM document text: 6.4.3.3 Error conditions and Queues within the Subsystem The SIM shall place its internal queue for a LUN into the frozen state for any status other than Request Completed Without Error and Request in Progress. When a LUN's queue is in the frozen state, the SIM shall not dispatch any CCBs |to that LUN. Peripheral Device Drivers can still send CCBs to the SIM for the |LUN or any other LUN. Any new CCBs received by the SIM shall be placed at the |end of the queue, unless SIM Queue Priority=1 forces them to the head. [ I added a line iterating that the peripheral device drivers can still send CCBs to a frozen SIM queue. ] 6.6 Asynchronous Callback In an event such as a SCSI Bus Reset or an Asynchronous Event Notification the XPT has to be able to make a callback to the peripheral driver(s), even though there may be no CCBs active for the peripheral driver(s). Callback routines have the same privileges and restrictions as hardware interrupt service routines. During system startup and driver initialization, the peripheral driver should register an Asynchronous Callback routine for all the SCSI devices with which it is working. In order for a peripheral driver to receive asynchronous callbacks, it shall issue a CCB function "Set Asynchronous Callback" with the "Asynchronous Event" fields set to 1 for those events the peripheral driver |wishes to be notified of through an asynchronous callback. The peripheral |driver must explicitly register for the path IDs, targets, and LUNs. |The use of a wild card is not supported for the SET ASYNC CALLBACK CCB. [ I added these lines to make sure the a peripheral device driver will only get async callbacks for the path IDs/targets/LUNs that it knows about. ] 7.1.2.1 From the Peripheral Driver The XPT provides functions to obtain CAM system resources for the peripheral driver. These functions are used to allocate and free CCB resources and to allocate and free DMA resources. There are two routines used in the handling the CCB resources. The two routines are: CCB *xpt_ccb_alloc() and void xpt_ccb_free(CCB *): - The xpt_ccb_alloc() routine returns a pointer to the allocated CCB. The peripheral driver can now use this CCB for it's SCSI/XPT requests. - The xpt_ccb_free() routine takes a pointer to the CCB that the peripheral driver has finished with, and can now be returned to the CAM subsystem CCB pool. - The pointer to the CCB returned from the xpt_ccb_alloc() call shall be large enough to contain any of the possible XPT/SIM function request CCBs. | - The CCB can only be used, send to the XPT, once. Once the CCB has | completed it must be returned using the xpt_ccb_free() routine. [ I needed to add this requirement for the Unix OSD. Some amount of pre-initialization is done on a CCB before it is sent to the driver. Once the CCB has been used the initialization has to be done again before it can be sent to the XPT/SIM again. ] 7.1.6.1 Functions for Peripheral Driver Support b) CCB *xpt_ccb_alloc() This routine is used whenever a peripheral device driver needs a CCB (the common data structure for processing SCSI requests). It returns a pointer to the allocated CCB which the peripheral driver can now use as the CCB for it's |SCSI/XPT requests. The returned CCB will be properly initialized for use |as a SCSI I/O CCB. The SIM private data space is already setup to be used |by the XPT and SIM. This space must not be modified by the peripheral |device driver. It is recommended that the CCB be returned to the XPT |following its use, do not reuse CCBs. |There are no arguments and the return value is a pointer to an initialized |CCB. [ I added a description about the CCB that is returned from this call in the Unix OSD space. At this time both Darryl R. and myself pre- initialize our request blocks prior to giving them to the drivers. ] 8.2.1 Get Device Type This function shall return non-zero CAM Status. - CAM Status of Request Completed Without Error indicates that the specified device is installed and the peripheral device type field is valid. - CAM Status of SCSI Device Not Installed indicates that the peripheral device type field is not valid. | - CAM Status of Invalid Path ID indicates that the Path ID is invalid. [ I would recommend that INVALID PATH ID status be used to remain consistent. ] 8.2.2 Path Inquiry This function shall return non-zero CAM Status. - CAM Status of Request Completed Without Error indicates that the other returned fields are valid. |- CAM Status of Invalid Path ID indicates that the specified Path ID (HBA) is not installed. [ I would recommend that INVALID PATH ID status be used to remain consistent. ] 9.1.4 CAM Status TABLE 9-2 CAM STATUS +------+--------------------------------------+ | 00h | Request in progress | | 01h | Request completed without error | | 02h | Request aborted by host | | 03h | Unable to Abort Request | | 04h | Request completed with error | | 05h | CAM Busy | | 06h | Invalid Request | | 07h | Invalid Path ID | | 08h | SCSI device not installed | | 09h | Unable to Terminate I/O Process | | 0Ah | Target Selection Timeout | | 0Bh | Command Timeout | | 0Dh | Message Reject received | | 0Eh | SCSI Bus Reset Sent/Received | | 0Fh | Uncorrectable Parity Error Detected | | | 10h | Autosense Request Sense Cmd Failed | | 11h | No HBA detected | | 12h | Data OverRun/UnderRun | | 13h | Unexpected Bus Free | | 14h | Target bus phase sequence failure | | 15h | CCB Length Inadequate | | 16h | Cannot Provide Requested Capability | | | 17h | Bus Device Reset Sent | | 18h | Terminate I/O Process | |19-37h| reserved | | | Target Mode Only Status | | 38h | Invalid LUN | | 39h | Invalid Target ID | | 3Ah | Function not Implemented | | 3Bh | Nexus not Established | | 3Ch | Invalid Initiator ID | | 3Dh | SCSI CDB Received | | 3Eh | LUN Already Enabled | | 3Fh | SCSI bus Busy | +------+--------------------------------------+ | - 10h Autosense Request Sense Command Failed: The SIM attempted to obtain | sense data and failed. | - 17h Bus Device Reset Sent: This CCB was terminated because a Bus Device | Reset was sent to the target. [ I added the word "Sent" to the Bus Device Reset CAM status. I added the word "Autosense" to the Request Sense Command Failed CAM status. There has been some discussion here as to the usefulness of the status in conjunction with the Autosense valid bit.] 9.1.8.1 Byte 1 Bits 2 Linked CDB - When set to 1 this CDB is a linked command. If this bit is set, then the Control field in the CDB shall have bit 0=1. If not, the results are unpredictable. See 9.2. | 1 Tag Queue Actions are to be enabled. 0 If the CDB is identified as a Pointer, the first four bytes of the CDB field contain a pointer to the location of the CDB. [ I added the word "Tag" to be consistent with the CAM flags table. ] 4) Changes and additions of a more major nature :-). I would like to add a field to the SCSI I/O CCB, for use in the Unix OSD. This new field will really be a rename of one of the "reserved (SOD) fields. In the Unix space an additional pointer, the buf structure pointer (bp), is needed for information necessary to "map" user space requests. The following text and TABLE 9-1 changes are needed to support this addition. First the change to the CCB. TABLE 9-1 SCSI I/O REQUEST CCB +----+---+ |Size|Dir| SCSI I/O Request +----+---+--------------------------------------+ | | | - - - - - OSD - - - - - | | 4 | O | Address of this CCB | | 2 | O | CAM Control Block Length | | 1 | O | Function Code | | 1 | I | CAM Status | | 1 | | fill | | | | - - - - Common - - - - - | | 1 | O | Path ID | | 1 | O | Target ID | | 1 | O | LUN | | 4 | O | CAM Flags (OSD) | | | 4 | O | Next CCB Pointer | | | 4 | O | Request Mapping Information (OSD) | | 4 | O | Callback on Completion | | | 4 | O | Peripheral Driver Pointer | | 4 | O | SG List/Data Buffer Pointer | | 4 | O | Data Transfer Length | | 4 | O | Sense Info Buffer Pointer | | 1 | O | Sense Info Buffer Length | | 1 | O | CDB Length | | 2 | O | Number of Scatter/Gather entries | | 4 | | reserved (OSD) | | 1 | I | SCSI Status | | 3 | | reserved (OSD) | | 4 | I | Residual Length | | 12 | O | CDB | | | | - - - - - OSD - - - - - | | 4 | O | Timeout Value | | 4 | O | Message Buffer Pointer (Target-only)| | 2 | O | Message Buffer Length (Target-only)| | 2 | O | VU Flags | | 1 | O | Tag Queue Action | | 3 | | reserved (OSD) | | n | O | Private Data | +----+---+--------------------------------------+ [ This copy of TABLE 9-1 includes the previous changes. ] |9.1.19 Request Mapping Information (OSD) | | This field is a pointer to an OSD dependent data structure which is | associated with original I/O request. Now the additional text to the Unix OSD section, 7.1. I would like this to follow the section on Callback on Completion, it also has to do with the SCSI I/O CCB. |7.1.4 Request Mapping Information - Unix OSD definition | | This field is expected to contain a pointer to the buf structure, that | the SCSI I/O CCB was created for. This copy of the buf structure pointer, | bp, is used by the SIM to get to the I/O mapping information needed | to access the data buffers allocated by the application program. A | value of NULL is allowed if there is no need for the SIM to map the | data buffer addresses, ie data count is zero, the buffer is internal | to the kernel or the addresses are physical. In the new section on HBA engines, Section 11, the Execute Engine Request CCB was introduced. This CCB is intended to be treated just like a SCSI I/O Request CCB. It has the potential to be stored in the SIM, on an internal queue, and contains a private data space for the SIMs. In the Unix OSD we pre-allocate our CCBs within the XPT layer. As a result our CCBs must be of "known" length. We need to treat all the CCBs as "overlays" onto the same raw memory buffers. We are able to pre-initialize the SCSI I/O SIM private data space because none of the other CCBs make use of this "hidden" information. Now the Execute Engine Request CCB also has a SIM private data space to be used in exactly the same way. In Unix we need both the Execute Engine Request CCB and the SCSI I/O Request CCB to have exactly the same offsets within the CCB for the SIM private data space. With this requirement I request that the Execute Engine Request CCB have the following definition: TABLE 11-1 EXECUTE ENGINE REQUEST CCB +----+---+ |Size|Dir| Execute Engine Request +----+---+--------------------------------------+ | | | - - - - - OSD - - - - - | | 4 | O | Address of this CCB | | 2 | O | CAM Control Block Length | | 1 | O | Function Code | | 1 | I | CAM Status | | 1 | | fill | | | | - - - - Common - - - - - | | 1 | O | Path ID | | 1 | O | Target ID | | 1 | O | LUN | | 4 | O | CAM Flags (OSD) | | 4 | | reserved (OSD) | | 4 | O | Request Mapping Information (OSD) | | 4 | O | Callback on Completion | | 4 | O | Peripheral Driver Pointer | | 4 | O | SG List/Data Buffer Pointer | | 4 | O | Data Transfer Length | | 4 | O | Engine Buffer Data Pointer | | 1 | | reserved (OSD) | | 1 | | reserved (OSD) | | 2 | O | Number of Scatter/Gather Entries | | 4 | O | Destination Data Maximum Length | | 4 | I | Destination Data Length | | 4 | I | Source Residual Length | | 12 | | reserved (OSD) | | | | - - - - - OSD - - - - - | | 4 | O | Timeout Value | | 4 | | reserved (OSD) | | 2 | O | Engine Number | | 2 | O | VU Flags | | 1 | | reserved (OSD) | | 3 | | reserved (OSD) | | n | O | Private Data | +----+---+--------------------------------------+ [ I tried to keep similar field at the same offsets just for clarity. What is critical is to keep the SIM private data offset the same. My engineers also requested that the Peripheral Driver Pointer say at the same offset. For all the addition of space I still do not dare to attempt to provide any definitions for the different fields. I included some of the fields from the SCSI I/O CCB because I thought that they would be needed, ie Callback on Completion, and Number of Scatter/Gather Entries. I did not attempt to "combine" any of the reserved spaces, I thought the comparison with the SCSI I/O CCB would be easier this way.] Thanks for your time I hope all this was useful. John A. Gallant jag@decvax.dec.com Senior Software Engineer - Ultrix Engineering Group Digital Equipment Corp. (603) 881-2472 "A beautiful theory, killed by a nasty, ugly, little fact." Thomas H. Huxly %%% overflow headers %%% To: 2501752@mcimail.com, 4300421@mcimail.com, darryl@ivy.isc.com, decwrl!microsoft!andyc, decwrl!oliveb!sco!adapcal!judyp, decwrl!pacbell!cfcl!rdm, decwrl!uunet!congrunt!artk, decwrl!uwm!wuarchive!texbell!dell!alan, egt@hprnd.hp.com, jag@decvax.dec.com, jerry.armstrong@clemson.ncr.com, john.lohmeyer@wichita.ncr.com, jrj@aviary.kodak.com, lawlor@iwtio.att.com, michael.g.roche@waterloo.ncr.com, olson@sgi.com, philip.bair@dayton.ncr.com, rita@paramus.sony.com, snively@sun.com, steve.betts@dayton.ncr.com, tjw@aviary.kodak.com, ts@cup.portal.com, weiner@xylogics.com %%% end overflow headers %%%