CAM-3 March Working Group Minutes

Mark Evenson mevenson at
Wed Mar 29 19:22:00 PST 1995


> Subject: CAM-3 March Working Group Minutes (Text Version)
>  Date: Thu, 23 Mar 1995 17:08:05 -0500
>  From: Bill Dallas  <dallas at>
>  To: scsi at
> Minutes for CAM working group 
> Accredited Standards Committee
> X3, Information Processing Systems
> 	                           	            Doc. No.: X3T10/95-174
>          	                           	    Date: March 21, 1995
> 						    Project: CAM-2/990D
>                            	         	    Ref. Doc.: X3T10/792D-R12
> To:         Membership of X3T10
> From:       William Dallas
> Subject:  Minutes of March 1995 CAM-2 Working Group Meeting 
> ...
> Attachments:
> CAM-3 Data Type and Structure Size Definitions
> To allow easier transportability (not binary comparability) of 
> CAM-3 peripheral drivers, and SIM/HBA's between different 
> machine platforms and operating systems, a cam types definition 
> file (cam_types.h) shall  be provided by the supplier of the XPT.  
> The cam_types.h header file shall define all CAM-3 defined data 
> type sizes as specified by this International standard.
> The supplier of the XPT shall also the supply a CCB structure 
> definitions file (cam.h) which shall use the defined data types and 
> CAM_boundary rules as specified by CAM-3. The term 
> "CAM_boundary" is a defined term and refers to the address 
> alignment of a data type within a structure and the structure itself. 
> (refer to the Definitions Clause X.X and Structure Member 
> CAM_boundary rules Clause X.X for further details).  The CCBs 
> structure names, member names and sizes specified by CAM-3 
> shall be reflected in the cam.h file. 
> CAM_boundaries (address boundaries) of the CAM-3 defined 
> structures and its members is specified by CAM-3. It shall be the 
> responsibility of the XPT supplier to preserve those 
> CAM_boundaries as specified by the CAM_boundary rules. The 
> CAM_boundary rules allows CCB structures and members to be 
> aligned to a known offset for a 16, 32, 64, etc bit processors 
> regardless of the platform or O.S. defined pointer size.  
> Data Type Sizes
> CAM-3 defines the storage sizes and whether the storage class is 
> signed or unsigned for the structures it defines.  The supplier of 
> the XPT shall define the storage class in the cam_types.h as 
> follows:
> CAM_U8    -    An unsigned a 8 bit quantity
> CAM_S8    -    An signed a 8 bit quantity
> CAM_U16  -    An unsigned a 16 bit quantity.
> CAM_S16  -    An signed a 16 bit quantity.
> CAM_U32  -    An unsigned a 32 bit quantity.
> CAM_S32   -   An signed a 32 bit quantity.
> Pointer sizes within CAM-3 structures shall be a power of 32 bits 
> and shall be the size that the O.S. defines for its pointer size 
> rounded up to a power of 32 bits.  The following is an example of 
> pointer size rules:

Should change "power" to "multiple" in previous paragraph.

> OS Pointer Size
> CAM-3 Structure Storage Size
> 1 to 32 bits
> 32 bits
> 33 to 64 bits
> 64 bits
> 65 to 128 bits
> 128 bits

I suppose what you mean to say here is that this pointer size is really
a pad field (to pad it out to the specified size) plus the actual pointer
field.  The driver would dereference a 48 bit pointer field (for example)
and be oblivious to the 16-bit padding (just as it's oblivious to any
alignment padding).

> Structure Member CAM_boundary rules
> The defined CAM-3 data types and pointers declared within a 
> defined CAM-3 structure shall be naturally aligned to their 
> addressed boundary.  
> The XPT supplier shall ensure the CAM_boundary by padding the 
> structure so that the member aligns with its address boundary.  If 
> the next defined member data type does not naturally align then 
> the XPT supplier shall ensure this CAM_boundary by padding the 
> structure so that the member naturally aligns with it address 
> boundary. 
> All structures and structures within structures shall be aligned to 
> the naturally aligned pointer boundary.
> All arrays within a structure shall be aligned to the naturally 
> aligned pointer boundary.
> ...
> CCB * xpt_ccb_alloc( )
> The xpt_ccb_alloc service shall provide the means for the 
> peripheral drivers, SIMs and the XPT to allocate CCBs for use. It 
> shall be the responsibility of the XPT to ensure that the pointer of 
> the allocated CCB returned from the xpt_ccb_alloc service call 
> shall point to a memory buffer large enough to contain any of the 
> possible XPT/SIM function request CCBs.  The xpt_ccb_alloc 
> service shall return a NULL pointer if memory resources are not 
> immediately available.  
> The returned CCB shall be properly initialized for use as a SCSI 
> I/O request CCB.  The SIM private data area shall have been 
> already set up to be used by the XPT and SIM, and shall not be 
> modified by the peripheral driver.  The allocated CCB can only be 
> used (i.e., sent to the XPT), once.  Once the CCB has completed it 
> shall be returned using the xpt_ccb_free service.

Why can the CCB only be used once?  I noticed this in CAM-1 as well.

> Returns:
> Non NULL *:
> Indicates that the service has allocated a CCB for the 
> caller.
> NULL *:
> Indicates that the service could not allocate a CCB at 
> this time.
> void xpt_ccb_free(CCB *)
> The xpt_ccb_free service shall provide the means for the 
> peripheral drivers, SIMs and the XPT to free CCBs after use.  The 
> xpt_ccb_free service takes a pointer to the CCB that the caller has 
> finished with so it can be returned to the CAM subsystem.  
> CCB *:
> Pointer to the CCB to be freed to the CAM subsystem.
> Returns:
> None:  
> CAM_U32 xpt_action(CCB *) 
> All CAM CCB requests to the XPT or a SIM/HBA are placed 
> through this service call.  The CAM Status information for 
> callback on completion CCBs shall be obtained at the callback 
> point via the CAM status fields. The CAM Status information for 
> non callback on completion - non immediate CCBs shall be 
> obtained by polling the CAM status field for a non Request in 
> Progress status.

This kind of mechanism assumes single domain I/O subsystems in which
the PD, XPT and SIM are all in the same domain.  CAM-3 should in a
portable way anticipate more than this as a future looking standard.

>                   The CAM Status information for immediate 
> CCBs shall be obtain on return from the service call by examining 
> the CAM Status field.

Same as previous comment, plus is essentially precluded with implicit

> CCB *:
> Pointer to a CCB that shall be transported/passed to a 
> SIM/HBA or the XPT.
> Returns:
> 	For Immediate CCBs:
> 		Any Valid CAM Status
> 	For Queued CCBs
> 		Request In Progress - Indicates that the CCB has been 
> accepted.
> 		Any other valid CAM Status - Indicates that the CCB 
> has not been accepted
> 	The ultimate of any CCB shall be obtained from the CAM 
> status field of the CCB
> pm_offset_t phy_addr *  XPT_VIRT_TO_PHYS( 
> vm_offset_t addr *,  map *)

Shouldn't have the *'s on addr and phy_addr (it returns a phy_addr, and 
you pass in a vm_offset_t).

> The XPT_VIRT_TO_PHYS service shall convert the passed 
> virtual addr * argument to its associated physical address and 
> return that address.  The map * argument is O.S. dependent (see 
> O.S. dependent clauses for further details).  The caller shall pass a 
> null map * value if the map * is not available to the caller (e.g., 
> usually a kernel virtual I/O transfer does have an associated map).  
> The XPT_VIRT_TO_PHYS service shall ensure that if a map * is 
> passed,  that map * is associated with this virtual address.
> vm_offset_t addr *:
> Pointer of the virtual address that shall be converted to 
> a physical address.
> map *:
> Pointer to the O.S. dependent map structure if 
> available.  Refer to the O.S. specific clauses for more 
> information.
> Returns:
> 	All values:
> The value is the associated physical address.
> Example:
> routine ( CCB_SCSIIO *ccb)
> {
> pm_offset_t *p_addr;
> p_addr = XPT_VIRT_TO_PHYS( (vm_offset_t)ccb-
> >cam_data_ptr, ccb->cam_req_map);
> }
> ...
> Marc Gauthier's comments:
> >> 
> >> The CAM-2 working will be meeting on January 10, 1995 to discuss the
> >> options for CCB structure sizing and some general rules.  The CAM-2
> >> working group will then present to the general SCSI working group its
> >> thoughts on the structure and sizing of the CCBs and the reasoning for its
> >> selection.  It is hoped that the general SCSI working will comment on the
> >> selection (e.g., approve, disapprove,  enhance the CCB, sizing and
> >> general rules or have specific ideas).
> >> 
> >> The migration of the CAM specification which is based upon the SCSI-2
> >> specification to the CAM-2 specification and will be based on the SCSI-3
> >> presents a number of problems.  The largest problem that SCSI-3
> >> presents to CAM is the converting from a dense addressing space to a
> >> extremely sparse addressing space.  The expansion of the target and
> >> LUN from 8 bits to 64 bits will break all the currently written CAM
> >> software.  The term break used in the previous sentence means that the
> >> currently written software will have to be modified and restructured to
> >> conform with CAM-2 and SCSI-3.
> >
> >I concur; not only because the new CAM-2 CCBs will have bigger fields,
> >and different layout, but also because the assumption that one can
> >organise active logical units by arrays indexed by SCSI ID and LUN
> >is no longer true.  Even simple things like scanning the bus for
> >devices, ie. trying every possible ID (and LUN for responding IDs)
> >won't work in many environments.  Scanning for reachable devices
> >likely cannot be done by client/application software anymore, but
> >would have to be handled through CAM; appropriate primitives
> >(CAM commands) for getting a list of active devices need to be
> >adapted or added.
> The CAM-3 working group will be addressing the scanning issues and if you 
> have any specific proposals to address this issue we would like to consider 
> them.
> >
> >> Some other problems with the CAM specification today is its lack of
> >> expansion capability for new features to both the CAM and SCSI
> >> specifications and its assumption of a 32 bit processor world.  
> >> The CAM-2 working group will be working towards a specification 
> >> that is easily expanded, fully dynamic and processor word size
> >>independent.
> >> 
> >> The following are some ideas that should help in solving some of these
> >> issues.
> >> 
> >> Processor word size independence:
> >> To allow transportability (not binary comparability) of CAM-2 peripheral
> >> drivers, and SIM/HBA's between different machine platforms and
> >> operating systems, a cam types definition file (cam_types.h) will be
> >> provided by the supplier of the xpt.  The cam_types.h header file shall
> >> define the CCB structures member sizes.  While the CCB's size varies
> >> based on machine word size, they are of fixed size and structure
> >> member offsets are fixed for a specific machine word size.
> >
> >Your example definitions seem to imply, however, that the size would
> >stay the same for all architectures but only the way in which it is
> >defined varies (eg. defining two U_32's instead of one U_64).
> >Any change in size would be only due to an architecture's natural
> >alignment, and that could likely be normalized with proper padding.
> >
> Yes, that is correct that the structure members are defined the same 
> and the member sizes would remain the same.  The CCB size difference 
> would only be for alignment and this is due to the fact the most

Since the fields (other than pointers) have fixed size and alignment
requirements, the only size differences should be due to pointers.

> machine architectures today have restrictions on accessing storage
> elements not proper boundaries.  This is a little wasteful of memory
> but it allows a easier transport of drivers across platforms. 
> As for defining members (other then pointers) for a maximum of 32 bits
> ( CAM_U32 ) this is also done to allow for easier transport of code between
> machines ( no changes are needed in the code for casting).
> >> The supplier of the xpt shall also the supply a CCB definitions file (cam.h)
> >> which shall use the defined CCBs as specified by CAM-2.  The CCBs
> >> specified in CAM-2 shall specify specific member names of the CCB and
> >> the size of each member. 
> >
> >Why does it have to be supplied by the xpt vendor, and not be the same
> >for all architectures of a given word size?  (not many word sizes to deal
> >with, likely only 32 and 64 bits, though 16 bits and 128 bits might be
> >considered; each could have standard definition...?).
> The XPT is the anchor of CAM and there is one XPT per system, but there
> are multiple peripheral drivers and SIM/HBAs.  Having the XPT define
> the CCBs allows for just 1 definition file which prevents mismatched
> CCB definitions.

Huh?  There's 1 definition file for a given Machine in Marc's proposal
also.  A reason for doing it on a per XPT basis might be (if you assume
no mixed environments) that the XPT supplier knows the alignment rules
of the compiler that it prescribes for use with its system and can thus
properly specify the padding in the header file.

>                   Also having the a supplied definition file allows
> for applications to be compiled on a given machine that understand
> CAM (e.g., a passthru driver that lets an application send commands
> to a device).
> The working group feels that if we define 1 implementation (32 or 64 bits)
> as a reference and state the data type and alignment rules there is less chance
> of conflicts within the document, and the result is then same if we defined a 
> number of platform implementations.


> >> Alignment (address boundaries) of the CCB and its members shall also be
> >> specified by CAM-2. It shall be the responsibility of the xpt supplier to
> >> preserve those alignments as specified by the alignment rules. The
> >> alignment rules allows CCB members to at a known offset for a 16, 32,
> >> 64, etc bit processors. 
> >> 
> >> I have chosen the C language to represent the types definition file.
> >> The cam_types.h file contents shall contain type definitions of the follow:
> >>  typedef char   I_8;                    /* 8 bits */
> >
> >You should say "signed char" rather than "char", which may be signed
> >or unsigned depending on compiler and compiler options.  Actually, it is
> >probably safer to say "signed" in other signed definitions below.
> The editor took your idea and implemented data type rules in a more generic 
> fashion.
> >
> >>  typedef unsigned char    U_8;          /* 8 bits */
> >>  typedef short  I_16;                   /* 16 bits     */
> >>  typedef unsigned short   U_16;         /* 16 bits     */
> >> /* For different machine word sizes ( e.g., 32 or 64 bit )
> >> #ifdef 32_BIT
> >>  typedef  long I_32;                     /* 32 bits          */
> >>  typedef unsigned long  U_32;            /* 32 bits          */
> >> #endif /* 32_BIT */
> >> #ifdef 64_BIT
> >> typedef int I_32                         /* 32 bits     */
> >> typedef unsigned int U_32                /* 32 bits     */
> >> 
> >> Alignment Rules:
> >>      All (x)_8 (chars) shall being on a 8 bit address boundary and shall
> >>      be 8 bits in length.
> >> 
> >>      All (x)_16 (shorts) shall being on a on a 16 bit address boundary
> >>      and shall be 16 bits in length.
> >
> >Probably shouldn't say (shorts) or even (chars) as these don't necessarily
> >have the given number of bits.  ("Octets" should by definition 
> >however always consist of eight bits.)
> The editor believes he has resolved this based on your comments.
> Please refer to the CAM-3 Data Type and Structure Size Definitions 
> proposal,  if you have further comments please feel free to make them. 

The type definitions (given earlier) are the right approach from my
viewpoint.  It's essentially what IDL does (altho' IDL also has required
mappings to ANSI C and other languages).

> >>      All x_32  (32 bits) shall being on a 32 bit address boundary and
> >>      shall be 32 bits in length.
> >
> >Shouldn't the same apply for 64 bits?  Why not simply a general rule,
> >that any basic type should be aligned according to its size?
> >
> The working group believes that the specification of a 64 bit data type other 
> than pointers will have transportability issues.  The working group has choose 
> not to define a 64 bit data type.

Unfortunately, this least common denominator approach is probably
necessary to maximize portability.  But CAM-3 would have a lot more to 
do to truly maximize portability across multiple OS's and platforms
(eg. abstractions/interfaces for DMA, PIO, endianness, interrupts, 
MP synchronization, etc.), so one could argue as to the importance of
maximizing it in this one dimension unless the others are addressed also.  

> >>      All pointers shall begin on a machine word address boundary shall
> >>      be of machine word size.
> >
> >Careful, is machine word size necessarily the size of a pointer?
> >On 8086's (and 68000's?), I would think machine word size is 16,
> >but pointers are 32 bits.  (A general rule would sidestep this issue.)
> >
> The working group and the editor believes they has resolved this issue.
> Please refer to the CAM-3 Data Type and Structure Size Definitions 
> proposal,  if you have further comments please feel free to make them. 
> ...
> >> CAM (CAM-1) CCB header:
> >> /* Common CCB CAM header definition. 
> >>  * For 32 bit machines 
> >>  */
> >> typedef struct ccb_header {
> >>      void *my_addr;                /* The address of this CCB */
> >>      U_16 cam_ccb_len;             /* Length of the entire CCB */
> >>      U_8 cam_func_code;            /* XPT function code */
> >>      U_8 cam_status;               /* Returned CAM subsystem status */
> >>      U_16 cam_hrsvd0;              /* Reserved */
> >>      U_8 cam_path_id;              /* Path ID for the request */
> >>      U_8 cam_target_id;            /* Target device ID */
> >>      U_8 cam_target_lun;           /* Target LUN number */
> >>      U_32 cam_flags;               /* Flags for operation of the subsys */
> >> }CCB_HEADER;                       /* structure ends on 32 bit boundary */
> >> 
> >> CAM 2 CCB header:
> >> /* Common CCB CAM 2 header definition. 
> >>  * For 32 bit machines 
> >>  */
> >> typedef struct ccb_header2 {
> >
> >You might give thought to making identifiers unique in the first 8 chars,
> >for those older compilers that don't take any more into account.  This
> >hasn't been done in the past (e.g. cam_target_id vs. cam_target_lun),
> >but we've had to change structure definitions in our environment to
> >follow our portability guidelines.
> The working group believes that we are within the ANSI C rules for 
> identifiers (the editor will verify).  The actual field names have not been 
> defined yet since they were not defined in CAM-1.  There currently is an 
> informative annex that defined member names but the member names still 
> need to be resolved.

Yes, and CAM shouldn't limit itself for this case anyway, because mappers
|from the standard names to "unique-within-8-chars" can be created for
environments which require these more restrictive compilers.

> ...
> >I should mention that we are currently designing a SCSI support
> >architecture for our realtime message-passing multiprocessor OS, Harmony
> >(see <>).  Because it is fully
> >message-passing, without necessarily shared memory between processes
> >(eg. on multiprocessor systems connected by means other than a common 
> bus),
> >CAM request structures don't fit naturally (or efficiently) into messages.
> >So I have completely redesigned CAM structures to provide us with a
> >native SCSI interface between PD and SIM/HBA that is more easily and
> >efficiently translated into messages -- without, incidentally, losing
> >efficiency or simplicity for non-message-passing environments.
> >As such they are no longer CAM, but may be useful in helping design
> >CAM-2 structures.
> We must take care in designing the CAM-3 structure to preserve
> some compatibility.  If you would like a CAM definition for 
> message passing operating systems please write it up and the working will 
> consider your proposal.

The problem with this approach is that you end up with multiple 
incompatible ways of doing the same driver depending on whether it's on
a shared memory architecture or not, multi-domain I/O system or not, etc.

> ...
> >One thing that has been avoided is duality of meaning of certain fields,
> >such as a data pointer being either a pointer to a block of data or a
> >pointer to a scatter/gather list.  Rather, only a pointer to a scatter/
> >gather list is present, but a single scatter/gather element (pointer +
> >size) is present in the I/O request structure, which the scatter/gather
> >list pointer can point to in the simple case of a single block of data
> >being transferred.  A similar thing is being done for the command data
> >bytes (CDB).  However, one point that is still unclear to me is whether
> >SCSI-3 commands can ever exceed 16 bytes.  If not, then the simplest
> >thing to do is to provide 16 bytes for the CDB and not support
> >interpretation of the CDB as a pointer to the real CDB's.  If they can,
> >and there is no fixed and reasonably short bound on the size of the CDB,
> >then what we'd do is provide 16 bytes for the CDB and a separate pointer
> >to the CDB.  The pointer would always be used, and may point either to
> >the 16-byte CDB array within the request structure, or to an external
> >array of command data bytes if more than 16 bytes are needed.
> >Avoiding such duality makes the code easier to understand, and *more
> >efficient* (since code doesn't have to check the meaning of a field;
> >it simply always accesses the same, more general field).
> Vendor unique commands can be greater then 16 bytes.  The duality of fields 
> does present some overhead checking in the SIM/HBAs but it also allows 
> flexibility in the peripheral drivers to accomplish their tasks.  

You've got the same flexibility with Marc's approach -- the pointer
can either point to the inline CDB (for CDBs 16 bytes or smaller), or
to an out-of-line CDB -- but the advantage is that you don't have the
same field having dual meanings and you can thus get rid of the 
corresponding flag which indicated which meaning was intended.

> >I would have liked to attend the CAM WG meeting, however this is
> >beyond the scope of our budget at this time.  I am, nevertheless,
> >very much interested in what happens in the development of CAM-2.
> We will keep you posted and I appreciate your interest, ideas and
> criticisms ( it is the only way to develop a complete software standard).
> >Regards,
> >
> >-Marc
> >--
> >Marc E. Gauthier
> >Software Engineering Lab, Institute for Information Technology (SEL,IIT)
> >National Research Council Canada, Bldg M-50,  Ottawa ON Canada  K1A


 Mark Evenson		Hewlett-Packard       Cupertino Open Systems Lab
 mevenson at	Voice: 408/447-5601   Fax: 408/447-4380
 19447 Pruneridge Ave	MS 47LA		      Cupertino, CA 95014

More information about the T10 mailing list