A SAS Outstanding Xfer_RDY Question.

Nixon, Bob Bob.Nixon at emulex.com
Tue Jul 1 09:14:13 PDT 2003


* From the T10 Reflector (t10 at t10.org), posted by:
* "Nixon, Bob" <Bob.Nixon at Emulex.Com>
*
This message is in MIME format. Since your mail reader does not understand
this format, some or all of this message may not be legible.

------_=_NextPart_001_01C33FEB.CFBC8990
Content-Type: text/plain

"...things will get ugly real quick but so be it..."
 
George, this is not an acceptable answer for a commercial product.
Given the current SAS standard:
 
1)  There is no reason to expect various initiators to coordinate
setting of first burst.
 
2)  In fact, this very issue will lead them NOT to...each initiator may
see some advantage (a hair better performance, an increase in stability)
to choosing their own value (or zero) for first burst.
 
3 The result of my not designing for noncoordinated settings is that MY
initiator will hang.  Doesn't matter whose fault it is, in the eyes of
the customer.  In fact, if I don't design for the worst case, it IS my
fault.
 
4) Designing for the worst case as pointed out earlier in this thread
requires active assurance that the setting of first burst is at least
known before every comand is sent.  Since this involves awareness of
unit attentions and sending of MODE SENSE/SELECT commands, the transport
layer becomes highly tangled with the application.  This was exactly
what 02-403 was intended to untangle, but I believe it didn't go far
enough.
 
5)  As Rob pointed out, I can't simplify by forcing my own preferred
value:  I have to be able to detect and support first burst of whatever
current size in order to change first burst to my preferred size.
 
6)  SPC-3 hints that using first burst of zero to disable first burst is
an unacceptable solution...in the specification of the first burst field
it says "SCSI transport protocols supporting this field shall provide an
additional mechanism to enable and disable the first burst function."
Additional to what?  The only reasonable interpretation of the adjective
"additional" in this context is "other than the mode page setting".  SAS
does not provide such an additional mechanism, so I would claim it is
out of compliance with SPC-3.
 
7)  FCP-2 provides an "additional mechanism" as a flag during login.
Then the PLDA interoperability profile requires that flag to be set so
as to prohibit using first burst at all.  Given the above, this is no
surprise, and the precedent should be considered carefully by SAS.  The
major vendors of FCP equipment still are able to demonstrate 99%+ of
theoretical performance, so the value-add of the first burst feature is
highly suspect.
 
Taking all that into consideration, my company and I would strongly
recommend that one of the following resolutions be incorporated in SAS
1.1 (in our preference order):
 
1)  Obsolete first burst.
 
2)  Provide a true "additional mechanism" to disable it on a per
initiator basis.  Our proposal to do it with a flag in the COMMAND IU is
one such mechanism.
 
3)  Make first burst a fixed value by standard (e.g., "the value of
first burst shall be 10000h").
 
4)  As a marginally acceptable resolution, make first burst a fixed
property of the target (i.e., "the value of first burst shall not be
settable by MODE SELECT).
 
   - Bob
 
 
-----Original Message-----
From: George Penokie [mailto:gop at us.ibm.com]
Sent: Tuesday, July 01, 2003 6:24 AM
To: Nixon, Bob
Cc: t10 at t10.org
Subject: RE: A SAS Outstanding Xfer_RDY Question.



Bob, 

A value of zero in the first burst size field in the
disconnect/reconnected mode page turns off first burst. There is no need
for any other mechanism. If initiators start making changes to mode
pages without cooperation from other initiators things will get ugly
real quick but so be it, that's the way mode pages work and have always
worked. There are many examples of this with first burst only being one.
I believe the way it is defined is just fine and that adding anything to
the command to turn it on/off would only add complexity with no added
practical value.

Bye for now,
George Penokie

Dept 2C6  114-2 N212
E-Mail:    gop at us.ibm.com
Internal:  553-5208
External: 507-253-5208   FAX: 507-253-2880





	"Nixon, Bob" <Bob.Nixon at emulex.com> 
Sent by: owner-t10 at t10.org 


06/30/2003 03:51 PM 


        
        To:        t10 at t10.org 
        cc:         
        Subject:        RE: A SAS Outstanding Xfer_RDY Question. 




* From the T10 Reflector (t10 at t10.org), posted by:
* "Nixon, Bob" <Bob.Nixon at Emulex.Com>
*
How does an initiator indicate to a target whether to use first burst or
not?  

Both SAM-3 and SAS specify that the Send SCSI Command and SCSI Command
Received transport services include a parameter indicating whether first
burst is enabled or not.  The prior discussion in this thread makes it
obvious the choice of the initiator has to be communicated to the target
in
order that XFER_RDY can be sent when needed.

We could claim setting the first burst size to zero in the
disconnect-reconnect mode page accomplishes the necessary communication.
But this would lead to possibly disastrous inefficiency...the first
burst
size setting affects all initiators for a port and can be changed by any
of
them at any time.  Every initiator would have to assure the correct mode
page setting before every command.  The means of doing this are messy at
best.

FCP-2 is the only other protocol that I could find that supports first
burst.  It implements the required means to disable first burst by a
per-initiator login flag (though this falls a little short of the
per-command implication of the transport service specification in
SAM-3).  I
would recommend SAS 1.1 meet the requirement by adding a bit to the SSP
COMMAND IU indicating whether the target is to honor (bit=0) or ignore
(bit=1) first burst size for the command.

I'll write this up for next week unless one of you educates me before
then.


  - Bob

-----Original Message-----
From: Evans, Mark [mailto:Mark_Evans at maxtor.com]
Sent: Friday, June 27, 2003 2:29 PM
To: 'Sheffield, Robert L'; t10 at t10.org
Subject: RE: A SAS Outstanding Xfer_RDY Question.


* From the T10 Reflector (t10 at t10.org), posted by:
* "Evans, Mark" <Mark_Evans at maxtor.com>
*
Hi Bob,

If the value in the FIRST BURST SIZE field is not zero, all of the rules
for
first burst are the same as for an XFER_RDY, that is, a SAS target port
shall not send an XFER_RDY until all of the data for the first burst has
been sent.  This is explicit in the standard in 10.2.6.1.5 FIRST BURST
SIZE
field.  The following is the fourth paragraph in that clause:

"If the amount of data to be transferred for the command is less than
the
amount of data specified by the FIRST BURST SIZE field, the SSP target
port
shall not transmit an XFER_RDY frame for the command.  If the amount of
data
to be transferred for the command is greater than the amount of data
specified by the FIRST BURST SIZE field, the SSP target port shall
transmit
an XFER_RDY frame after it has received all of the data specified by the
FIRST BURST SIZE field from the initiator.  All data for the command is
not
required to be transferred during the same connection in which the
command
is transferred."

I guess we could make this stronger by changing, "...the SSP target port
shall transmit an XFER_RDY frame after it has received all of the data
specified by the FIRST BURST SIZE field..." to "...the SSP target port
shall
not transmit an XFER_RDY frame until it has received all of the data
specified by the FIRST BURST SIZE field...", but the first was intended
to
mean the second.

Regards,

Mark Evans
Maxtor Corporation

-----Original Message-----
From:                  Sheffield, Robert L
[mailto:robert.l.sheffield at intel.com] 
Sent:                 Friday, June 27, 2003 12:44 PM
To:                 t10 at t10.org
Subject:                 RE: A SAS Outstanding Xfer_RDY Question.

* From the T10 Reflector (t10 at t10.org), posted by:
* "Sheffield, Robert L" <robert.l.sheffield at intel.com>
*
I didn't see this the last time I looked into the question, but it does
look
like only one XFER_RDY can be outstanding at a time. I looked into it
before
because of a question about the first burst transfer, and what sort of
timeout might occur in case the first burst never completes. I assumed
that
since the first burst is an implicit XFER_RDY, that the timeout would be
the
same whether it's the first burst or a transfer associated with an
explicit
XFER_RDY (the Initiator Response Timeout timer). What I'm left wondering
now
is if the target must wait for the first burst transfer to complete
before
sending the first XFER_RDY? Is that the intent, and is is stated
anywhere?

Bob Sheffield

-----Original Message-----
From: Sriram Srinivasan [mailto:srirams at lsil.com]
Sent: Friday, June 27, 2003 11:31 AM
To: t10 at t10.org 
Subject: RE: A SAS Outstanding Xfer_RDY Question.


* From the T10 Reflector (t10 at t10.org), posted by:
* Sriram Srinivasan <srirams at lsil.com>
*

 If sas-r04a.pdf is the current SAS spec, I see that in "10.2.1.8
Receive 
Data-Out transport protocol service" it talks about limiting number of 
outstanding XFER_RDY frames per command to 1.  I don't see where it
allows 
more than 1 outstanding XFER_RDY frame per command.
 
 
 \Sriram\
 
" 
" * From the T10 Reflector (t10 at t10.org), posted by:
" * "Bill Galloway" <BillG at breatech.com>
" *
" I believe the current SAS spec allows more than one outstanding
XFER_RDY 
per
" command.  There was some talk of limiting this but it never went
anywhere.
" 
" Bill Galloway
" Pivot3, Inc.
" BillG at pivot3.com
" P: (281) 530-3063
" F: (281) 988-0398 
" 
" -----Original Message-----
" From: owner-t10 at t10.org [mailto:owner-t10 at t10.org] On Behalf Of
Whitehill,
" Cliff
" Sent: Friday, June 27, 2003 11:03 AM
" To: t10 at t10.org
" Subject: A SAS Outstanding Xfer_RDY Question.
" 
" 
" * From the T10 Reflector (t10 at t10.org), posted by:
" * "Whitehill, Cliff" <cwhitehi at lsil.com>
" *
" All,
" 
" My current understanding is that for SAS you can have only one
outstanding
" Xfer_RDY per command.  Is this true? If so, where does it say this in
the
" SAS specification?
" 
" Regards,
" 
" --------------------------------------------
" Clifford A. Whitehill
" Staff Systems Engineer
" LSI Logic Corp.
" 2001 Danfield Ct.
" Fort Collins, CO. 80525
" Tel: 970-206-5024
" Fax: 970-206-5244
" email: cliff.whitehill at lsil.com
" 
" *
" * For T10 Reflector information, send a message with
" * 'info t10' (no quotes) in the message body to majordomo at t10.org
" 
" *
" * For T10 Reflector information, send a message with
" * 'info t10' (no quotes) in the message body to majordomo at t10.org



----------------------------------------------------------------------
                "FORGET NOT THAT THE EARTH DELIGHTS TO FEEL YOUR BARE
FEET,
              AND THE WINDS LONG TO PLAY WITH YOUR HAIR"
                                         -Khalil Gibran

Sriram Srinivasan                       Sriram.Srinivasan at lsil.com
ASIC Design Engineer, LSI Logic,
2001 Danfield Ct.,
Phone: 970-206-5847
Fort Collins, CO 80525
FAX  : 970-206-5244
----------------------------------------------------------------------

*
* For T10 Reflector information, send a message with
* 'info t10' (no quotes) in the message body to majordomo at t10.org
*
* For T10 Reflector information, send a message with
* 'info t10' (no quotes) in the message body to majordomo at t10.org
*
* For T10 Reflector information, send a message with
* 'info t10' (no quotes) in the message body to majordomo at t10.org
*
* For T10 Reflector information, send a message with
* 'info t10' (no quotes) in the message body to majordomo at t10.org




------_=_NextPart_001_01C33FEB.CFBC8990
Content-Type: text/html

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">

 "...things will get ugly real quick but so be it..."
  
 George, this is not an acceptable answer for a commercial product.  Given the current SAS standard:
  
 1)  There is no reason to expect various initiators to coordinate setting of first burst.
  
 2)  In fact, this very issue will lead them NOT to...each initiator may see some advantage (a hair better performance, an increase in stability) to choosing their own value (or zero) for first burst.
  
 3 The result of my not designing for noncoordinated settings is that MY initiator will hang.  Doesn't matter whose fault it is, in the eyes of the customer.  In fact, if I don't design for the worst case, it IS my fault.
  
 4) Designing for the worst case as pointed out earlier in this thread requires active assurance that the setting of first burst is at least known before every comand is sent.  Since this involves awareness of unit attentions and sending of MODE SENSE/SELECT commands, the transport layer becomes highly tangled with the application.  This was exactly what 02-403 was intended to untangle, but I believe it didn't go far enough.
  
 5)  As Rob pointed out, I can't simplify by forcing my own preferred value:  I have to be able to detect and support first burst of whatever current size in order to change first burst to my preferred size.
  
 6)  SPC-3 hints that using first burst of zero to disable first burst is an unacceptable solution...in the specification of the first burst field it says "SCSI transport protocols supporting this field shall provide an additional mechanism to enable and disable the first burst function."  Additional to what?  The only reasonable interpretation of the adjective "additional" in this context is "other than the mode page setting".  SAS does not provide such an additional mechanism, so I would claim it is out of compliance with SPC-3.
  
 7)  FCP-2 provides an "additional mechanism" as a flag during login.  Then the PLDA interoperability profile requires that flag to be set so as to prohibit using first burst at all.  Given the above, this is no surprise, and the precedent should be considered carefully by SAS.  The major vendors of FCP equipment still are able to demonstrate 99%+ of theoretical performance, so the value-add of the first burst feature is highly suspect.
  
 Taking all that into consideration, my company and I would strongly recommend that one of the following resolutions be incorporated in SAS 1.1 (in our preference order):
  
 1)  Obsolete first burst.
  
 2)  Provide a true "additional mechanism" to disable it on a per initiator basis.  Our proposal to do it with a flag in the COMMAND IU is one such mechanism.
  
 3)  Make first burst a fixed value by standard (e.g., "the value of first burst shall be 10000h").
  
 4)  As a marginally acceptable resolution, make first burst a fixed property of the target (i.e., "the value of first burst shall not be settable by MODE SELECT).
  
    - Bob
  
  
 -----Original Message-----
From: George Penokie [mailto:gop at us.ibm.com]
Sent: Tuesday, July 01, 2003 6:24 AM
To: Nixon, Bob
Cc: t10 at t10.org
Subject: RE: A SAS Outstanding Xfer_RDY Question.



Bob, 

A value of zero in the first burst size field in the disconnect/reconnected mode page turns off first burst. There is no need for any other mechanism. If initiators start making changes to mode pages without cooperation from other initiators things will get ugly real quick but so be it, that's the way mode pages work and have always worked. There are many examples of this with first burst only being one. I believe the way it is defined is just fine and that adding anything to the command to turn it on/off would only add complexity with no added practical value.

Bye for now,
George Penokie

Dept 2C6  114-2 N212
E-Mail:    gop at us.ibm.com
Internal:  553-5208
External: 507-253-5208   FAX: 507-253-2880




 "Nixon, Bob" <Bob.Nixon at emulex.com> 
Sent by: owner-t10 at t10.org 06/30/2003 03:51 PM 
        
        To:        t10 at t10.org 
        cc:         
        Subject:        RE: A SAS Outstanding Xfer_RDY Question. 



* From the T10 Reflector (t10 at t10.org), posted by:
* "Nixon, Bob" <Bob.Nixon at Emulex.Com>
*
How does an initiator indicate to a target whether to use first burst or
not?  

Both SAM-3 and SAS specify that the Send SCSI Command and SCSI Command
Received transport services include a parameter indicating whether first
burst is enabled or not.  The prior discussion in this thread makes it
obvious the choice of the initiator has to be communicated to the target in
order that XFER_RDY can be sent when needed.

We could claim setting the first burst size to zero in the
disconnect-reconnect mode page accomplishes the necessary communication.
But this would lead to possibly disastrous inefficiency...the first burst
size setting affects all initiators for a port and can be changed by any of
them at any time.  Every initiator would have to assure the correct mode
page setting before every command.  The means of doing this are messy at
best.

FCP-2 is the only other protocol that I could find that supports first
burst.  It implements the required means to disable first burst by a
per-initiator login flag (though this falls a little short of the
per-command implication of the transport service specification in SAM-3).  I
would recommend SAS 1.1 meet the requirement by adding a bit to the SSP
COMMAND IU indicating whether the target is to honor (bit=0) or ignore
(bit=1) first burst size for the command.

I'll write this up for next week unless one of you educates me before then.


  - Bob

-----Original Message-----
From: Evans, Mark [mailto:Mark_Evans at maxtor.com]
Sent: Friday, June 27, 2003 2:29 PM
To: 'Sheffield, Robert L'; t10 at t10.org
Subject: RE: A SAS Outstanding Xfer_RDY Question.


* From the T10 Reflector (t10 at t10.org), posted by:
* "Evans, Mark" <Mark_Evans at maxtor.com>
*
Hi Bob,

If the value in the FIRST BURST SIZE field is not zero, all of the rules for
first burst are the same as for an XFER_RDY, that is, a SAS target port
shall not send an XFER_RDY until all of the data for the first burst has
been sent.  This is explicit in the standard in 10.2.6.1.5 FIRST BURST SIZE
field.  The following is the fourth paragraph in that clause:

"If the amount of data to be transferred for the command is less than the
amount of data specified by the FIRST BURST SIZE field, the SSP target port
shall not transmit an XFER_RDY frame for the command.  If the amount of data
to be transferred for the command is greater than the amount of data
specified by the FIRST BURST SIZE field, the SSP target port shall transmit
an XFER_RDY frame after it has received all of the data specified by the
FIRST BURST SIZE field from the initiator.  All data for the command is not
required to be transferred during the same connection in which the command
is transferred."

I guess we could make this stronger by changing, "...the SSP target port
shall transmit an XFER_RDY frame after it has received all of the data
specified by the FIRST BURST SIZE field..." to "...the SSP target port shall
not transmit an XFER_RDY frame until it has received all of the data
specified by the FIRST BURST SIZE field...", but the first was intended to
mean the second.

Regards,

Mark Evans
Maxtor Corporation

-----Original Message-----
From:                  Sheffield, Robert L [mailto:robert.l.sheffield at intel.com] 
Sent:                 Friday, June 27, 2003 12:44 PM
To:                 t10 at t10.org
Subject:                 RE: A SAS Outstanding Xfer_RDY Question.

* From the T10 Reflector (t10 at t10.org), posted by:
* "Sheffield, Robert L" <robert.l.sheffield at intel.com>
*
I didn't see this the last time I looked into the question, but it does look
like only one XFER_RDY can be outstanding at a time. I looked into it before
because of a question about the first burst transfer, and what sort of
timeout might occur in case the first burst never completes. I assumed that
since the first burst is an implicit XFER_RDY, that the timeout would be the
same whether it's the first burst or a transfer associated with an explicit
XFER_RDY (the Initiator Response Timeout timer). What I'm left wondering now
is if the target must wait for the first burst transfer to complete before
sending the first XFER_RDY? Is that the intent, and is is stated anywhere?

Bob Sheffield

-----Original Message-----
From: Sriram Srinivasan [mailto:srirams at lsil.com]
Sent: Friday, June 27, 2003 11:31 AM
To: t10 at t10.org 
Subject: RE: A SAS Outstanding Xfer_RDY Question.


* From the T10 Reflector (t10 at t10.org), posted by:
* Sriram Srinivasan <srirams at lsil.com>
*

 If sas-r04a.pdf is the current SAS spec, I see that in "10.2.1.8 Receive 
Data-Out transport protocol service" it talks about limiting number of 
outstanding XFER_RDY frames per command to 1.  I don't see where it allows 
more than 1 outstanding XFER_RDY frame per command.
 
 
 \Sriram\
 
" 
" * From the T10 Reflector (t10 at t10.org), posted by:
" * "Bill Galloway" <BillG at breatech.com>
" *
" I believe the current SAS spec allows more than one outstanding XFER_RDY 
per
" command.  There was some talk of limiting this but it never went anywhere.
" 
" Bill Galloway
" Pivot3, Inc.
" BillG at pivot3.com
" P: (281) 530-3063
" F: (281) 988-0398 
" 
" -----Original Message-----
" From: owner-t10 at t10.org [mailto:owner-t10 at t10.org] On Behalf Of Whitehill,
" Cliff
" Sent: Friday, June 27, 2003 11:03 AM
" To: t10 at t10.org
" Subject: A SAS Outstanding Xfer_RDY Question.
" 
" 
" * From the T10 Reflector (t10 at t10.org), posted by:
" * "Whitehill, Cliff" <cwhitehi at lsil.com>
" *
" All,
" 
" My current understanding is that for SAS you can have only one outstanding
" Xfer_RDY per command.  Is this true? If so, where does it say this in the
" SAS specification?
" 
" Regards,
" 
" --------------------------------------------
" Clifford A. Whitehill
" Staff Systems Engineer
" LSI Logic Corp.
" 2001 Danfield Ct.
" Fort Collins, CO. 80525
" Tel: 970-206-5024
" Fax: 970-206-5244
" email: cliff.whitehill at lsil.com
" 
" *
" * For T10 Reflector information, send a message with
" * 'info t10' (no quotes) in the message body to majordomo at t10.org
" 
" *
" * For T10 Reflector information, send a message with
" * 'info t10' (no quotes) in the message body to majordomo at t10.org



----------------------------------------------------------------------
                "FORGET NOT THAT THE EARTH DELIGHTS TO FEEL YOUR BARE FEET,
              AND THE WINDS LONG TO PLAY WITH YOUR HAIR"
                                         -Khalil Gibran

Sriram Srinivasan                       Sriram.Srinivasan at lsil.com
ASIC Design Engineer, LSI Logic,
2001 Danfield Ct.,                                                                     Phone: 970-206-5847
Fort Collins, CO 80525                                                                    FAX  : 970-206-5244
----------------------------------------------------------------------

*
* For T10 Reflector information, send a message with
* 'info t10' (no quotes) in the message body to majordomo at t10.org
*
* For T10 Reflector information, send a message with
* 'info t10' (no quotes) in the message body to majordomo at t10.org
*
* For T10 Reflector information, send a message with
* 'info t10' (no quotes) in the message body to majordomo at t10.org
*
* For T10 Reflector information, send a message with
* 'info t10' (no quotes) in the message body to majordomo at t10.org




------_=_NextPart_001_01C33FEB.CFBC8990--




More information about the T10 mailing list