ordering across I_Ts

George Penokie gop at us.ibm.com
Tue Dec 20 14:11:23 PST 2005


* From the T10 Reflector (t10 at t10.org), posted by:
* George Penokie <gop at us.ibm.com>
*
This is a multipart message in MIME format.
--=_alternative 00798CE9862570DD_=
Content-Type: text/plain; charset="US-ASCII"


Rob is correct on the last two questions about when the command can
start. In other words, in both cases it has to wait. 

Bye for now,
George Penokie

Dept 2C6  114-2 N212
E-Mail:    gop at us.ibm.com
Internal:  553-5208
External: 507-253-5208   FAX: 507-253-2880





"Elliott, Robert (Server Storage)" <Elliott at hp.com> 
Sent by: owner-t10 at t10.org 


12/20/2005 01:29 PM 

To
<t10 at t10.org> 

cc

Subject
RE: ordering across I_Ts

	





* From the T10 Reflector (t10 at t10.org), posted by:
* "Elliott, Robert (Server Storage)" <Elliott at hp.com>
*
[Pulling out question #3...]

> -----Original Message-----
> From: owner-t10 at t10.org [mailto:owner-t10 at t10.org] On Behalf 
> Of Gerry.Houlder at seagate.com
> Sent: Monday, December 19, 2005 9:49 AM
> To: Frederick.Knight at netapp.com
> Cc: t10 at t10.org
> Subject: Re: TMF, ordering across I_Ts, and ALUA questions
> 
> * From the T10 Reflector (t10 at t10.org), posted by:
> * Gerry.Houlder at seagate.com
> *
> My responses added as GAH.

> From: "Knight, Frederick" <Frederick.Knight @netapp.com>

> To: <t10 at t10.org>          
> Subject: TMF, ordering across I_Ts, and ALUA questions              
> Date: 12/16/2005 01:51 PM

> 
> 3) Relative ordering of ordered and head of queue tagged commands
> 
>    Summary
>    -------
>    When a TST=000b target receives a task with the Ordered or
>    Head_Of_Queue task tag on an I_T_L Nexus A, do the ordering
>    requirements apply to tasks from other I_T_L Nexuses?
> 
> GAH: No, they do not.

I agree about the L part, but since the example below talks
just about I_T nexuses, I assume the question was intended
to be "other I_T nexuses."

sam4r04 section 8.2 includes these definitions:
"All older tasks ended:
If the TST field in the Control mode page (see SPC-3) equals 
000b, all tasks received <<on all I_T nexuses>> and accepted
earlier in time than the referenced task have ended.

All head of queue and older ordered tasks ended:
If the TST field equals 000b, all the following tasks
received <<on all I_T nexuses>> have ended:
 a) All head of queue tasks; and
 b) All ordered tasks accepted earlier in time than the 
    referenced task."

Section 8.6.3 uses them to define handling of ordered tasks:
"The task shall not enter the enabled task state until 
all head of queue tasks and all older tasks in the task set 
have ended (see 8.4)."

So, given the words highlighted in << >>, an ordered task has 
to wait on tasks received on all I_T nexuses (within the 
same logical unit).

>    Details
>    -------
>    Section 8 of SAM3R14 describes task set management, including
>    relative ordering of tasks.  For example:
> 
>      - Tasks with the Ordered Task Attribute cannot enter the ENABLED
>        state until all in-progress head of queue and all older tasks
>        in the task set have completed (section 8.6.3)
>      - Tasks with the Simple Task Attribute cannot enter the ENABLED
>        state until all head of queue tasks and older ordered
>        tasks have completed (section 8.6.2)
>      - etc.
> 
>    If the target maintains one task set per LUN for all initiator
>    ports (Control mode page TST = 000b), then the task set may
>    include tasks submitted by multiple initiators, over multiple
>    I_T nexuses.
> 
>    Since the ordering semantics described in SAM3R14 section 8 are
>    described in terms of the task set, this implies an ordering
>    constraint on commands and responses across different I_T
>    nexuses.
> 
>    But section 4.6.3 of SAM3R14 states the following:
> 
>      "The SCSI architecture model makes no assumption about and places
>       no requirement on the ordering of requests or responses for
>       different I_T nexuses."

That relates to the ordering between sending and receiving SCSI 
ports (e.g. over the FC, SAS, iSCSI, etc. interface).  It doesn't 
override any rules for a device server.  However, if a device
server sends a first response out target port A and then sends a 
second response out target port B, it means the application client 
may receive the second response before the first.

>    So, consider the case of two I_T Nexuses submitting IO requests
>    to the same LUN on a TST=000b target.  Assume the initiators
>    submit the following commands in the following order, and also
>    assume that the target receives the commands in the same order:
> 
>       Nexus A submits simple command A1
>       Nexus B submits simple command B1
>       Nexus B submits simple command B2
>       Target moves all of A1, B1, and B2 to the Enabled state
>         and starts work on them
>       Nexus A submits ordered command A2
> 
>    When is the target allowed to begin processing A2:
> 
>    - After completion of A1 only?
>    - Or after completion of A1, B1, and B2?
> 
> GAH: A2 can begin processing as soon as A1 completes.

I think it has to wait.

>    Here's another case:
> 
>       Nexus A submits ordered command A3
>       Target moves A3 to the Enabled state and starts work on it
>       Nexus B submits simple command B3
>       Nexus B submits simple command B4
> 
>    Does the presence in the task set of A3 prevent the target from
>    beginning work on B3?
> 
> GAH: The T10 rules would allow B3 and/or B4 to be processed ahead
>      of command A3. That doesn't mean that a target is required to do
>      that ordering, however. Thats what "no requirement on ordering
>      between commands from different ports" means.

I think it has to wait.

--
Rob Elliott, elliott at hp.com
Hewlett-Packard Industry Standard Server Storage Advanced Technology
https://ecardfile.com/id/RobElliott



*
* For T10 Reflector information, send a message with
* 'info t10' (no quotes) in the message body to majordomo at t10.org



--=_alternative 00798CE9862570DD_=
Content-Type: text/html; charset="US-ASCII"


<br><font size=2 face="sans-serif">Rob is correct on the last two questions
about when the command can start. In other words, in both cases it has
to wait.</font>
<br><font size=2 face="sans-serif"><br>
Bye for now,<br>
George Penokie<br>
<br>
Dept 2C6 &nbsp;114-2 N212<br>
E-Mail: &nbsp; &nbsp;gop at us.ibm.com<br>
Internal: &nbsp;553-5208<br>
External: 507-253-5208 &nbsp; FAX: 507-253-2880<br>
<br>
</font>
<br>
<br>
<br>
<table width=100%>
<tr valign=top>
<td width=40%><font size=1 face="sans-serif"><b>"Elliott, Robert (Server
Storage)" <Elliott at hp.com&gt;</b> </font>
<br><font size=1 face="sans-serif">Sent by: owner-t10 at t10.org</font>
<p><font size=1 face="sans-serif">12/20/2005 01:29 PM</font>
<td width=59%>
<table width=100%>
<tr>
<td>
<div align=right><font size=1 face="sans-serif">To</font></div>
<td valign=top><font size=1 face="sans-serif"><t10 at t10.org&gt;</font>
<tr>
<td>
<div align=right><font size=1 face="sans-serif">cc</font></div>
<td valign=top>
<tr>
<td>
<div align=right><font size=1 face="sans-serif">Subject</font></div>
<td valign=top><font size=1 face="sans-serif">RE: ordering across I_Ts</font></table>
<br>
<table>
<tr valign=top>
<td>
<td></table>
<br></table>
<br>
<br>
<br><font size=2><tt>* From the T10 Reflector (t10 at t10.org), posted by:<br>
* "Elliott, Robert (Server Storage)" <Elliott at hp.com&gt;<br>
*<br>
[Pulling out question #3...]<br>
<br>
> -----Original Message-----<br>
> From: owner-t10 at t10.org [mailto:owner-t10 at t10.org] On Behalf <br>
> Of Gerry.Houlder at seagate.com<br>
> Sent: Monday, December 19, 2005 9:49 AM<br>
> To: Frederick.Knight at netapp.com<br>
> Cc: t10 at t10.org<br>
> Subject: Re: TMF, ordering across I_Ts, and ALUA questions<br>
> <br>
> * From the T10 Reflector (t10 at t10.org), posted by:<br>
> * Gerry.Houlder at seagate.com<br>
> *<br>
> My responses added as GAH.<br>
<br>
> From: "Knight, Frederick" <Frederick.Knight @netapp.com&gt;<br>
<br>
> To: <t10 at t10.org&gt; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;<br>
> Subject: TMF, ordering across I_Ts, and ALUA questions &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;<br>
> Date: 12/16/2005 01:51 PM<br>
<br>
> <br>
> 3) Relative ordering of ordered and head of queue tagged commands<br>
> <br>
> &nbsp; &nbsp;Summary<br>
> &nbsp; &nbsp;-------<br>
> &nbsp; &nbsp;When a TST=000b target receives a task with the Ordered
or<br>
> &nbsp; &nbsp;Head_Of_Queue task tag on an I_T_L Nexus A, do the ordering<br>
> &nbsp; &nbsp;requirements apply to tasks from other I_T_L Nexuses?<br>
> <br>
> GAH: No, they do not.<br>
<br>
I agree about the L part, but since the example below talks<br>
just about I_T nexuses, I assume the question was intended<br>
to be "other I_T nexuses."<br>
<br>
sam4r04 section 8.2 includes these definitions:<br>
"All older tasks ended:<br>
If the TST field in the Control mode page (see SPC-3) equals <br>
000b, all tasks received <<on all I_T nexuses>> and accepted<br>
earlier in time than the referenced task have ended.<br>
<br>
All head of queue and older ordered tasks ended:<br>
If the TST field equals 000b, all the following tasks<br>
received <<on all I_T nexuses>> have ended:<br>
 &nbsp;a) All head of queue tasks; and<br>
 &nbsp;b) All ordered tasks accepted earlier in time than the <br>
 &nbsp; &nbsp; referenced task."<br>
<br>
Section 8.6.3 uses them to define handling of ordered tasks:<br>
"The task shall not enter the enabled task state until <br>
all head of queue tasks and all older tasks in the task set <br>
have ended (see 8.4)."<br>
<br>
So, given the words highlighted in << >>, an ordered task has
<br>
to wait on tasks received on all I_T nexuses (within the <br>
same logical unit).<br>
<br>
> &nbsp; &nbsp;Details<br>
> &nbsp; &nbsp;-------<br>
> &nbsp; &nbsp;Section 8 of SAM3R14 describes task set management, including<br>
> &nbsp; &nbsp;relative ordering of tasks. &nbsp;For example:<br>
> <br>
> &nbsp; &nbsp; &nbsp;- Tasks with the Ordered Task Attribute cannot
enter the ENABLED<br>
> &nbsp; &nbsp; &nbsp; &nbsp;state until all in-progress head of queue
and all older tasks<br>
> &nbsp; &nbsp; &nbsp; &nbsp;in the task set have completed (section
8.6.3)<br>
> &nbsp; &nbsp; &nbsp;- Tasks with the Simple Task Attribute cannot
enter the ENABLED<br>
> &nbsp; &nbsp; &nbsp; &nbsp;state until all head of queue tasks and
older ordered<br>
> &nbsp; &nbsp; &nbsp; &nbsp;tasks have completed (section 8.6.2)<br>
> &nbsp; &nbsp; &nbsp;- etc.<br>
> <br>
> &nbsp; &nbsp;If the target maintains one task set per LUN for all
initiator<br>
> &nbsp; &nbsp;ports (Control mode page TST = 000b), then the task set
may<br>
> &nbsp; &nbsp;include tasks submitted by multiple initiators, over
multiple<br>
> &nbsp; &nbsp;I_T nexuses.<br>
> <br>
> &nbsp; &nbsp;Since the ordering semantics described in SAM3R14 section
8 are<br>
> &nbsp; &nbsp;described in terms of the task set, this implies an ordering<br>
> &nbsp; &nbsp;constraint on commands and responses across different
I_T<br>
> &nbsp; &nbsp;nexuses.<br>
> <br>
> &nbsp; &nbsp;But section 4.6.3 of SAM3R14 states the following:<br>
> <br>
> &nbsp; &nbsp; &nbsp;"The SCSI architecture model makes no assumption
about and places<br>
> &nbsp; &nbsp; &nbsp; no requirement on the ordering of requests or
responses for<br>
> &nbsp; &nbsp; &nbsp; different I_T nexuses."<br>
<br>
That relates to the ordering between sending and receiving SCSI <br>
ports (e.g. over the FC, SAS, iSCSI, etc. interface). &nbsp;It doesn't
<br>
override any rules for a device server. &nbsp;However, if a device<br>
server sends a first response out target port A and then sends a <br>
second response out target port B, it means the application client <br>
may receive the second response before the first.<br>
<br>
> &nbsp; &nbsp;So, consider the case of two I_T Nexuses submitting IO
requests<br>
> &nbsp; &nbsp;to the same LUN on a TST=000b target. &nbsp;Assume the
initiators<br>
> &nbsp; &nbsp;submit the following commands in the following order,
and also<br>
> &nbsp; &nbsp;assume that the target receives the commands in the same
order:<br>
> <br>
> &nbsp; &nbsp; &nbsp; Nexus A submits simple command A1<br>
> &nbsp; &nbsp; &nbsp; Nexus B submits simple command B1<br>
> &nbsp; &nbsp; &nbsp; Nexus B submits simple command B2<br>
> &nbsp; &nbsp; &nbsp; Target moves all of A1, B1, and B2 to the Enabled
state<br>
> &nbsp; &nbsp; &nbsp; &nbsp; and starts work on them<br>
> &nbsp; &nbsp; &nbsp; Nexus A submits ordered command A2<br>
> <br>
> &nbsp; &nbsp;When is the target allowed to begin processing A2:<br>
> <br>
> &nbsp; &nbsp;- After completion of A1 only?<br>
> &nbsp; &nbsp;- Or after completion of A1, B1, and B2?<br>
> <br>
> GAH: A2 can begin processing as soon as A1 completes.<br>
<br>
I think it has to wait.<br>
<br>
> &nbsp; &nbsp;Here's another case:<br>
> <br>
> &nbsp; &nbsp; &nbsp; Nexus A submits ordered command A3<br>
> &nbsp; &nbsp; &nbsp; Target moves A3 to the Enabled state and starts
work on it<br>
> &nbsp; &nbsp; &nbsp; Nexus B submits simple command B3<br>
> &nbsp; &nbsp; &nbsp; Nexus B submits simple command B4<br>
> <br>
> &nbsp; &nbsp;Does the presence in the task set of A3 prevent the target
from<br>
> &nbsp; &nbsp;beginning work on B3?<br>
> <br>
> GAH: The T10 rules would allow B3 and/or B4 to be processed ahead<br>
> &nbsp; &nbsp; &nbsp;of command A3. That doesn't mean that a target
is required to do<br>
> &nbsp; &nbsp; &nbsp;that ordering, however. Thats what "no requirement
on ordering<br>
> &nbsp; &nbsp; &nbsp;between commands from different ports" means.<br>
<br>
I think it has to wait.<br>
<br>
--<br>
Rob Elliott, elliott at hp.com<br>
Hewlett-Packard Industry Standard Server Storage Advanced Technology<br>
https://ecardfile.com/id/RobElliott<br>
<br>
<br>
<br>
*<br>
* For T10 Reflector information, send a message with<br>
* 'info t10' (no quotes) in the message body to majordomo at t10.org<br>
</tt></font>
<br>
--=_alternative 00798CE9862570DD_=--





More information about the T10 mailing list