SAS SL_CC handling of CLOSE

Sheffield, Robert L robert.l.sheffield at intel.com
Wed Apr 2 13:39:39 PST 2003


* From the T10 Reflector (t10 at t10.org), posted by:
* "Sheffield, Robert L" <robert.l.sheffield at intel.com>
*
Let's see first if I can restate the problem to make sure I'm clear on it...

The STP/SATA bridge is responsible for deciding when to request a close on behalf of a SATA device. The SATA device knows whether it has frames to send, but the bridge doesn't, and there doesn't seem to be a reliable way for the SATA device to inform the bridge whether or not it will have frames to send. So whatever heuristics the bridge uses to decide when to send a close, there's a race condition where the SATA device attempts to send frames just as the bridge decides to close the connection. I'm not sure what we can do about that.

If we simply accept that this race condition occurs and requires cleanup of some kind (close connection, reset device, retry command???) then perhaps there's a more straightforward solution:
1. Prohibit STP target ports (or perhaps just SATA/STP bridges) from generating CLOSE.
2. Require STP target ports to close immediately upon receipt of CLOSE without requiring a preceding Request Close from the Link Layer.

This leaves it entirely up to the STP initiator port to manage closing connections. Since an STP/SATA bridge doesn't ever generate CLOSE, the STP initiator port doesn't ever have to worry about receiving a CLOSE without first having received a Request Close from the link layer. The race condition at the STP/SATA bridge still exists, but the STP initiator sees all of the traffic from the SATA target just like the STP/SATA bridge, so it has at least as much information to work with as the STP/SATA bridge, plus some concept of the state of execution of outstanding commands, so it ought to be able to do as good a job of deciding when to close the connection as the bridge could.

This just applies to SATA devices attached through an STP/SATA bridge. We might want STP target devices to behave differently since they can account for the internal state of command execution in deciding when to close connections.

This isn't a formal proposal to resolve the problem, just some thoughts. What do you think?

Bob

Bob Sheffield
Intel Corporation - CH6-333
Storage Components Division (SCD)
5000 W. Chandler Blvd
Chandler, AZ 85226-3699
Phone: 480-554-8597
Fax: 480-554-6617


-----Original Message-----
From: George Penokie [mailto:gop at us.ibm.com]
Sent: Wednesday, April 02, 2003 12:39 PM
To: Elliott, Robert (Server Storage)
Cc: owner-t10 at t10.org; t10 at t10.org
Subject: Re: SAS SL_CC handling of CLOSE


* From the T10 Reflector (t10 at t10.org), posted by:
* George Penokie <gop at us.ibm.com>
*




Although Rob's description of the issue he describes below may be accurate
for STP it is not complete for SSP and SMP. Also the proposed solution
makes a CLOSE redundant with a BREAK.

My comments to Rob's points are:

1- There is no doubt a problem with STP but I don't see why the solution
has to turn SSP and SMP on it's head. Define STP so it can deal with CLOSE
in whatever fashion is needed. Making this kind of significant change to
SSP and SMP this late in the schedule will cause unforeseen consequences to
other parts of the standard.

2-If a CLOSE comes at a point in time that is illegal then having 1 msec
timeouts and BREAKs is the right thing the have happen. This is a error
situation and it should not be handled as if everything is OK, it's not.
There is no problem with SSP or SMP if the rules are followed in sending
DONEs and CLOSEs. If the rules   are not followed then the device that does
this gets what it deserves and probably will be quickly fixed. And you are
not going to get a false CLOSE as it is a triple primitive.

3-If BREAKs cause problems in STP then fix it in STP.

The proposed solution of responding to CLOSEs  any time a connection
receives a CLOSE is no different than how a BREAK is responded to in the
current SL_CC machines. (i.e., any time a BREAK is received the is a
transition to a state the transmits a BREAK which is exactly what is being
proposed to happen when  a CLOSE is received). Also, you cannot just throw
in a few states into SL_CC to handle this and expect no repercussions in
the upper layers unless, of course, you treat it like just like a BREAK. If
it's not just like a BREAK then how are the upper layers going to deal with
for example a data transfer that is interrupted with an illegal CLOSE?

Bye for now,
George Penokie

Dept 2C6  114-2 N212
E-Mail:    gop at us.ibm.com
Internal:  553-5208
External: 507-253-5208   FAX: 507-253-2880




                                                                                                                                       
                      "Elliott, Robert                                                                                                 
                      (Server Storage)"        To:       <t10 at t10.org>                                                                 
                      <Elliott at hp.com>         cc:                                                                                     
                      Sent by:                 Subject:  SAS SL_CC handling of CLOSE                                                   
                      owner-t10 at t10.org                                                                                                
                                                                                                                                       
                                                                                                                                       
                      03/31/2003 09:44                                                                                                 
                      PM                                                                                                               
                                                                                                                                       
                                                                                                                                       




While connected, the SL_CC state machine currently only honors an incoming
CLOSE after the protocol-specific link layer (SSP, STP, or SMP) has sent a
Request Close message.  In SSP, Request Close is sent after DONEs are
exchanged.  In SMP, Request Close is sent after the RESPONSE frame is
transmitted.  No such time exists for STP and there is no way to define
when Request Close is needed.

This has a few problems:
1. There is no definition for when the STP link layer sends Request Close,
and in fact there is no time similar to SSP or STP where both sides of a
connection can expect the connection to be closed.  Either side is allowed
to request a close when the bus is idle (no R_RDY granted).  One side's
SL_CC can be waiting for its STP link layer to signal Request Close when a
CLOSE arrives.  Per the current standard, it is supposed to ignore the
CLOSE - definitely not the intention.  [this is the issue prompting
attention]


2. The XL state machine responds to CLOSE immediately; it has no
protocol-specific knowledge.  If a CLOSE shows up at the wrong time in an
SSP or SMP connection, the XL state machine starts closing down the
connection, but the SL_CC at the end of the connection ignores it.  This
eventually leads to 1 ms timeouts and BREAKs; it would be much cleaner if
SL_CC behaved the same as XL.

3. Scenarios that devolve into BREAKs mess up CLOSE (CLEAR AFFILIATION)
usage, since there is no BREAK (CLEAR AFFILIATION).


To solve this, we propose that SL_CC honor CLOSEs without waiting for
protocol-specific link layer permission.  This requires these changes:
1.  SL_CC honors CLOSE in the SL_CC3:Connected state.  If it sees CLOSE, it
transitions to a new state SL_CC7:Disconnect.

2. SL_CC7:Disconnect transmits a CLOSE and goes to SL_CC0:Idle.

3. Both SL_CC4:DisconnectWait and SL_CC7:Disconnect send Enable Disable SSP
(Disable), Enable Disable SMP (Disable), and Enable Disable STP (Disable)
messages to disable the protocol-specific link layers.  (rather than
waiting for SL_CC0:Idle to do so).

Some other alternatives are:
a) make SL_CC honor CLOSE during STP connections but ignore it during SSP
and SMP connections; or
b) make the STP link layer detect CLOSE and send, in negative or zero time,
an Allow Close message to SL_CC so it also detects the CLOSE.


Specific wording:
7.14.4.5 SL_CC3:Connected state
7.14.4.5.1 State description
This state enables the SSP, STP, or SMP link layer state machine to
transmit dwords during a connection.
If this state is entered from SL_CC1:ArbSel state or the SL_CC2:Selected
state with an argument of Open
SMP Connection then this state shall send an Enable Disable SMP (Enable)
message to the SMP link layer
state machines (see 7.18.4).

If this state is entered from SL_CC1:ArbSel state or the SL_CC2:Selected
state with an argument of Open
SSP Connection then this state shall send an Enable Disable SSP (Enable)
message to the SSP link layer
state machines (see 7.16.7).

If this state is entered from SL_CC1:ArbSel state or the SL_CC2:Selected
state with an argument of Open
STP Connection then this state shall send an Enable Disable STP (Enable)
message to the STP link layer
state machines (see 7.17.7).

This state shall request idle dwords be transmitted by repeatedly sending
Transmit Idle Dword messages to
the SL transmitter until the SSP, SMP, or STP link layer state machine
starts transmitting.

7.14.4.5.2 Transition SL_CC3:Connected to SL_CC4:DisconnectWait
This transition shall occur if a Request Close message is received.

7.14.4.5.3 Transition SL_CC3:Connected to SL_CC5:BreakWait
This transition shall occur if a Request Break message is received and a
BREAK Received message has not
been received.

7.14.4.5.4 Transition SL_CC3:Connected to SL_CC6:Break
This transition shall occur if a BREAK Received message is received and
after sending a Connection Closed
(Break Received) confirmation to the port layer.

7.14.4.5.n Transition SL_CC3:Connected to SL_CC7:Disconnect [new]
This transition shall occur if a CLOSE Received message is received.


7.14.4.6 SL_CC4:DisconnectWait state
7.14.4.6.1 State description
This state closes the connection and releases all resources associated with
the connection.

[new paragraph:]
Upon entry to this state, this state shall send Enable Disable SSP
(Disable), Enable Disable SMP (Disable),
and Enable Disable STP (Disable) messages to the SSP, SMP, and STP link
layer state machines.

This state shall:
1) send a Transmit CLOSE (Normal) message or Transmit CLOSE (Clear
Affiliation) message to the SL transmitter;
2) send at least three Transmit Idle Dword messages to the SL transmitter;
and
3) initialize and start the Close Timeout timer.
A CLOSE Received message may be received at any time while in this state.

7.14.4.6.2 Transition SL_CC4:DisconnectWait to SL_CC0:Idle
This transition shall occur after:
a) sending a Transmit CLOSE message to the SL transmitter;
b) receiving a CLOSE Received message; and
c) sending a Connection Closed (Normal) confirmation to the port layer.
[Editor's note: I hate these implied "shall send Connection Closed" rules
that are not directly stated, but leave them in this proposal.]

7.14.4.6.3 Transition SL_CC4:DisconnectWait to SL_CC5:BreakWait
This transition shall occur if:
a) a BREAK Received message has not been received;
b) no CLOSE Received message is received in response to a Transmit CLOSE
message before the Close Timeout timer expires; and
c) after sending a Connection Closed (Close Timeout) confirmation to the
port layer.

7.14.4.6.4 Transition SL_CC4:DisconnectWait to SL_CC6:Break
This transition shall occur after receiving a BREAK Received message and
after sending a Connection Closed
(Break Received) confirmation to the port layer.


7.14.4.n SL_CC7:Disconnect state [new section]
7.14.4.n.1 State description
This state closes the connection and releases all resources associated with
the connection.

Upon entry to this state, this state shall send Enable Disable SSP
(Disable), Enable Disable SMP (Disable),
and Enable Disable STP (Disable) messages to the SSP, SMP, and STP link
layer state machines.

This state shall:
1) send a Transmit CLOSE (Normal) message or Transmit CLOSE (Clear
Affiliation) message to the SL transmitter; and
2) send at least three Transmit Idle Dword messages to the SL transmitter;
and
3) send a Connection Closed (Normal) confirmation to the port layer.

7.14.4.n.2 Transition SL_CC4:Disconnect to SL_CC0:Idle
This transition shall occur after sending a Connection Closed (Normal)
confirmation.


--
Rob Elliott, elliott at hp.com
Hewlett-Packard Industry Standard Server Storage Advanced Technology
https://ecardfile.com/id/RobElliott





*
* For T10 Reflector information, send a message with
* 'info t10' (no quotes) in the message body to majordomo at t10.org
*
* For T10 Reflector information, send a message with
* 'info t10' (no quotes) in the message body to majordomo at t10.org




More information about the T10 mailing list