[T10] [IMPORTANT] SPL-5 Specification queries : Expander phys forwarding dwords and deletable extended binary primitives

Tim Symons tim.symons at microsemi.com
Fri Jul 20 11:01:15 PDT 2018


Deleted images of tables to reduce E-mail size for T10 reflector

From: Tim Symons
Sent: Friday, July 20, 2018 10:15 AM
To: 'Mansi Chadha' <Mansi.Chadha at synopsys.com>; t10 at t10.org
Cc: Elliott, Robert (Persistent Memory) <elliott at hpe.com>
Subject: RE: [IMPORTANT] SPL-5 Specification queries : Expander phys forwarding dwords and deletable extended binary primitives

Mansi,
The important aspect of deletable primitives is that there must be at least 1 deletable primitive every 128 dwords.
When forwarding, an expander receiver will require 1:128 deletable primitives to satisfy it's elasticity buffers, but when the data is forwarded then the transmitter is required to ensure that there is at least 1:128 deletable primitives which it will do by inserting or removing deletable primitives.

The text for this is in section 6.5.4

6.5.4 Expander phys forwarding dwords and deletable extended binary primitives
An expander device that is forwarding dwords (i.e., is not originating dwords) is allowed to insert or delete as many deletable primitives or deletable extended binary primitives as required to match the transmit and
receive connection rates.
<..>
An expander device shall increase or reduce the number of deletable primitives, deletable binary primitives, or deletable extended binary primitives based on clock frequency differences between the expander device's receiving phy and the expander device's transmitting phy (e.g., if receiving at -100 ppm and transmitting at +100 ppm, then it transmits more deletable primitives than it receives).
The expander device is also required to insert deletable primitives or scrambled idle segments for rate matching (see 6.17). During an STP connection, the expander device shall:

a)       preserve the incoming rate of any additional deletable primitives or deletable extended binary primitives that it receives that are not discarded because of physical link rate tolerance management or rate matching (e.g., the 1/128 deletable primitives received from an originating STP initiator phy compliant with SAS-1.1 for STP initiator phy throttling); or

b)      transmit one deletable primitive within every 128 dwords, without discarding any data dwords or primitives.

Regards
Tim

From: Mansi Chadha <Mansi.Chadha at synopsys.com<mailto:Mansi.Chadha at synopsys.com>>
Sent: Friday, July 20, 2018 12:57 AM
To: Tim Symons <tim.symons at microsemi.com<mailto:tim.symons at microsemi.com>>; Mansi Chadha <Mansi.Chadha at synopsys.com<mailto:Mansi.Chadha at synopsys.com>>; t10 at t10.org<mailto:t10 at t10.org>
Cc: Elliott, Robert (Persistent Memory) <elliott at hpe.com<mailto:elliott at hpe.com>>
Subject: RE: [IMPORTANT] SPL-5 Specification queries : Expander phys forwarding dwords and deletable extended binary primitives

EXTERNAL EMAIL
Hi Tim,

Thanks for your response.

Specs here talks about for  "Phys which originating dwords" (6.5.2 Phys originating dwords while in the SAS dword mode).
What I don't fully get is and what specs doesn't say explicitly is for the "Phys which are forwarding dwords".

Do below tables (table 159 and table 160) are also applicable when phys are forwarding?

Please confirm.

Thanks & Regards,
Mansi


From: Tim Symons [mailto:tim.symons at microsemi.com]
Sent: Thursday, July 19, 2018 7:23 PM
To: Mansi Chadha <Mansi.Chadha at synopsys.com<mailto:Mansi.Chadha at synopsys.com>>; t10 at t10.org<mailto:t10 at t10.org>
Cc: Elliott, Robert (Persistent Memory) <elliott at hpe.com<mailto:elliott at hpe.com>>
Subject: RE: [IMPORTANT] SPL-5 Specification queries : Expander phys forwarding dwords and deletable extended binary primitives

Mansi,

Yes the rate of inserting deletable primitives is the same to ensure elasticity buffers do not overrun. The requirements for deletable primitives is defined in table 159 for dword mode and 160 for SPL packet mode.


<Table 159>

<table 160>


All SAS-4 devices are required to be capable of negotiating down to SAS-3 or SAS-2 link rates, which also means they are capable of supporting dword operation.

Given this background you can see that the elasticity buffering and requirements for deletable primitives are aligned with dword mode of operation, and the SPL packets are required to retain compatibility to those definitions.

I hope this helps to answer your questions.

Regards
Tim Symons
(Editor SPL-5)


Tim Symons | Storage Architect
Microsemi Corporation
Tim.Symons at microchip.com<mailto:Tim.Symons at microchip.com>

[Microsemi_Subsidiary_Logo_4C_BLK (003)]





From: Mansi Chadha <Mansi.Chadha at synopsys.com<mailto:Mansi.Chadha at synopsys.com>>
Sent: Wednesday, July 18, 2018 11:12 PM
To: t10 at t10.org<mailto:t10 at t10.org>
Cc: Tim Symons <tim.symons at microsemi.com<mailto:tim.symons at microsemi.com>>; Elliott, Robert (Persistent Memory) <elliott at hpe.com<mailto:elliott at hpe.com>>
Subject: [IMPORTANT] SPL-5 Specification queries : Expander phys forwarding dwords and deletable extended binary primitives

EXTERNAL EMAIL
Hi,

This is regarding the similar query I raised a few days back on the below section of spl5r04:

6.5.4 Expander phys forwarding dwords and deletable extended binary primitives
matching (see 6.17). During an STP connection, the expander device shall:
a) preserve the incoming rate of any additional deletable primitives or deletable extended binary
primitives that it receives that are not discarded because of physical link rate tolerance management
or rate matching (e.g., the 1/128 deletable primitives received from an originating STP initiator phy
compliant with SAS-1.1 for STP initiator phy throttling); or
b) transmit one deletable primitive within every 128 dwords, without discarding any data dwords or
primitives.

Here, it says about (--- seems for dword mode) :

"During an STP connection, the expander device shall :
b) transmit one deletable primitive within every 128 dwords, without discarding any data dwords or
primitives"

However, it doesn't say about for packet mode scenario in this case.
Kindly confirm if the above specification point is also valid for packet mode and how?

Will appreciate a quick response.

Thanks and Regards,
Mansi


From: Mansi Chadha
Sent: Wednesday, July 18, 2018 12:42 PM
To: 'Tim Symons' <tim.symons at microsemi.com<mailto:tim.symons at microsemi.com>>; Mansi Chadha <Mansi.Chadha at synopsys.com<mailto:Mansi.Chadha at synopsys.com>>; Elliott, Robert (Persistent Memory) <elliott at hpe.com<mailto:elliott at hpe.com>>; t10 at t10.org<mailto:t10 at t10.org>
Cc: Monika Talwar <monika.talwar at synopsys.com<mailto:monika.talwar at synopsys.com>>
Subject: RE: [T10] [IMPORTANT] SPL-5 Specification queries :Physical link rate tolerance management requirements : (SPL-5 Revision 04 16 April 2018) : 6.19.2 XL transmitter and receiver

Thanks Tim for a quick response.
It clarifies the concept.

Also spec doesn't clearly say for the packet mode. Does that mean this rule also applies for packet mode too as in 4 deletable packets within every 512 packets?

Regards,
Mansi

From: Tim Symons [mailto:tim.symons at microsemi.com]
Sent: Monday, July 16, 2018 9:12 PM
To: Mansi Chadha <Mansi.Chadha at synopsys.com<mailto:Mansi.Chadha at synopsys.com>>; Elliott, Robert (Persistent Memory) <elliott at hpe.com<mailto:elliott at hpe.com>>; t10 at t10.org<mailto:t10 at t10.org>
Cc: Monika Talwar <monika.talwar at synopsys.com<mailto:monika.talwar at synopsys.com>>
Subject: RE: [T10] [IMPORTANT] SPL-5 Specification queries :Physical link rate tolerance management requirements : (SPL-5 Revision 04 16 April 2018) : 6.19.2 XL transmitter and receiver

The expander is required to transit at least one deletable primitive ever 128 dwords to ensure sufficient elasticity.
When a deletable primitive is received on an ingress phy, it may or may not get deleted by the ingress phy.
If it is not deleted then it may be transmitted on the egress phy. Regardless of whether a deletable primitive is forwarded or originated, the transmitter phy must ensure that the rate of deletable primitives is at least one every 128 dwords. It can be more.

I hope this helps to add clarity for this issue.
Regards
Tim.

From: Mansi Chadha <Mansi.Chadha at synopsys.com<mailto:Mansi.Chadha at synopsys.com>>
Sent: Monday, July 16, 2018 8:26 AM
To: Tim Symons <tim.symons at microsemi.com<mailto:tim.symons at microsemi.com>>; Mansi Chadha <Mansi.Chadha at synopsys.com<mailto:Mansi.Chadha at synopsys.com>>; Elliott, Robert (Persistent Memory) <elliott at hpe.com<mailto:elliott at hpe.com>>; t10 at t10.org<mailto:t10 at t10.org>
Cc: Monika Talwar <monika.talwar at synopsys.com<mailto:monika.talwar at synopsys.com>>
Subject: RE: [T10] [IMPORTANT] SPL-5 Specification queries :Physical link rate tolerance management requirements : (SPL-5 Revision 04 16 April 2018) : 6.19.2 XL transmitter and receiver

EXTERNAL EMAIL
Hi Tim,

That is true, however does the keyword "any additional" means extra deletable primitive apart from link rate management and rate matching?

Reference  :
a) preserve the incoming rate of any additional deletable primitives or deletable extended binary
primitives that it receives that are not discarded because of physical link rate tolerance management
or rate matching

Thanks,
Mansi
From: Tim Symons [mailto:tim.symons at microsemi.com]
Sent: Monday, July 16, 2018 8:40 PM
To: Mansi Chadha <Mansi.Chadha at synopsys.com<mailto:Mansi.Chadha at synopsys.com>>; Elliott, Robert (Persistent Memory) <elliott at hpe.com<mailto:elliott at hpe.com>>; t10 at t10.org<mailto:t10 at t10.org>
Cc: Monika Talwar <monika.talwar at synopsys.com<mailto:monika.talwar at synopsys.com>>
Subject: RE: [T10] [IMPORTANT] SPL-5 Specification queries :Physical link rate tolerance management requirements : (SPL-5 Revision 04 16 April 2018) : 6.19.2 XL transmitter and receiver

Mansi,
Yes. The expander phys are the same as all other SAS phys require link rate management to ensure their elasticity buffers are correctly maintained.

Tim.

From: Mansi Chadha <Mansi.Chadha at synopsys.com<mailto:Mansi.Chadha at synopsys.com>>
Sent: Monday, July 16, 2018 3:58 AM
To: Mansi Chadha <Mansi.Chadha at synopsys.com<mailto:Mansi.Chadha at synopsys.com>>; Elliott, Robert (Persistent Memory) <elliott at hpe.com<mailto:elliott at hpe.com>>; Tim Symons <tim.symons at microsemi.com<mailto:tim.symons at microsemi.com>>; Mansi Chadha <Mansi.Chadha at synopsys.com<mailto:Mansi.Chadha at synopsys.com>>; t10 at t10.org<mailto:t10 at t10.org>
Cc: Monika Talwar <monika.talwar at synopsys.com<mailto:monika.talwar at synopsys.com>>
Subject: RE: [T10] [IMPORTANT] SPL-5 Specification queries :Physical link rate tolerance management requirements : (SPL-5 Revision 04 16 April 2018) : 6.19.2 XL transmitter and receiver

EXTERNAL EMAIL
Hi Robert,

I have one more query :

Should expander always send one align per every 128 dwords in addition to physical link rate tolerance management aligns and rate matching

As per : slp5r04
6.5.4 Expander phys forwarding dwords and deletable extended binary primitives
matching (see 6.17). During an STP connection, the expander device shall:
a) preserve the incoming rate of any additional deletable primitives or deletable extended binary
primitives that it receives that are not discarded because of physical link rate tolerance management
or rate matching (e.g., the 1/128 deletable primitives received from an originating STP initiator phy
compliant with SAS-1.1 for STP initiator phy throttling); or
b) transmit one deletable primitive within every 128 dwords, without discarding any data dwords or
primitives.
The expander device may reduce the length of repeated primitive sequences (i.e., primitive, SATA_CONT,
and data dword sequences).
NOTE 18 - One possible implementation for expander devices forwarding dwords is for the expander device
to delete all deletable primitives received and to insert deletable primitives at the transmit phy whenever its
elasticity buffer is empty.


Please update on this as well.

Thanks,
Mansi

From: Mansi Chadha [mailto:mansi at synopsys.com]
Sent: Monday, July 16, 2018 2:44 PM
To: Elliott, Robert (Persistent Memory) <elliott at hpe.com<mailto:elliott at hpe.com>>; 'Tim Symons' <tim.symons at microsemi.com<mailto:tim.symons at microsemi.com>>; Mansi Chadha <Mansi.Chadha at synopsys.com<mailto:Mansi.Chadha at synopsys.com>>; t10 at t10.org<mailto:t10 at t10.org>
Subject: RE: [T10] [IMPORTANT] SPL-5 Specification queries :Physical link rate tolerance management requirements : (SPL-5 Revision 04 16 April 2018) : 6.19.2 XL transmitter and receiver

Hi Robert,

Thanks for your response and clarification on Physical link rate tolerance management requirements.
I appreciate T10 for taking up this query and resolve quickly.

Best Regards,
Mansi

From: Elliott, Robert (Persistent Memory) [mailto:elliott at hpe.com]
Sent: Friday, July 13, 2018 9:01 PM
To: 'Tim Symons' <tim.symons at microsemi.com<mailto:tim.symons at microsemi.com>>; Mansi Chadha <Mansi.Chadha at synopsys.com<mailto:Mansi.Chadha at synopsys.com>>; t10 at t10.org<mailto:t10 at t10.org>
Subject: RE: [T10] [IMPORTANT] SPL-5 Specification queries :Physical link rate tolerance management requirements : (SPL-5 Revision 04 16 April 2018) : 6.19.2 XL transmitter and receiver

>     b) when transmitting the last idle dword before a connection is established (i.e., after receiving OPEN_ACCEPT);
> Should XL transmitter insert a deletable primitive (dword mode) or LRM (link rate management, packet mode) just before the last idle dword or after the last idle dword

After.

If the forwarded stream includes the worst-case number of dwords before a forwarded deletable primitive occurs, then the dword immediately before the first forwarded dword needs to be deletable.  Otherwise, you'd have one more dword than allowed between deletable primitives, which could result in a buffer overflow and losing one of the dwords (if there is a worst-case combination of ppm rates, buffer sizes, etc.).

---
Robert Elliott, HPE Persistent Memory



From: t10-bounces at t10.org<mailto:t10-bounces at t10.org> [mailto:t10-bounces at t10.org] On Behalf Of Tim Symons
Sent: Thursday, July 12, 2018 7:18 PM
To: Mansi Chadha <Mansi.Chadha at synopsys.com<mailto:Mansi.Chadha at synopsys.com>>; t10 at t10.org<mailto:t10 at t10.org>
Subject: Re: [T10] [IMPORTANT] SPL-5 Specification queries :Physical link rate tolerance management requirements : (SPL-5 Revision 04 16 April 2018) : 6.19.2 XL transmitter and receiver

Mansi,
I will bring your enquiry to the attention of the T10 working group for review at the next meeting (17-19 July).

I don't believe that the precise location is particularly critical as will never cause an error event. It really should not matter if it is inserted before or after the final dword (or SPL packet), the objective is to ensure that there is a deletable primitive prior to the longer sequences described in the standard.

My opinion is driven by Note 36:

"NOTE 36 - This ensures that physical link rate tolerance management requirements are met, even if the
forwarded dword stream does not include a deletable primitive until the last possible dword."

I will ask the group to comment whether there is a need for further definition to clarify the purpose of this requirement, and provide feedback to you by E-mail.

Regards
Tim Symons
T10 Editor of SPL-5


Tim Symons | Storage Architect
Microsemi Corporation
8555 Baxter Place, Burnaby, BC V5A 4V7. Canada
Tel. 604 415 6000
Tim.Symons at microsemi.com<mailto:Tim.Symons at microsemi.com>

[Microsemi_Subsidiary_Logo_4C_BLK (003)]
http://www.microsemi.com/<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.microsemi.com_&d=DwMFAg&c=DPL6_X_6JkXFx7AXWqB0tg&r=4K_qy8eQTNSAPJd0gTGscqyZfk7xXUwmZFuOPX8n0nM&m=m1BOXYN1KrD-YsRA9sGJlqBT2qy0AXWbtNB6KuiF-lQ&s=xzQOEIksgWHWWYM1WMSAqmVOawGwm3wd52x1doh2c6k&e=>


From: t10-bounces at t10.org<mailto:t10-bounces at t10.org> <t10-bounces at t10.org<mailto:t10-bounces at t10.org>> On Behalf Of Mansi Chadha
Sent: Thursday, June 28, 2018 10:33 AM
To: t10 at t10.org<mailto:t10 at t10.org>
Subject: [T10] [IMPORTANT] SPL-5 Specification queries :Physical link rate tolerance management requirements : (SPL-5 Revision 04 16 April 2018) : 6.19.2 XL transmitter and receiver

EXTERNAL EMAIL

Hi,

I have certain queries on below section of the specification (SPL-5 Revision 04 16 April 2018) on  the physical link rate tolerance management requirements.
These queries are inline highlighted in blue and bolded. Please suggest the valid expectation as per specification interpretation.

6.19.2 XL transmitter and receiver

The XL transmitter shall insert a deletable primitive or a deletable extended binary primitive before switching
from originating dwords or originating SPL packets to forwarding dwords or forwarding SPL packets, including,
for example:

b) when transmitting the last idle dword before a connection is established (i.e., after receiving
OPEN_ACCEPT);

Ø  Should XL transmitter insert a deletable primitive (dword mode) or LRM (link rate management, packet mode) just before the last idle dword or after the last idle dword


c) while transmitting a SATA frame to a SAS logical link during an STP connection, when transmitting the
last dword from the STP flow control buffer in response to release of SATA_HOLD;

d) while transmitting a SATA frame to a SAS logical link during an STP connection, when transmitting the
last SATA_HOLDA in response to release of SATA_HOLD (e.g., if the STP flow control buffer is
empty); and

Ø  XL transmitter is expected to insert a deletable primitive (dword mode) or LRM (link rate management, packet mode)  after last SATA_HOLDA in response to release of SATA_HOLD

Ø  OR

Ø  XL transmitter is expected to insert a deletable primitive (dword mode) or LRM (link rate management, packet mode)  before last SATA_HOLDA in response to release of SATA_HOLD


e) while receiving dwords of a SATA frame from a SAS logical link during an STP connection, when
transmitting the last SATA_HOLD.

Ø  XL transmitter is expected to insert a deletable primitive (dword mode) or LRM (link rate management, packet mode) after last SATA_HOLD

Ø  OR

Ø  XL transmitter is expected to insert a deletable primitive (dword mode) or LRM (link rate management, packet mode)  before last SATA_HOLD



Also, apart from above mentioned examples we have identified some other scenarios which could be possible switches from originating to forwarding for XL transmitter as below:
Please suggest if otherwise.

1)       Transmitting SATA_SOF after X_RDY is one of the switch case from Originating to forwarding which is not listed in examples list.

a.       X_RDY followed by CONT is to be treated as Originating sequence and for transmitting SATA_SOF, Expander should switch to forwarding Phy, and put a deletable primitive (dword mode) or LRM (link rate management, packet mode) before SATA_SOF

b.       X_RDY (not followed by CONT) is to be treated as Originating sequence and for transmitting SATA_SOF, Expander should switch to forwarding Phy, and put a deletable primitive (dword mode) or LRM (link rate management, packet mode) before SATA_SOF

c.        X_RDY followed by CONT and then X_RDY, is to be treated as Originating sequence and for transmitting SATA_SOF, Expander should switch to forwarding Phy, and put a deletable primitive (dword mode) or LRM (link rate management, packet mode) before SATA_SOF

                                                               i.      E.g. SATA_X_RDY -> SATA_X_RDY -> SATA_CONT -> SATA_X_RDY -> SATA_SOF

2)       Read DATA transaction(DATA FIS is transmitted by Expander to STP Initiator) "

a.       Expander shall put a deletable primitive (dword mode) or LRM (link rate management, packet mode) while transmitting the  last SATA_HOLD i.e. before the last SATA_HOLD or after the last SATA_HOLD?



3)       while receiving dwords of a SATA frame from a SAS logical link during an STP connection, when

transmitting the last SATA_HOLD

a.       Write DATA transaction(DATA FIS is transmitted by STP initiator to  Expander) "

                                                               i.      Expander shall put a deletable primitive (dword mode) or LRM (link rate management, packet mode) while transmitting the  last SATA_HOLD i.e. before the last SATA_HOLD or after the last SATA_HOLD?


Please suggest the specification (SPL-5) interpretation for the above mentioned points related to physical link rate tolerance management requirements.
Hope to receive your response as soon as possible.

Thanks & Regards,
Mansi





-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.t10.org/pipermail/t10/attachments/20180720/ea06131a/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.jpg
Type: image/jpeg
Size: 5152 bytes
Desc: image003.jpg
URL: <http://www.t10.org/pipermail/t10/attachments/20180720/ea06131a/attachment-0002.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image004.jpg
Type: image/jpeg
Size: 5163 bytes
Desc: image004.jpg
URL: <http://www.t10.org/pipermail/t10/attachments/20180720/ea06131a/attachment-0003.jpg>


More information about the T10 mailing list