UAS data-in sent to status synchronization comment/issue

Bigstone bigstone.gm at hscjpn.co.jp
Tue Feb 9 20:22:40 PST 2010


* From the T10 Reflector (t10 at t10.org), posted by:
* "Bigstone" <bigstone.gm at hscjpn.co.jp>
*
Dear Sirs,
I had been sick in bed yesterday, so I don't know last discussion about this 
issue.
Our device firmware engineer talked me following comments.
----------------------------------------------------------------------------
Please see my comments as below(in blue)
----------------------------------------------------------------------------
MCCI's comments
Point 1
- UAS requires that the status ERDY for SuperSpeed operation is not issued 
for the status endpoint until after the last transaction  for the Data-in 
pipe (for a read or bi-directional command) has been acknowledged by the 
host.
Yes, according to UAS "4.3 Data transfers".
UAS "4.3 Data transfers" defines:
After the last byte of data is transferred and acknowledged, the UAS target 
device shall return a SENSE IU on the Status pipe to indicate command 
completion. After the command is complete, the associated Data-out pipe or 
Data-in pipe may be used to transfer data for a different command.
Point 2
- This synchronization requirement requires that USB3 device silicon 
indicate to the device firmware/logic the receipt of the host ACK for the 
data-in pipe.
Agreed
Point 3
- Several very popular device IP cores and commercial USB 2.0 silicon all 
lack the ability to let you quickly and conveniently know that data is 
really transferred to the host on an IN pipe.  You get an interrupt when 
data has been loaded into the FIFO, and the core takes care of moving to the 
host.
Agreed
Point 4
- These cores are used in things like cell phones, media players, and other 
devices that are multi-function devices, presenting mass storage plus other 
functions to the host PC as composite devices.	 Virtually every cell phone 
shipped today uses one of these general purpose cores, and offers mass 
storage; so this is not a trivial or low-volume use case.
Agreed
Point 5
- This lack of precision for data transfer has not been a problem in 
practice until now (for UAS) -- so the new USB3 cores will not require this 
feature, and this will force use of BOT instead of UAS.
We hope not...
Point 6
- Since the reporting of the completion events of transfer requests of data 
and status endpoints to the UAS driver can occur in any order in any case 
(due to xHCI interrupt reporting and OS scheduling variations),
Agreed
Point 7
- this seems like an unnecessary requirement for the transactions on the 
SuperSpeed bus.
We probably need to discuss some more.
Point 8
- Note that this will also be an issue for USB2 UAS operation.	The ERDY is 
not present, but the device shall not respond with the STATUS_IU on the 
status pipe until after the host ACK on the last data-in transaction.  And 
lots of current silicon doesn't have support for that transition.
Agreed
Point 9
- Further, in USB2, the data-in ACK can be smashed.  If the ACK is smashed, 
the device won't respond with the STATUS_IU; as there is no normal retry on 
ACK smash in USB2.  This smash case can be recovered if there is a 
subsequent data-in transaction (as it will be detected by data toggle 
state).  However, since the next transfer (for the next command) isn't 
allowed until the status IU is returned, the device will just wait.
Agreed
Point 10
Presumably, the host (driver) will timeout the command; but this could turn 
a successful command into an error case with a long delay.
Agreed
----------------------------------------------------------------------------
NVIDIA's comments
Point 1
- If you transmit status on another pipe before receiving acknowledgement, 
it is possible for the host to send an indication of completion of status 
before receiving all of the data. This could lead to data corruption, system 
crashes, and all sort of bad behavior.	Especially in multi-threaded, 
multi-processor systems.
According to MCCI's Point 6, above may happen even if you transmit status on 
another pipe AFTER receiving acknowledgement.  Shouldn't it be host's 
responsibility to receive(attempt to receive) all of the data before 
processing command completion status?
Point 2(in response to MCCI's Point 9)
- In your USB-2 case, if the ACK does get smashed - the host will issue the 
next command (or commands) they will eventually time out, and error recovery 
begins. This is ok.
I can think of two cases.  Current UAS spec is the first case.
Case 1: If USB2.0/1.1 IP cores/silicons CAN detect ACK smash, the device 
will not transmit status until device receiving ACK for the last packet.  If 
ACK for the last packet is smashed, host thinks all data has been 
transferred but device thinks all data has NOT been transferred and 
acknowledged. Then device will not transmit the status and host waits for 
the status and eventually times out.
Case 2: If USB2.0/1.1 IP cores/silicons CAN'T detect ACK smash, the device 
will transmit status believing data transfer is done(i.e. all the data is in 
FIFO).	If ACK for the last packet is smashed, both host and device thinks 
all data has been transferred. Then host will detect wrong data toggle 
state(because device thinks the data transfer for the last packet is not 
done yet) on next data transfer(next command).	Is it host's responsibility 
to discard a packet with wrong data toggle?
----------------------------------------------------------------------------
Best Regards,
Yuji Oishi
HAGIWARA SYS-COM Co., Ltd.
Hamamatsu R&D Lab.
>* From the T10 Reflector (t10 at t10.org), posted by:
> * John Garney <jig at mcci.com>
> *
> Mark,
> My only "position" was to point out possible issues with the current
> UAS definition, and possible issues with current device silicon
> supporting that definition.  So I'm not sure what you are disagreeing
> with.
>
> Given the fact that the OS (and even possibly the host controller) can
> schedule the notification of transfer completions in any order across
> pipes (even if the transfers on those pipes occur in a specific order
> on the bus), you can't depend on the UAS class driver getting the data-
> in transfer completion notification for one pipe before the status
> transfer completion for another pipe.  So if out-of-order data-in/
> status pipe notifications can lead to data corruption, system crashes,
> etc. (that you refer to), you have that problem now for both UAS on
> HighSpeed and SuperSpeed.  And you are correct that depending on
> ordering of this sort in a multi-threaded, multi-processor system is
> even more of a problem.  That is exactly one of the problems to which
> I am trying to call attention.
>
> For the USB-2 case, as I described originally; the ACK smash can be
> detected by a timeout on the respective command.  Given that UAS
> allows command "queuing", the ACK smash is independent of the issuing
> of other commands; i.e. additional commands can "always" be issued
> (subject to availability of target resources).  So, yes, I suppose
> other commands could also timeout if the error recovery takes long
> enough.   But the error recovery of the ACK smash would be triggered
> by the timeout of its respective command (not some other command).
>
> My point for the ACK smash case was that since the UAS driver can't
> rely on ordered transfer completions between data-in and status pipes,
> why require it on the bus; especially if it further stimulates
> additional unnecessary error recovery in the ACK smash corner case?
> If you didn't require such ordering on the bus; the status transfer
> would complete, the data transfer would complete (unbeknownst to the
> host, the ACK was smashed during transmission to the device).  The
> next command would start processing, and the next data-in transaction
> would automatically recover the (bus level) data-in ACK smash due to
> data-toggle detection.  No "command level" error recovery would be
> done nor required.
>
> If UAS requires its host driver to receive and process status/data-in
> completion notifications in the order in which they occur on the bus,
> I don't see how that can be achieved with current operating systems.
> Well, unless you don't allow the UAS driver to issue the status
> transfer request until after it has received the data-in completion.
> But doing that requires a round trip through the driver between every
> data/status sequence, which seems to defeat the intended benefit of
> the performance improvement potential of UAS.
>
> I also pointed out that the data-in ACK notification (before allowing
> the status pipe transaction) on the device side is not supported with
> popular current device HighSpeed silicon.
>
>
> On Feb 8, 2010, at 9:28 AM, Mark Overby wrote:
>
>> * From the T10 Reflector (t10 at t10.org), posted by:
>> * Mark Overby <MOverby at nvidia.com>
>> *
>> I disagree with your position.
>>
>> If you transmit status on another pipe before receiving
>> acknowledgement, it is possible for the host to send an indication
>> of completion of status before receiving all of the data. This could
>> lead to data corruption, system crashes, and all sort of bad behavior.
>>
>> Especially in multi-threaded, multi-processor systems.
>>
>> In your USB-2 case, if the ACK does get smashed - the host will
>> issue the next command (or commands) they will eventually time out,
>> and error recovery begins. This is ok.
>>
>> ________________________________________
>> From: owner-t10 at t10.org [owner-t10 at t10.org] On Behalf Of John Garney
>> [jig at mcci.com]
>> Sent: Tuesday, February 02, 2010 7:46 AM
>> To: T10 at t10.org
>> Subject: UAS data-in sent to status synchronization comment/issue
>>
>> * From the T10 Reflector (t10 at t10.org), posted by:
>> * John Garney <jig at mcci.com>
>> *
>> Curtis,
>> UAS requires that the status ERDY for SuperSpeed operation is not
>> issued for the status endpoint until after the last transaction  for
>> the Data-in pipe (for a read or bi-directional command) has been
>> acknowledged by the host.  This synchronization requirement requires
>> that USB3 device silicon indicate to the device firmware/logic the
>> receipt of the host ACK for the data-in pipe.
>>
>> Several very popular device IP cores and commercial USB 2.0 silicon
>> all lack the ability to let you quickly and conveniently know that
>> data is really transferred to the host on an IN pipe.  You get an
>> interrupt when data has been loaded into the FIFO, and the core takes
>> care of moving to the host.
>>
>> These cores are used in things like cell phones, media players, and
>> other devices that are multi-function devices, presenting mass storage
>> plus other functions to the host PC as composite devices.   Virtually
>> every cell phone shipped today uses one of these general purpose
>> cores, and offers mass storage; so this is not a trivial or low-volume
>> use case.
>>
>> This lack of precision for data transfer has not been a problem in
>> practice until now (for UAS) -- so the new USB3 cores will not require
>> this feature, and this will force use of BOT instead of UAS.
>>
>> Since the reporting of the completion events of transfer requests of
>> data and status endpoints to the UAS driver can occur in any order in
>> any case (due to xHCI interrupt reporting and OS scheduling
>> variations), this seems like an unnecessary requirement for the
>> transactions on the SuperSpeed bus.
>>
>> Note that this will also be an issue for USB2 UAS operation.  The ERDY
>> is not present, but the device shall not respond with the STATUS_IU on
>> the status pipe until after the host ACK on the last data-in
>> transaction.  And lots of current silicon doesn't have support for
>> that transition.
>>
>> Further, in USB2, the data-in ACK can be smashed.  If the ACK is
>> smashed, the device won't respond with the STATUS_IU; as there is no
>> normal retry on ACK smash in USB2.  This smash case can be recovered
>> if there is a subsequent data-in transaction (as it will be detected
>> by data toggle state).  However, since the next transfer (for the next
>> command) isn't allowed until the status IU is returned, the device
>> will just wait.  Presumably, the host (driver) will timeout the
>> command; but this could turn a successful command into an error case
>> with a long delay.
>> /jig/
>>
>> *
>> * For T10 Reflector information, send a message with
>> * 'info t10' (no quotes) in the message body to majordomo at t10.org
>>
-----------------------------------------------------------------------------
------
>> This email message is for the sole use of the intended recipient(s)
>> and may contain
>> confidential information.  Any unauthorized review, use, disclosure
>> or distribution
>> is prohibited.  If you are not the intended recipient, please
>> contact the sender by
>> reply email and destroy all copies of the original message.
>>
-----------------------------------------------------------------------------
------
>>
>> *
>> * For T10 Reflector information, send a message with
>> * 'info t10' (no quotes) in the message body to majordomo at t10.org
>
> *
> * For T10 Reflector information, send a message with
> * 'info t10' (no quotes) in the message body to majordomo at t10.org 
*
* For T10 Reflector information, send a message with
* 'info t10' (no quotes) in the message body to majordomo at t10.org



More information about the T10 mailing list