Link Rate within Wide Link
pak-lung.seto at intel.com
Thu Oct 17 06:07:06 PDT 2002
* From the T10 Reflector (t10 at t10.org), posted by:
* "Seto, Pak-lung" <pak-lung.seto at intel.com>
It is not as simple as just another field in the routing table.
>From the initiator's point of view:
It is true that Initiator has a choice of selecting what "end-to-end" link
rate it use in the OPEN Address Frame. But I don't believe the "HBA" will
have enough intelligent to decide what link rate it is going to use for a
particular connection for a particular IO. If the choice has to be made, I
believe it will be the Application's decision. I am not sure there is a
communication exist to pass "link rate" from Application to the HBA or the
"link rate" information pass up to the application from HBA.
Even if the initiator/application wants to choose a "link rate" for a
particular IO to a target. There can be multiple wide links segments
between the initiator and the target. The decision making process will be
much more complicated. The application will no longer be transport
independent unless the HBA can figure out how it can make decision of what
link rate should be used for a particular IO.
By default, I would think the initiator will always use the lowest
negotiated "link rate" of all link segments to a particular target.
>From the expander's point of view:
There is no need for the expander to look into the "link rate" field of the
OPEN Address frame to do routing IF the lowest link rate of a wide link
group is used, because the expander is already know the negotiated link rate
of each PHY within the expander.
If we allow the initiator to choose higher negotiated link rate within a
wide link group. The expander will have to look at the "link rate" field of
the OPEN Address Frame to sort out which link within a particular wide links
group are allowed for this particular connection request.
Since (I believe) the initiator will always use the lowest link rate within
a wide link group, the expander will have function which will never be used
in real life application but it has to put in such functionality because the
standard allowed. It will increase expander's complexity and verification
for no return on investment.
I would think for the standard to require connection request to use the
lowest negotiated link rate within a wide links group will provide more
predictable system behavior and better for interoperability. It will also
simplify expander complexity and eliminate functionality that may never be
There is another issue that the standard has not been addressed. Since the
target (disk drive) is not required to support SMP to perform discovery.
Target SHALL be required to "remember" the "link rate" from the OPEN Address
Field of a particular initiator. Will the target required to "remember" the
"link rate" on a "per IO" base, per "connection - since the last connection"
or per initiator base, since the standard allows the initiator to use
different "link rate" in the OPEN Address Frame to connect to a target.
There are two situations that target may return information back to the
- Connection initiated by the initiator: in this situation, the target SHALL
use the "link rate" information to return data back to the initiator during
- Connection initiated by the target: in this situation, the target MUST use
SOME SAVED "link rate" associated to that particular initiator. What "link
rate" the target must be used?
Since a connection can transfer information from more than one IO. I would
think the "link rate" will not be associated with IO but with
"initiator/target" pair. In order to fully utilize all the available links,
the initiator will not have a choice but to use the lowest negotiated link
rate within a wide links group, because the target will need to "remember"
the link rate to return data.
This leads to another issue that is being discussed in the WG/Conf. call.
Since the target is required to remember the "link rate" for returning
connection to the initiator. If there is a link reset of a particular link
within a wide links group. I would think the CHANGE primitive may need to
send to all links within the wide links group to notify the link that the
"saved" link rate may no longer be valid (because the target may manage the
connection link rate by hardware). The CHANGE primitive will also have to
send to both initiator/target (target especially) and not just initiator.
The target will have to temporary reset the "saved" link rate to the minimum
rate (1.5 Gb/s as defined by the standard), it does not require the link to
be operated at the minimum rate and it only affect the "end-to-end link
rate". The target SHALL use this temporary end-to-end link rate to make
connection back to the initiator until new connection has been initiated
|from initiator to the target in order for the target to "save" the new value
of the "link rate" because the target may be holding the "transfer
initiative" (in FC term) and the initiator is waiting to hear from the
target first during the "CHANGE" happens before it can initiate a connection
back to the target.
From: Elliott, Robert (Server Storage) [mailto:Elliott at hp.com]
Sent: Wednesday, October 16, 2002 6:36 PM
To: Seto, Pak-lung
Subject: RE: How many SAS addresses should an expander has?
> The expander never reports a link rate for the port; it
> reports everything on a phy by phy basis. If a wide link consists of
> physical links running at different rates, the initiator will know.
> Assume there is a wide link with both 1.5 Gbps and 3.0 Gbps
> physical links. If an initiator sends an OPEN (3.0) that
> needs to use that wide link, that connection can only use a
> 3.0 Gbps physical link
> of that wide link. If the initiator sends an OPEN (1.5), the
> connection may use a 1.5 Gbps link or a 3.0 Gbps link (with
> rate matching by the expanders).
> [Seto] This will make the expander a little bit more complex to build
It's basically another field in the routing table lookups; not too
bad compared to other issues.
* For T10 Reflector information, send a message with
* 'info t10' (no quotes) in the message body to majordomo at t10.org
More information about the T10