FW: How do SAS self-configuring expanders self configure?

Tim Symons Tim_Symons at pmc-sierra.com
Mon Jan 30 10:39:49 PST 2006


* From the T10 Reflector (t10 at t10.org), posted by:
* Tim Symons <Tim_Symons at pmc-sierra.com>
*


It is unnecessary for managment application clients to send BROADCAST (CHANGE) on phys with an attached device type of 001b.

I think that the timer suggestion, it to provide a solution to the case where a CONFIGURING bit is set to one in the REPORT GENERAL response, and the managment application client is required to wait for the BROADCAST (CHANGE) notification in order to issue a REPORT GENERAL request to test whether the CONFIGURING bit is zero. 

HBA management application client software has to deal with this condition today with SAS1.1. There is no difference, however a failure condition may exist and does not necessarily get resolved in the current definition on how a managment application client is defined.

Regards
     Tim Symons.

Tim Symons
Principal Engineer
PMC-Sierra Ltd.
Burnaby

Cell: 778 998 5025
E-mail Tim_Symons at pmc-sierra.com


-----Original Message-----
From: owner-t10 at t10.org [mailto:owner-t10 at t10.org]On Behalf Of Johnson,
Steve
Sent: Friday, January 27, 2006 6:00 PM
To: t10 at t10.org
Subject: RE: How do SAS self-configuring expanders self configure?


* From the T10 Reflector (t10 at t10.org), posted by:
* "Johnson, Steve" <Steve.Johnson at lsil.com>
*
I'm encouraged to see that more folks are in agreement that
self-configuration or "expander based configuration" needs better
definition. I also hope that we can agree that it can be performed in a
more efficient manner than SAS 1.1 might suggest. 

I think some other goals for SAS-2 are:

	1) Improve scalability 
	2) Improve efficiency 
	3) Insure interoperability
	4) Provide reporting and access for validation and debug

I don't see why an expander that uses "expander based configuration"
would bother sending BROADCAST (CHANGE) to end devices to inform it that
"something" changed in the topology and then make them wait to receive
yet another BROADCAST (CHANGE) after the expander has completed
configuration??? The behavior should be to wait until configuration is
complete before sending BROADCAST (CHANGE) to end devices. Adding a
timer to this seems even more problematic.

The discovery process for an end device would be:

>Receive BROADCAST (CHANGE)
>Send SMP Report General Request
>Check self-configuration bit for zero
>If not, start timer
>Wait for another BROADCAST (CHANGE)
>Send SMP Report General Request
>Check self-configuration bit zero
>....
>Once self configuration is zero (stop timer)
>Continue process for each expander within the zone

It seems highly inefficient for all the other expanders to then go make
sure everyone is done configuring and have things "thrash" for a while
before setting down.

and all this happens for each BROADCAST (CHANGE) which we keep sending
to inform us that things have settled down...

As topologies grow to hundreds of end devices this process does not
scale well and the trashing just gets worse.

I agree it works buts it's the "long and winding road" with some traffic
jams along the way.

If we are going through all the trouble of having to elect a supervisor
and not allow end devices access to the domain until zoning election and
zoning configuration is complete which cannot happen until there is an
elected supervisor.... why not just get right to it and efficiently
elect a supervisor and have it configure the topology with a little help
|from it's supervisor capable buddies and stop all the unnecessary
thrashing and superfluous BROADCAST (CHANGE)?

Steve



-----Original Message-----
From: owner-t10 at t10.org [mailto:owner-t10 at t10.org] On Behalf Of Elliott,
Robert (Server Storage)
Sent: Friday, January 27, 2006 11:49 AM
To: t10 at t10.org
Subject: RE: How do SAS self-configuring expanders self configure?

* From the T10 Reflector (t10 at t10.org), posted by:
* "Elliott, Robert (Server Storage)" <Elliott at hp.com>
*
One of the goals of SAS-2 is to improve the rules for self-configuring
expanders.  I guess we're starting that topic :-)

If an expander is busy configuring, OPENs to devices attached to that
expander will work but OPENs to devices attached to expanders attached
to that expander might not.

Kevin raises good points about defining how long a management
application client (either in a self-configuring expander or elsewhere)
should allow for CONFIGURING to change to 0 before giving up on that
expander.  To be robust and tolerate errors, some timeout is necessary.

Each expander sends BROADCAST (CHANGE) when it finishes a discover
process and needs the others to rescan.  The BROADCAST (CHANGE)s may
thrash for a while, but will stop when the expanders are not detecting
any more changes.

This rule about when to send BROADCAST (CHANGE) needs to be modified so
it is not sent if there ended up being no changes detected:
"h) after a self-configuring expander device has completed configuration
and has changed its CONFIGURING bit from one to zero in the SMP REPORT
GENERAL function (see 10.4.3.3);"

Just basing it on the bit toggling won't resolve.
--
Rob Elliott, elliott at hp.com
Hewlett-Packard Industry Standard Server Storage Advanced Technology
https://ecardfile.com/id/RobElliott

 

> -----Original Message-----
> From: owner-t10 at t10.org [mailto:owner-t10 at t10.org] On Behalf 
> Of Ralph Weber
> Sent: Friday, January 27, 2006 10:48 AM
> To: t10 at t10.org
> Subject: Re: How do SAS self-configuring expanders self configure?
> 
> * From the T10 Reflector (t10 at t10.org), posted by:
> * Ralph Weber <roweber at ieee.org>
> *
> To me, this looks like a fine kettle of fish, particularly
> in light of Kevin's comments.
> 
> If all the Expanders in a SAS Domain are self-configuring,
> how do they avoid tripping over each other while they
> march through the full topology tree? Who waits for whom
> to set their Configuring bit to zero, and how long?
> How many Broadcast (Changes) does an HBA have to see
> before it can really scan the SAS Domain for interesting
> End Devices?
> 
> It seems to me that Self-Configuring Expanders were defined
> back when the only Expander that needed to be Self-Configuring
> was the one and only Fanout Expander allowed in a SAS Domain.
> The rules for and operation of Self-Configuring Expanders have
> not been put under the interoperability magnifying glass since
> those heady, early days.
> 
> If Self-Configuring Expanders are to become the linchpin of SAS
> Domain configuration, I think the time has come for a careful
> look at these issues.
> 
> All the best,
> 
> .Ralph
> 
> 
> Elliott, Robert (Server Storage) wrote:
> 
> >* From the T10 Reflector (t10 at t10.org), posted by:
> >* "Elliott, Robert (Server Storage)" <Elliott at hp.com>
> >*
> >Yes, it performs the full discover process.  If there is a 
> >configurable expander somewhere in the topology, the management 
> >application client in the self-configuring expander is as
> >responsible as any other for programming its routing tables.
> >
> >If the domain contains only self-configuring expanders (how it
> >learns this is vendor-specific), then it has no interest in the
> >addresses found beyond a subtractive phy; it only needs to know 
> >the addresses below its table routing phys.
> >
> >If we get rid of subtractive phys in SAS-2, then exploring the
> >whole domain will be necessary even in that case.
> >
> >My feeling is that having all expanders perform just discovery 
> >(but never have to program anyone else's routing tables) yields
> >reasonable amounts of SMP traffic - one REPORT GENERAL per
> >expander + one DISCOVER per expander phy.  Better-optimized
> >functions like DISCOVER LIST will improve that.  Where
> >discovery takes an awful long time is when remote
> >application clients are configuring routing tables, adding
> >one CONFIGURE PHY ROUTE INFORMATION per table entry (e.g.
> >hundreds/thousands of SMP function calls).  Again, a LIST 
> >function could help optimize that, but it's still more 
> >traffic than not configuring at all.
> >
> >--
> >Rob Elliott, elliott at hp.com
> >Hewlett-Packard Industry Standard Server Storage Advanced Technology
> >https://ecardfile.com/id/RobElliott
> >
> 
> *
> * For T10 Reflector information, send a message with
> * 'info t10' (no quotes) in the message body to majordomo at t10.org
> 
*
* For T10 Reflector information, send a message with
* 'info t10' (no quotes) in the message body to majordomo at t10.org


*
* For T10 Reflector information, send a message with
* 'info t10' (no quotes) in the message body to majordomo at t10.org
*
* For T10 Reflector information, send a message with
* 'info t10' (no quotes) in the message body to majordomo at t10.org





More information about the T10 mailing list