Routing Table Entries

Elliott, Robert (Server Storage) Elliott at hp.com
Thu Feb 27 07:41:32 PST 2003


* From the T10 Reflector (t10 at t10.org), posted by:
* "Elliott, Robert (Server Storage)" <Elliott at hp.com>
*
This is a multi-part message in MIME format.

------_=_NextPart_001_01C2DE76.B3C4685D
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

We obviously need to work on these rules and I welcome wording
suggestions.  I'd also be willing to put your example in the standard =
if
you find it more helpful than the ones already there.
=20
=20
1. The SAS address of the device directly attached to the phy (the 2100
in your example) in question is NOT put in the routing table.  At one
time we discussed having each phy own a routing table with at least one
entry, where the first entry would contain that directly-attached SAS
address.  However, we chose to say that the directly-attached SAS
addresses are outside of the routing tables visible via SMP and are =
just
automatically known to the expander.
=20
In this wording:
"For purposes of configuring the expander route table, the edge =
expander
devices attached to the expander
phy are assigned levels:
1) the attached edge expander device is considered level 1;
2) devices attached to it are considered level 2;
3) devices attached to level 2 edge expander devices are considered
level 3; and
4) etc.
=20
The expander route table shall be configured starting from expander
route index 0 by level (e.g., all level 1
entries first, then all level 2 entries, then all level 3 entries, =
etc.)
up to the value of the EXPANDER ROUTE
INDEXES field reported by the SMP REPORT GENERAL function (see
10.3.1.2).
=20
Assuming the attached edge expander devices each have N phys, the first
N entries shall be the SAS
addresses of the devices attached to the attached edge expander device,
ordered from expander phy 0
through expander phy N."
=20
The last paragraph doesn't say anything about putting the attached edge
expander device's address in the table; it jumps right to the addresses
of the devices attached to the attached edge expander device.
=20
=20
2. You are correct that address 1000 entries are disabled.  If that
address reaches the fanout expander, it shouldn't match in the phy 0
routing table and also match its own address.
=20
=20
3. The discover process does not probe the end devices; only expanders.
So, the last entry is #21 (probing the last phy in the expander with
address 4300).
=20
This wording intends to communicate that:
"For each of the level 2 devices that:
a) is an edge expander device with M expander phys; and
b) is attached to an expander phy in the level 1 edge expander device
with the table routing attribute,
the next M entries shall be the SAS addresses of the devices (level 3)
attached to that level 2 edge expander
device.
=20
This process shall repeat for all levels of edge expander devices in =
the
edge expander device set.
"

Case a) fails when probing after #21.

4. We don't seem to have a rule anywhere that states when no address is
attached to one of the remote phys, that entry in the local table is
still there but marked disabled (entries 7, 8, 9 in your example).
Changing:

Assuming the attached edge expander devices each have N phys, the first
N entries shall be the SAS
addresses of the devices attached to the attached edge expander device,
ordered from expander phy 0
through expander phy N.

to something like:

 Assuming the attached edge expander devices each have N phys, the =
first
N entries shall be used for expander phy 0 through expander phy N. If
the corresponding expander phy is attached to a device, the expander
route entry shall contain the SAS address of the device and be enabled.
If the corresponding expander phy is not attached, the expander route
entry shall be disabled.

(all the uses of "device" are suspect here and other cleanups are
pending)

--=20
Rob Elliott, elliott at hp.com=20
Hewlett-Packard Industry Standard Server Storage Advanced Technology=20
https://ecardfile.com/id/RobElliott
<https://ecardfile.com/id/RobElliott> =20


-----Original Message-----
From: Dennis Moore [mailto:dmoore at ix.netcom.com]=20
Sent: Wednesday, February 26, 2003 6:56 PM
To: Johnson, Stephen B.; t10 at t10.org
Subject: Re: Routing Table Entries


Stephen,
There are a few more differences between your table and my
interpretation. I will omit the 2100 entry for the moment and address
that later. I believe you left out several entries in your table. I
think the table should be as follows where E=3Denabled and =
D=3Ddisabled:
=20

0	 1000	 D=09
1	 1000	 D=09
2	 1000	 D=09
3	 1000	 D=09
4	 A123	 E=09
5	 4200	 E=09
6	 4200	 E=09
7	 xxxx	 D=09
8	 xxxx	 D=09
9	 xxxx	 D=09
10	 4300	 E=09
11	 4300	 E=09
12	 2100	 D=09
13	 2100	 D=09
14	 6001	 E=09
15	 6002	 E=09
16	 6003	 E=09
17	 2100	 D=09
18	 2100	 D=09
19	 6001	 E=09
20	 6002	 E=09
21	 6003	 E=09
22	 2100	 D=09
23	 2100	 D=09
24	 7001	 E=09
25	 7002	 E=09
26	 7003	 E=09
27	 2100	 D=09
28	 2100	 D=09
29	 7001	 E=09
30	 7002	 E=09
31	 7003	 E=09
=20
Let me know what you think,
dmoore
=20

----- Original Message -----=20
From: Johnson, Stephen   B.=20
To: Dennis Moore   ; t10 at t10.org
<mailto:t10 at t10.org> =20
Sent: Tuesday, February 25, 2003 5:38 PM
Subject: RE: Routing Table Entries


Denis,
=20
I'll take a stab at this for you.
=20
Attached (xls spread sheet) are my results for discovery and for each =
of
the expander route tables starting from Initiator C345.
=20
I like to build up an ordered table All the PHY's in the topology (see
attachment).=20
The table is built in the order of discovery where everything is =
scanned
by PHY identifier number from 0-n.
Once you have the table it is easy to determine each of the Route Table
entries by looking for
each of the PHY's that have the attribute Table Routing and making each
PHY behind the connected PHY an entry in the table.
=20
The actual algorithm you use for determining the tables is up you. I
have verified the recursive one in the back
of the spec and have also created a few others.
=20
Since you didn't say I assumed some PHY's are subtractive. The =
initiator
needs to
get this info about each PHY using the SMP Discover response.
=20
I haven't really thought too much about if the PHY's are enabled or not
but,
So someone please correct me if I'm wrong here.
=20
I consider all PHY's that are attached to something to have one of the
"enabled" values (0,2,3,8, or 9) in table 154 of the SAS-r03d spec.
=20
So, for PHY 0 of the Fanout I get the following Route Table: ( note: =
all
phy's are enabled but 7,8,9 )
=20


0
1000

1
1000

2
1000

3
1000

4
A123

5
4200

6
4200

7
...

8
...

9
...

10
4300

11
4300

12
2100

13
2100

14
6001

15
6002

16
6003

17
2100

18
2100

19
7001

20
7002

21
7003

=20
=20
If you start from another initiator the order of my "All PHY's Table" =
is
different but the Route Tables are exactly the same.
=20
Hope this helps.



Steve Johnson=20
LSI Logic=20
719 533 7511=20
sjohnson at lsil.com=20

-----Original Message-----
From: Dennis Moore [mailto:dmoore at ix.netcom.com]
Sent: Tuesday, February 25, 2003 9:31 AM
To: t10 at t10.org
Subject: SAS: Routing Table Entries


Rob,
I think I reject your reject of my comments on the expander routing
table stuff. I have conducted four SAS Seminars at four different
companies and have yet to get agreement among even one of groups on =
what
the contents of the routing table should be. I do an exercise during
class where I ask the students to construct the routing table entries
for Phy 0 of the fanout expander in the domain shown in the attached
file. Even after an hour of discussion I have not been able to get a
consensus on what it should look like. So, can you and anyone else that
thinks they know please send me what you think should be the routing
table entries for the attached domain drawing fanout expander Phy 0. I
ask the students for a table that gives the index, the SAS address, and
the Enable/Disable bit, and it looks something like this:
=20
Index    Phy 0     Enable/Disable
   0       2100           D
   1       1000           D
   2       1000           D
   3       1000           D
   4       1000           D
   5       A123          E
   6       4200           E
etc.
=20
I would very much appreciate anyone's input on what all the entries for
Phy 0 on the fanout expander should be.
Thanks,
dmoore
  =20
=20


------_=_NextPart_001_01C2DE76.B3C4685D
Content-Type: text/html;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">

Message We=20 obviously need to work on these rules and I welcome wording = suggestions. =20 I'd also be willing to put your example in the standard if you find it = more=20 helpful than the ones already there.
  
  
 1. The=20 SAS address of the device directly attached to the phy (the 2100 in = your=20 example) in question is NOT put in the routing table.  At one time = we=20 discussed having each phy own a routing table with at least one entry, = where the=20 first entry would contain that directly-attached SAS address.  = However, we=20 chose to say that the directly-attached SAS addresses are outside of = the routing=20 tables visible via SMP and are just automatically known to the=20 expander.
  
 In=20 this wording:
 "For purposes = of configuring=20 the expander route table, the edge expander devices attached to the=20 expander
 phy are assigned=20 levels:
 1) the attached = edge expander=20 device is considered level 1;
 2) devices = attached to it are=20 considered level 2;
 3) devices = attached to level 2=20 edge expander devices are considered level 3; and
 4) = etc.
  
 The expander route = table shall be=20 configured starting from expander route index 0 by level (e.g., all = level=20 1
 entries first, = then all level 2=20 entries, then all level 3 entries, etc.) up to the value of the = EXPANDER ROUTE
 INDEXES=20 field reported by the = SMP REPORT=20 GENERAL function (see 10.3.1.2).
  
 Assuming the = attached edge=20 expander devices each have N phys, the first N entries shall be the=20 SAS
 addresses of the = devices attached=20 to the attached edge expander device, ordered from expander phy = 0
 through = expander phy=20 N."
  
 The=20 last paragraph doesn't say anything about putting the attached edge = expander=20 device's address in the table; it jumps right to the addresses of the = devices=20 attached to the attached edge expander device.
  
  
 2. You=20 are correct that address 1000 entries are disabled.  If that = address=20 reaches the fanout expander, it shouldn't match in the phy 0 routing = table and=20 also match its own address.
  
  
 3. The discover process does not probe = the end=20 devices; only expanders.  So, the last entry is #21 (probing the = last phy=20 in the expander with address 4300).
  
 This=20 wording intends to communicate that:
 "For each of the level 2 devices = that:
 a) is an edge expander = device with M=20 expander phys; and
 b) is attached to an = expander phy in the=20 level 1 edge expander device with the table routing = attribute,
 the next M entries shall = be the SAS=20 addresses of the devices (level 3) attached to that level 2 edge=20 expander
 device.
  
 This process shall repeat for all levels of edge = expander=20 devices in the edge expander device set.
 "
 Case a) = fails when=20 probing after #21. 4. We don't seem to have a rule = anywhere that=20 states when no address is attached to one of the remote phys, that = entry in the=20 local table is still there but marked disabled (entries 7, 8, 9 in your = example).  Changing: Assuming = the attached edge=20 expander devices each have N phys, the first N entries shall be the=20 SAS
 addresses = of the devices=20 attached to the attached edge expander device, ordered from expander = phy=20 0
 through = expander phy=20 N.
 to=20 something like:  Assuming the attached edge expander = devices each have=20 N phys, the first N entries shall be used for expander phy 0 through expander phy N. If the corresponding expander phy is = attached to=20 a device, the expander route entry shall contain the SAS address of the = device=20 and be enabled. If the corresponding expander phy is not attached, the = expander=20 route entry shall be=20 disabled.
 (all the uses of "device" are suspect here = and other=20 cleanups are pending) --=20 
Rob Elliott,=20 elliott at hp.com 
Hewlett-Packard Industry Standard Server Storage Advanced=20 Technology 
https://ecardfile.com/id/Ro= bElliott=20 

-----Original Message-----
From: = Dennis Moore=20 [mailto:dmoore at ix.netcom.com] 
Sent: Wednesday, February = 26, 2003=20 6:56 PM
To: Johnson, Stephen B.; = t10 at t10.org
Subject: Re:=20 Routing Table Entries


 Stephen,
 There are a few more differences = between your=20 table and my interpretation. I will omit the 2100 entry for the = moment and=20 address that later. I believe you left out several entries in your = table. I=20 think the table should be as follows where E=3Denabled and=20 D=3Ddisabled:
  
 0 1000 D 1 1000 D 2 1000 D 3 1000 D 4 A123 E 5 4200 E 6 4200 E 7 xxxx D 8 xxxx D 9 xxxx D 10 4300 E 11 4300 E 12 2100 D 13 2100 D 14 6001 E 15 6002 E 16 6003 E 17 2100 D 18 2100 D 19 6001 E 20 6002 E 21 6003 E 22 2100 D 23 2100 D 24 7001 E 25 7002 E 26 7003 E 27 2100 D 28 2100 D 29 7001 E 30 7002 E 31 7003 E
  
 Let me know what you think,
 dmoore
  
 ----- Original Message ----- 
black">From:=20 Johnson, Stephen=20 B. 
To: Dennis Moore ; t10 at t10.org 
Sent: Tuesday, February 25, = 2003 5:38=20 PM
 Subject: RE: Routing Table=20 Entries
 

Denis,
  
 I'll take a stab at this for = you.
  
 Attached (xls spread sheet) are my results = for=20 discovery and for each of the expander route tables starting from = Initiator=20 C345.
  
 I like to build up an ordered = table All the=20 PHY's in the topology (see attachment). 
The table is built in the order of = discovery where=20 everything is scanned by PHY identifier number from = 0-n.
 Once you have the table it is easy to = determine=20 each of the Route Table entries by looking = for
 each of the PHY's that have the=20 attribute Table Routing and making each PHY behind the = connected PHY an=20 entry in the table.
  
 The actual algorithm you use for = determining the tables=20 is up you. I have verified the recursive one in the = back
 of the spec and have also created a=20 few others.
  
 Since you didn't say I assumed some PHY's = are=20 subtractive. The initiator needs to
 get this info about each PHY using the SMP = Discover=20 response.
  
 I haven't really thought too much about if = the PHY's=20 are enabled or not but,
 So someone please correct me if I'm wrong=20 here.
  
 I consider all PHY's that are attached to = something to=20 have one of the
 "enabled" values (0,2,3,8, or 9) in table = 154 of the=20 SAS-r03d spec.
  
 So, for PHY 0 of the Fanout I get the = following Route=20 Table: ( note: all phy's are enabled but 7,8,9 = )
  
 0
 1000
 1
 1000
 2
 1000
 3
 1000
 4
 A123
 5
 4200
 6
 4200
 7
 ...
 8
 ...
 9
 ...
 10
 4300
 11
 4300
 12
 2100
 13
 2100
 14
 6001
 15
 6002
 16
 6003
 17
 2100
 18
 2100
 19
 7001
 20
 7002
 21
 7003

=  
 = Unicode"=20 color=3D#000080 size=3D2> 
 = Unicode"=20 color=3D#000080 size=3D2>If you start from another initiator the = order of my=20 "All PHY's Table" is different but the Route Tables = are exactly the=20 same.
  
 Hope this helps.
 
Steve Johnson 
LSI Logic 
719 533 = 7511=20 
sjohnson at lsil.com=20 
-----Original Message-----
From: Dennis Moore=20 [mailto:dmoore at ix.netcom.com]
Sent: Tuesday, February 25, = 2003=20 9:31 AM
To: t10 at t10.org
Subject: SAS: Routing = Table=20 Entries


 Rob,
 I think I reject your reject of my = comments on=20 the expander routing table stuff. I have conducted four SAS = Seminars at four=20 different companies and have yet to get agreement among even = one of=20 groups on what the contents of the routing table should be. I do an = exercise=20 during class where I ask the students to construct the routing = table entries=20 for Phy 0 of the fanout expander in the domain shown in the = attached file.=20 Even after an hour of discussion I have not been able to get a = consensus on=20 what it should look like. So, can you and anyone else that thinks = they know=20 please send me what you think should be the routing table entries = for the=20 attached domain drawing fanout expander Phy 0. I ask the students = for a=20 table that gives the index, the SAS address, and = the Enable/Disable=20 bit, and it looks something like this:
  
 Index    Phy=20 0     Enable/Disable
   =20 0      =20 2100          =20 D
   =20 1      =20 1000          =20 D
   =20 2      =20 1000          =20 D
   =20 3      =20 1000          =20 D
   =20 4      =20 1000          =20 D
   =20 5      =20 A123          = E
   =20 6      =20 4200          =20 E
 etc.
  
 I would very much appreciate = anyone's input on=20 what all the entries for Phy 0 on the fanout expander should=20 be.
 Thanks,
 dmoore
  
  


------_=_NextPart_001_01C2DE76.B3C4685D--




More information about the T10 mailing list