Minutes from 9/94 FC-AL Ad Hoc Meeting
kc at core.rose.hp.com
Thu Sep 29 12:31:23 PDT 1994
From: Kurt Chan, HP (kc at core.rose.hp.com)
To: FC, SCSI Reflectors
Subj: Minutes of 9/7/94 FC-AL Ad Hoc Meeting
FC-AL Direct Attach Disk Ad Hoc Meeting Minutes
Red Lion Stapleton
Dal Allan, ENDL 250-1752 at mcimail.com
Scott Bruns, NCR scott.bruns at ftcollinsco.ncr.com
Kurt Chan, HP kc at core.rose.hp.com
Jim Coomes, Seagate jim_coomes at notes.seagate.com
Steve Dean, HP dean at core.rose.hp.com
Brian Doore, Seagate brian_doore at notes.seagate.com
David Ford, Cambex dford at cambex.com
Giles Frazier, IBM gfrazier at ausvm6.vnet.ibm.com
Norm Harris, Adaptec nharris at eng.adaptec.com
Tim Johnson, Cray Research tjohnson at cray.com
Jean Kodama, QLogic j_kodama at qlc.com
Glen Koziuk, NCR glen.koziuk at ftcollinsco.ncr.com
Scott Mindemann, Interphase scottm at iphase.com
Yogi Schaffner, Silicon Systems yogi at tus.ssi1.com
Greg Scherer, Emulex g_scherer at emulex.com
Ken Smith, Conner ken.smith at conner.com
Jeff Stai, Western Digital stai at dt.wdc.com
Horst Truestedt, IBM truested at vnet.ibm.com
Fanny Wong, IBM fwong at vnet.ibm.com
1. FL_PORT COEXISTENCE WITH PRIVATE LOOP DEVICES
- Private loop devices must observe normal FC-AL rules that require a
device which has a "hard" address to defer to a Fabric assigned
- Private loop addresses must have zeroes in the upper 16 bits.
- Public loop devices may not have all zeroes in the upper 16 bits.
- FL_Ports are prohibited from forwarding frames into a loop if the
frames contain all zeroes in the upper 16 bits. This is considered
frame misrouting. Neither the fabric nor any devices on the other
side of the fabric should be attempting to communicate with Private
2. LOGIN AND TARGET DISCOVERY
The standard contains the following Login rules:
- Private loop devices are only allowed to log in with other
private loop devices, and may not attempt login with public loop
devices or with FL_Port.
- Public loop devices may log in with all three - public private,
and FL ports and must attempt FL_Port login as part of their
initialization procedure (per FC-PH).
Profiles may further restrict these login rules. For example, the
following table defines the Logins allowed by the Private Loop Disk
Profile when Private, Public, and FL ports coexist on the same loop.
| Private Public Private Public
LOGIN INITIATOR | Initiator Target Target Initiator Fabric
------------------- | --------- ------ ------- --------- ------
Private SCSI Init | Yes No Yes* No No
Public SCSI Target | Yes Yes No** No** Yes*
Private SCSI Target | No No No No No
Public SCSI Init | Yes Yes** Yes Yes Yes*
No = prohibited by standard and/or profile
Yes = allowed by standard, but not yet required or prohibited
by any loop profile
Yes* = required, either by a standard or by Private Loop profile
No** = anticipated to be prohibited by Public Loop Profile
Yes** = anticipated to be required by Public Loop Profile
If a Public Initiator only supports class 2, it cannot successfully
log into a Private Target, which only supports class 3.
For this profile, PLOGI shall only be attempted by SCSI Initiators to
devices with D_ID = 0000+AL_PA hex in the upper 16 bits. (Public loop
devices performing discovery functions may need to attempt two PLOGIs to
each AL_PA, one to 0000+AL_PA and one to xxxx+AL_PA, where xxxx is the
upper 16 bits of the attached FL_Port address).
Login reception was discussed when login resource limits are reached.
Rather than reject new PLOGIs, it was recommended that all PLOGIs be
accepted, and the "oldest" PLOGI be implicitly logged out when resource
limits are reached. [Ed note: discuss at next meeting whether this
behavior needs to be specified for interoperability, and if so whether
explicit LOGO to "oldest PLOGI" be attempted before ACC to newest PLOGI is
transmitted. Also discuss requirement to transmit LOGO in response to
frames received from a device which is not logged in.]
3. ADDRESS RESOLUTION
1) For the upper 16 bits of an NL_Port native address identifier, FC-AL
now requires that:
- all private loop devices or public loop devices which do not
share the loop with an FL_Port use 0000h.
- all public loop devices which share the loop with an FL_Port
use the (nonzero) value assigned to them by the FL_Port.
2) For devices which are configured with a hard address, if the device
does not acquire it's hard address, it shall accept a soft address
and accept PRLI with the response code '0101' in word 1, bits
[11:8] ("a predefined condition precludes establishing this image
pair - PRLI shall not be retried") and implicitly log out all
3) For devices which are NOT configured with a hard address, if the
device does not receive it's previously acquired address, it shall
accept a new soft address and implicitly log out all devices.
4) For ALL devices, if neither a hard nor soft address is acquired,
the device shall go to non-participating mode and implicitly log
out all devices.
4. LOOP INITIALIZATION - NOS/OLS/LR/LRR
In most current implementations, an NL_port will not respond per FC-AL when
it receives a NOS, OLS, LR, or LRR while in the OPENED state. This is
because these Primitive Sequences were intended to initialize point-point
N_Port-F_Port links, not link segments on a shared medium where the
output and input of any given port do not go to the same N_Port. Requiring
NOS/OLS to work across a Loop Circuit would be analagous to requiring
class 1 connections to stay up while performing this protocol.
Since LIPs are used as link-level resets in this profile, private NL_Ports
are prohibited from issuing NOS, OLS, LR, or LRR unless in OLD-PORT state.
What NL_Ports do if they receive any of these Primitive Sequences while in
the OPENED state is vendor-specific. If the port is in MONITORING state,
it should propagate the Primitive Sequence.
5. LOOP INITIALIZATION - LIPs
There are three basic types of Loop Initialization Primitives. Here's my
understanding of how these LIPs are now being used, and their effect on
private loop NL_Ports (some of this is borrowed from Version 0.40). We
will review these on Oct 4.
Private loop devices are prohibited from setting the L_bit. It is
expected that the L_bit will be used only by FL_Ports. [ed note - do we
anticipate the use of NFL_Ports in the private loop profile? If so,
should NFL_Ports be allowed to set the L_bit?]
b) Power-on reset
LIP(AL_PD,AL_PS) is a request from the transmitting NL_Port (at AL_PA =
AL_PS) to the receiving NL_port (at AL_PA = AL_PD) to perform a power-on
reset. There is no broadcast LIP which causes a power-on reset. Only
SCSI Initiators may transmit this LIP. See the table on page 22 of the
0.50 profile for what objects are reset by this function.
c) Loop initialization
Both SCSI Initiators and SCSI Targets may transmit this LIP type.
LIP(F7,F7) indicates the transmitting NL_Port is attempting to acquire
an AL_PA due to either new insertion or power-on initialization, or has
been unable to win arbitration within E_D_TOV. Receiving devices shall
go immediately to OPEN-INIT state, regardless of their current state. On
private loops, only the Sequence in progress may be disrupted, resulting
in standard error recovery (ULP timeout) if any frames in transit are
LIP(F7,AL_PS) indicates the transmitting NL_Port (with AL_PA = AL_PS) is
attempting to reinitialize the Loop. Receiving devices shall go
immediately to the OPEN-INIT state.
d) Loop Failure
Both SCSI Initiators and SCSI Targets may transmit this LIP type.
LIP(F8,F7) indicates the transmitting NL_Port has detected a Loop
failure at its receiver before it has acquired a valid AL_PA. Receiving
devices shall go immediately to OPEN-INIT state.
LIP(F8,AL_PS) indicates the transmitting NL_Port (with AL_PA = AL_PS)
has detected a Loop failure at its receiver. Receiving devices shall go
immediately to OPEN-INIT state.
6. BYPASS OPERATION
The new Primitive Sequence name is Loop Port Enable (LPE) and Loop Port
Bypass (LPB). My understanding of the desired behavior is that a bypassed
a) upon receipt of LPB, shall go to MONITORING state and shall
not change state until LPE is received.
b) loses it's AL_PA (goes to non-participating mode) if a LIP is received,
but does not go to OPEN-INIT state and does not clear the P_BYPASS state
variable [ed note: is this also true for LIP(AL_PD,AL_PS)?]
c) does not propagate OPNyy or OPNyx addressed to it (does propagate
OPNfr and OPNyr, or OPNs to other ports).
d) does not attempt arbitration
e) sets the P_BYPASS state variable which will activate any Loop Bypass
Circuit, if present.
Furthermore, any port which does not pass self-test shall bypass
itself until the it is functional. LPE or REQ(init) to a bypassed
port will reset the P_BYPASS variable.
7. OLD-PORT STATE
For private loop devices, OLD-PORT state is optional (since FL_Port
communication is prohibited anyhow). However, silicon designers
looking toward the future may want to design this function in for both
point-point fabric connect or FL_Port compatibility.
OPNyr (selective replicate) will be documented in FC-AL 4.34, but
generation of OPNyr will not be mandatory in the private loop profile.
HP will not support generation on first implementation since OPN is
sent as part of automated Sequence transmission and HP silicon lacks
the "phase control" needed to transmit consecutive OPNs. [ed note:
do the OPNr rules also apply to OPNyr, or was selective replicate
intended to be a reliable service?]
Note that OPNyr FC-AL broadcast is even LESS reliable from a link
layer perspective than class 3 since R_RDYs are not used and class 3
frame reception is subject to local buffer availability in the
receiving NL_Ports. Flow control is completely ULP-managed.
9. FAIRNESS PROBLEMS
The following descriptions were transcribed from Jean Kodama notes due to
mental siezures on the part of the editor. The discussion was more related
to FC-AL than this profile.
+--> B --> C ---+
+--- F <-- E <--+
There are three problems uncovered that ultimately manifest themselves
as some form of unfairness on the loop. It was anticipated that
a) fairness is optional in FC-AL,
b) FL_Ports and SCSI Initiators will be allowed to be unfair in some profiles,
c) there are no FC-AL limits on how long a port can hold a circuit, which
is another form of unfairness
that the following cornercases would not impact Loop functionality
Problem 1: bottom half of loop could starve
- Port_A receives ARB(F0) and begins transmitting IDLs
- Port_B sees IDL, transmits one IDL, and begins to arbitrate
- Port_D receives one IDL then a stream of ARBx's while transmitting a frame
- ARBx is the fill word when frame transmission is complete
- IDL is lost and Port_E and Port_F do not get a chance to arbitrate
This used to work because ARB(F0) changed to IDL only when it was guaranteed
that no frames were in flight. Possible fix is to go back to original scheme
(don't change ARB(F0) to IDL until a CLS is received).
Problem 2: can lose fairness token (IDL) due to smoothing
- Port_A sends OEF and 2 ARB(F0)s
- Port_A receives an ARB(F0) and begins sending IDLs
- Port_B passes through the EOF and 2 ARB(F0)s
- Port_B sees the IDLs, transmits one of them, and begins arbitration
- Port_C needs to perform a fill word deletion because of clock skew,
but has to wait for EOF and two fill words to pass
- Port_C receives the EOF and 2 ARB(F0)s, then deletes the next IDL
- IDL is lost, and nobody else gets to arbitrate
Problem 3: opened node could starve
- Port_D is arbitrating and is the last device in the current arbitration window.
- Port_D is opened. When it receives ARB(F0) it replaces its CFW (own ARBx)
with ARB(F0) per FC-AL rules
- Port_A receives ARB(F0) and resets the access window
A possible fix is, if attempting to arbitrate, to test the priority of
an incoming ARBx before replacing the CFW in OPENED, TRANSMITTED CLS,
or RECEIVED CLS states. This may require a history variable. Also,
a port controller should keep REQ(arb) asserted until ARBITRATION WON
state is achieved to prevent "rogue" ARBx's.
10. Error Recovery
There were a couple of problems uncovered by a subsequent FCSI meeting
with respect to Target-Initiated ABTS that will be discussed at the
October meeting. It is possible that ACA and UA definitions in SAM
are inadequate for networked disks. The suggested wording changes I
posted to fix FCP rev 9 were not incorporated since there are other
problems related to Task Management which making Target-Initiated ABTS
optional does not solve. Oct 4 we will discuss whether or not
creating a new IU (FCP_RSP to Task Mgmt functions) is a necessary but
insufficient solution to the problem.
The table on page 22 of the profile is split here into two tables, one
for FC-1/FC-2 actions, and the other for FCP actions. We will review
this in Oct. Note numbers are in (parentheses).
LIP(y,x) LR LOGO, FLOGI ABTS PRLI,
(7) (12) PLOGI (4) (8) PRLO
----- FC2 Objects ----------- --------- ------ ----- -------- ------ ------
PLOGI parmameters Y(8) N Y N(2) N N
Open Seq's, per Initiator Y N(9) Y N(1,2) Y Y
Open Seq's, all Initiators Y N(9) N N(1,2) N N
EE_Credit_CNT Y Y(10) Y N N N
BB_Credit_CNT Y Y Y Y N N
AL_PA N(1) na N Y N N
FLOGI parms na N N Y N N
---- FC4, ULP Objects -------
PRLI parms Y(7) N Y(3) N N Y
Single open Task, per Init Y N(13) Y(3) N Y Y
All open Tasks, per Init Y N Y(3) N N Y
All open Tasks, all Inits Y(5) N N N N N
Mode page parameters Y(5) N Y(3) N N Y(3)
Target Task Task
Reset Set Set
----- FC2 Objects ----------- --------- ------- -------
PLOGI parmameters N N N
Open Seq's, per Initiator N(14) N(14) N(14)
Open Seq's, all Initiators N(14) N(14) N(14)
EE_Credit_CNT N N N
BB_Credit_CNT N N N
AL_PA N N N
FLOGI parms N N N
---- FC4, ULP Objects -------
PRLI parms N N N
Single open Task, per Init Y Y Y
All open Tasks, per Init Y Y Y
All open Tasks, all Inits Y(5) Y(5) N
Mode page parameters Y(5) N N
1. Unless AL_PA's have changed (i.e., preferred address has already
been taken by another port)
2. Unless Worldwide Port Name of FL_Port has changed, or unless
Worldwide Port Name of N_Port has changed in a point-point
3. For Process or N_Port Login Initiator only, not for all
4. Public loop device must discard all frames until FLOGI to AL_PA=00
has been completed.
5. Parameters cleared for all Initiators. Unit Attention conditions
shall be created per SAM requirements.
6. Unless this object is retained across power-cycles.
7. Also known as LIP(AL_PS,AL_PD) or FC-AL Selective Reset. The
selected Loop device performs power-on reset upon receipt of this
primitive. FL_Ports and Public Loop Devices are prohibited from
issuing Selective Reset.
8. ABTS with L_S=1 in the ACC is how the SAM Abort Task function is
performed in FCP.
9. In Class 1 all active/open Sequences are terminated. In class 2,
open Sequences may eventually be aborted when E_D_TOV timeouts
occur as a result of missing frames/ACKs. If credit happens to be
balanced when the LR/LRR protocol is invoked, open Sequences may
be unaffected. See FC-PH 4.3, 188.8.131.52.
10. EE_Credit_CNT only affected for class 1. Class 2 EE_Credit_CNT
11. Disregard this row for Private Loop devices (which have no Fabric
12. This profile prohibits NOS, LR, ARB+OPN+NOS, and ARB+OPN+LR on a
Loop. NOS and LR only affect Loop devices when they are in OLD
PORT state. A port which has received NOS or OLS cannot become
active without participating in the LR/LRR protocol (see FC-PH
4.3, Annex Q). Therefore, only the effects of LR are listed
13. Open Tasks may be affected depending upon class of service and
BB_Credit_CNT when LR is received - see note 10.
14. Tasks are cleared internally within the Target, but open FC-PH
Sequences must be individually aborted by the SCSI Initiator via
The next two ad hoc meetings will be 8:30am at the Stapleton Red Lion:
Tues Oct 4
Tues Nov 1
I'd also like to discuss at the next meeting whether or not we should
continue to meet in Denver after November to develop the Open Loop
Profile versus attempting to meet during ANSI week, and your views on
which environment would result in the greatest productivity.
More information about the T10