iscsi : changes involving tgt portal group tag.

Mallikarjun C. cbm at rose.hp.com
Tue Mar 5 10:01:36 PST 2002


* From the T10 Reflector (t10 at t10.org), posted by:
* "Mallikarjun C." <cbm at rose.hp.com>
*
Jim,

> I'm confused here.  You say " LU views can be a function of the transport
> in iSCSI, as your bullet (b) clearly says right after this comment!" and
> then you argue that this can cause a problem.

I was merely objecting to your comment on layering violation.
It is allowed to associate LU views to a transport construct -
a (possibly virtual) iSCSI target node.  

My attempt was to understand the issues in associating TPGT with 
the LU views (the next level), but not to say that there are no
issues.  

> In other words, my case (b) should not be a problem even without the TPGT!

Agreed, I didn't think it was.

>If you chose to build a box with different LU views on
> different ports and then you swap that around under the covers, (and you
> don't UA the host to death)...

I wasn't convinced that the UA could be used for changes brought
about by iSCSI-level reconfiguration - which was my scenario A.  
But after further thought, I agree with your suggestion that UA
is usable even for scenario A - since the associated SCSI Port Name
is changing as well with the reconfiguration.

To summarize, relying on UA seems like a decent solution even 
when LU views are associated with TPGTs, in addition to node 
names.

Regards.
-- 
Mallikarjun 


Mallikarjun Chadalapaka
Networked Storage Architecture
Network Storage Solutions Organization
MS 5668	Hewlett-Packard, Roseville.
cbm at rose.hp.com


Jim Hafner wrote:
> 
> Mallikarjun,
> 
> I'm confused here.  You say " LU views can be a function of the transport
> in iSCSI, as your bullet (b) clearly says right after this comment!" and
> then you argue that this can cause a problem.  But, if the LU view changes
> by "(b) functions of iSCSI target node presentations" (my quote), then the
> NODE NAME changes.  When that happens (and the host doesn't detect it, then
> the host is logging in to a completely different node and the iSCSI
> (re)login will fail (the iSCSI node names won't be right).  That is, it
> will fail even without including the TPGT (if we add that requirement).
> Adding the TPGT only gives an extra check for the initiator logging in to
> the *same node*.
> 
> In other words, my case (b) should not be a problem even without the TPGT!
> 
> So again, I don't see an issue here as far as iSCSI is concerned (except
> totally locallized on the TPGT issue Santosh raised).   LU views are
> functions of the node configuration, as is the portal group arrangement.
> Within the SCSI standards, there are no "port-specific" view associations
> defined; but there is the UA mechanism to reflect changes to the view (as
> might happen fully within the SCSI standards via either the Access Controls
> stuff or the SCC-2 (controller command set).  Within iSCSI, if we define
> some rule along the lines Santosh is suggesting (including TPGT and some
> sort of rule that requires changes at the naming layer to reflect changes
> to the portal group assignment), then I think all our basis are covered.
> 
> There is a responsibility on the part of the target builder to do the least
> amount of reconfiguration that is disruptive to the potential data
> integrity issues.  If the host doesn't pay attention, tough cookies.  If
> the target doesn't provide the right hooks for the host, then that vendor
> will soon out of business.  SCSI has defined its mechanisms for this issue
> (the LU view changes within a given fixed device).  iSCSI can define its
> mechanism (the portal group view changes within a given fixed device).
> Anything that involves a change of device is already detectable by the
> host.
> 
> Your scenario A can happen in my opinion (according to the standards) only
> if (a) either the node name changed (which is covered already) or (b) the
> TPGT changes for both portal groups (which is covered by Santosh's
> recommendation).
> 
> You ask about "different views from different ports expressly forbidden by
> the SCSI architecture".  As with many things, the SCSI architecture can let
> you do a lot of things without violating the standard, but they can be
> risky as well. If you chose to build a box with different LU views on
> different ports and then you swap that around under the covers, (and you
> don't UA the host to death) you've problably broken your contract with the
> host.  The same thing can happen in many FC controllers that already do
> this mapping by target port.  You've stretched outside the SCSI
> architecture (haven't violated it, just gone outside it).  As such, you
> take the risk to your customer's data.
> 
> My point here is that I think the iSCSI standard can be silent on this
> specific issue of your case A.  T10 (and FCP and SRP and...) is silent
> (with the exception of UA), and I don't see any obligation on the part of
> iSCSI to do more.
> 
> Jim Hafner
> 
> "Mallikarjun C." <cbm at rose.hp.com> on 03/04/2002 05:38:18 PM
> 
> To:    Jim Hafner/Almaden/IBM at IBMUS
> cc:    <ips at ece.cmu.edu>, <t10 at t10.org>
> Subject:    Re: iscsi : changes involving tgt portal group tag.
> 
> Jim,
> 
> > Much as I would like to agree with you on most things (especially after
> you
> > agreed with me so much...)
> 
> Thanks for preparing me for what's coming, :-)  Let me attempt to
> paraphrase
> what you're saying.
> 
> > I think connecting LU views with TPGTs is a mistake and violates
> layering.
> 
> I am surprised.  LU views can be a function of the transport in iSCSI, as
> your
> bullet (b) clearly says right after this comment!  I would go so far as to
> say that
> one reason for creating multiple iSCSI target nodes is to simply "package"
> different LU views under each one.
> 
> If I understood Santosh's concern correctly, he was referring to the
> scenario of
> initiator (incorrectly) addressing a changed LU with the same LUN after a
> new
> session is instantiated.  Assuming for now that we introduced the "intended
> TPGT"
> into the Login Request header, I can see two ways how his scenario can
> still
> happen (both assume session and I/O termination, followed by a
> reinstantiation) -
> 
> A.  Two iSCSI portal groups (i.e. SCSI ports) with different LU views
>      had their LU views swapped "underneath" them as part of
>      reconfiguration.
> 
> B.  Certain LUNs had new LUs (with new LU identifiers) at the old address.
> 
> It wasn't clear to me from your comments that scenario A above (different
> views for
> different ports) is expressly forbidden by SCSI architecture.  Can you
> please
> clarify?
> 
> I agree with your reasoning for scenario B above that a UA (perhaps
> interlocked)
> would be
> adequate to deal with the changed LU.    Though it wasn't clear from SPC-3,
> I assume
> initiators are guaranteed to see this UA even after arbitrary amount of
> time (after
> the
> change) when they re-establish the nexus.  A brief digression: if a new
> initiator
> establishes
> a nexus after the change (for the first time), is it expected to see this
> UA?
> 
> Lastly,
> > This should be handled by LU Identifier (not just LUN) if this is a...
> 
> Agreed, I believe this Identifier is traditionally checked during the
> "open" time,
> and as you point out (should be) on the "REPORTED LUNS DATA HAS CHANGED"
> UA. [ If scenario A were legal, the checking would need to be done after
> the forced
> discovery (due to TPGT change) process as well....]
> 
> Regards.
> --
> Mallikarjun
> 
> Mallikarjun Chadalapaka
> Networked Storage Architecture
> Network Storage Solutions Organization
> Hewlett-Packard MS 5668
> Roseville CA 95747
> 
> ----- Original Message -----
> From: "Jim Hafner" <hafner at almaden.ibm.com>
> To: "Mallikarjun C." <cbm at rose.hp.com>
> Cc: <ips at ece.cmu.edu>; <t10 at t10.org>
> Sent: Monday, March 04, 2002 4:21 PM
> Subject: Re: iscsi : changes involving tgt portal group tag.
> 
> >
> > Mallikarjun,
> >
> > Much as I would like to agree with you on most things (especially after
> you
> > agreed with me so much...)
> >
> > I think connecting LU views with TPGTs is a mistake and violates
> layering.
> > Besides, LU views are
> > (a) functions of SCSI level or other access controls
> > (b) functions of iSCSI target node presentations
> > Within the standards that I know of, there is no defined LU view that is
> > different across the different target ports of any SCSI device (BUT see
> the
> > two notes in the P.S.)
> >
> > In case (b), the TPGT is already qualified by the node so there's no
> issue
> > here.
> > In case (a), the rules as defined by SCSI ACs are that the LU view is the
> > same across all target ports.
> >
> > I don't have any good "words" to smith into the document that makes clear
> > what "reconfiguration" should trigger a new name and what shouldn't (and
> I
> > haven't had time to think much about it either).
> >
> > As for your final comment about IOs getting directed to the correct LU.
> > This should be handled by LU Identifier (not just LUN) if this is a
> > significant concern.  There's a subtle requirement in this virtual world
> > we're moving towards that requires careful mapping of LUs to device
> objects
> > within hosts so that LUN changes don't cause such problems.  But LUN
> > changes can occur for many reasons, not just those raised here.  SCSI
> gives
> > "REPORT LUNS DATA CHANGED" Unit Attentions, but if they're ignored,
> > ...OOPS!.
> >
> > Finally, to Santosh:  there is no notion in SCSI (or iSCSI, I think)
> about
> > LUN OPEN/CLOSE.   LUs are "discovered" by INQUIRY and REPORT LUNs.  Once
> > discovered, there is no further action required.  So, a logical unit is
> > either "addressible" (in the view) or not (with a slight caveat about
> SCSI
> > ACs and AccessId registration, but that should happen before the
> discovery
> > phase anyway).   There may be layers about SCSI which have these notions
> > but they probably only manipulate internal OS datastructures and have no
> > "wire" protocol associated with them.   So, there should be no need to
> > think about such actions being required after re-login.
> >
> > Jim Hafner
> >
> > P.S.
> > Note1:  there are vendor-specific implementations that might seem to
> > violate this statement if taken literally, but because such
> implementations
> > are "pre-node names", one can view them as falling into the model (b)
> > above.
> > Note2: the "asymmetric port" stuff in SCSI (recent addition) doesn't
> change
> > the "view" (i.e., the list of LUNs) so much as the accessibility of the
> > addressed LU, so even this doesn't square with the issue at hand.
> >
> >
> >
> > "Mallikarjun C." <cbm at rose.hp.com>@t10.org on 03/04/2002 03:12:15 PM
> >
> > Sent by:    owner-t10 at t10.org
> >
> >
> > To:    "Santosh Rao" <santoshr at cup.hp.com>, <ips at ece.cmu.edu>
> > cc:    <t10 at t10.org>
> > Subject:    Re: iscsi : changes involving tgt portal group tag.
> >
> >
> >
> > * From the T10 Reflector (t10 at t10.org), posted by:
> > * "Mallikarjun C." <cbm at rose.hp.com>
> > *
> > Santosh and Jim,
> >
> > It appears a good idea to add in the portal group tag in the Login
> > Request header for a sanity check by the receiving target portal.
> >
> > I generally agree with Jim's comments that the duration of persistence
> > of a portal group tag is intricately linked to the extent of portal
> > reconfiguration.
> > Not every target reconfiguration may result in a portal group tag
> > reassignment.
> > OTOH, some reconfigurations may be so extensive as to cause even a node
> > name change.
> >
> > Some comments on Santosh's message -
> >
> > > "If a portal group is re-configured such that all its previously
> > > advertised network portals are no longer a part of the portal group,
> > > then, the portal group tag (and thereby, the port name/identifier)
> > > *MUST* be changed to indicate a new target port."
> >
> > I am not sure this solves the problem you're trying to get at.  If none
> of
> > the earlier IP addresses can get an initiator to the SCSI target port
> that
> > it
> > knew of (your scenario), it appears to me that it doesn't matter if the
> > portal group tags are changed or not....A new discovery process should
> > update the initiator of the changed portal addresses.
> >
> > I suggest the following text -
> >
> >    After a portal group reconfiguration which changed the view of LUs
> >    for an initiator with a given set of privileges, the target MUST
> change
> >    the portal group tag that represents the reconfigured target portal
> >    group.
> >
> > > > Under these events, shouldn't all "open/active I_T_L traffic" have
> been
> > > > aborted, closed or otherwise ended?  So this shouldn't be an issue at
> > all.
> > >
> > > On a session logout & re-login, it is not efficient/necessary to close
> > > and re-open each LUN behind that target port, due to the fact that a
> > > target port may have hundred's of LUNs behind it.
> >
> > I agree with Jim on this one - there should be *no* open/active I_T_L
> nexus
> > traffic after a reconfiguration, targets can enforce this via simple
> iSCSI
> > means
> > (reject initiator advances to continue the session for DefaultTime2Wait+
> > DefaultTime2Retain seconds).  In fact, Async logout request requires a
> > clean closure that implicitly aborts I/Os.
> >
> > What you're describing is typical O/S "LUN open" and "LUN close"
> > interactions.  I agree that they wouldn't have to be repeated, but care
> > must be taken to ensure that new I/Os (on the new session after
> > reconfiguration)
> > are not delivered to the incorrect LUs.  It seems that the addition of
> > TPGT in the login header and the proposed new text (above) would take
> > care of this.
> >
> > Regards.
> > --
> > Mallikarjun
> >
> > Mallikarjun Chadalapaka
> > Networked Storage Architecture
> > Network Storage Solutions Organization
> > Hewlett-Packard MS 5668
> > Roseville CA 95747
> >
> > ----- Original Message -----
> > From: "Santosh Rao" <santoshr at cup.hp.com>
> > To: <ips at ece.cmu.edu>
> > Cc: <t10 at t10.org>
> > Sent: Monday, March 04, 2002 10:40 AM
> > Subject: Re: iscsi : changes involving tgt portal group tag.
> >
> >
> > > * From the T10 Reflector (t10 at t10.org), posted by:
> > > * Santosh Rao <santoshr at cup.hp.com>
> > > *
> > > Jim,
> > >
> > > I agree that a "complete re-configuration" of a target port can result
> > > in a new port name & port identifier. However, the tricky part is the
> > > definition of a "complete re-configuration of an iscsi target port",
> due
> > > to the concepts of portal groups involving multiple network portals.
> > >
> > > For example, if a portal group (aka, an iscsi target port) were to be
> > > re-configured to include a new network portal [moved from another
> portal
> > > group], then, the target port itself has not changed, since it is still
> > > accessible through its previously used network portals. What has
> changed
> > > is the individual network portal that has moved from 1 target port to
> > > another.
> > >
> > > Hence, it is sufficient, in this case, to maintain persistence of the
> > > target port name/identifier, without requiring any change in port
> > > name/identifier. By requiring initiators to send the intended TPGT
> (scsi
> > > target port name/identifier) along with the login request, this allows
> > > the target port to detect that the network portal is being accessed to
> > > connect to a different target port and it can reject the login.
> > >
> > > It may be helpful to call out the specific case when a port
> > > name/identifier MUST change. How about something like :
> > >
> > > "If a portal group is re-configured such that all its previously
> > > advertised network portals are no longer a part of the portal group,
> > > then, the portal group tag (and thereby, the port name/identifier)
> > > *MUST* be changed to indicate a new target port."
> > >
> > > This would allow access to the target port through its un-altered
> > > network portals to continue un-disrupted. More comments inline, in
> > > response to some of your queries.
> > >
> > > Thanks,
> > > Santosh
> > >
> > > NOTE : In this discussion target port and target portal group are used
> > > to imply the same entity, within a target node.
> > >
> > >
> > > Jim Hafner wrote:
> > >
> > > > SAM-2 requires scsi port names to be persistent and
> world-wide-unique.
> > > > (SAM-2 Rev 22 Section 4.7.7). Quoting from SAM-2 :
> > > >
> > > > "A scsi port name shall never change and may be used to persistently
> > > > identify a scsi initiator port or target port...".
> > > >
> > > > <JLH>
> > > > There are different ways that one can interpret this "persistent"
> rule.
> > The
> > > > intent was that names shouldn't change over time when *identifying
> the
> > same
> > > > object*.   What that means is that if the object changes (in any
> > > > discernable way), then the name should change.  So, the object can
> move
> > to
> > > > a different piece of hardware, but it can still be named the same
> way.
> > If
> > > > the object changes, e.g., in this case, reconfigures as a different
> set
> > of
> > > > network portals with different addressing elements, then I would
> think
> > that
> > > > the name should change as well.   That's all the persistence one
> really
> > > > needs.
> > > >
> > > > I'm not sure what that implies about your recommendation.  I don't
> see
> > any
> > > > problem with it, either.
> > > > </JLH>
> > >
> > > I think we may be in agreement. (See reasoning above).
> > >
> > >
> > > >
> > > > The rationale for (2) is :
> > > > --------------------------
> > > > Consider an initiator node establishing multiple sessions to a scsi
> tgt
> > > > port, with each session established to a subset of the network
> portals
> > > > within the tgt portal group.
> > > >
> > > > Consider an iscsi transport event involving the re-configuration of
> > > > target portal groups on the iscsi target node. This may be preceeded
> by
> > > > the iscsi sessions seeing an async message "target requests logout",
> > > > followed by session logout, portal group re-configuration, and then,
> > the
> > > > initiator re-establishes sessions.
> > > >
> > > > Across a transport event that results in such session termination and
> > > > re-establishment, the initiator needs to authenticate that it is
> still
> > > > speaking to the same [i]scsi target port, in order to ensure that any
> > > > open/active I-T-L nexus traffic on that session is not incorrectly
> > > > routed to a wrong LUN after such a transport event.
> > >
> > > > <JLH>
> > > > Under these events, shouldn't all "open/active I_T_L traffic" have
> been
> > > > aborted, closed or otherwise ended?  So this shouldn't be an issue at
> > all.
> > >
> > > On a session logout & re-login, it is not efficient/necessary to close
> > > and re-open each LUN behind that target port, due to the fact that a
> > > target port may have hundred's of LUNs behind it.
> > >
> > > All outstanding I/Os need to be aborted. Once the session is
> > > re-established [and the target port is authenticated to be the same],
> > > further I/O traffic can be resumed without requiring the SCSI ULP to
> > > close and re-open each LUN. Some transport specific clearing effects
> may
> > > have occurred, which may require additional LUN level activity, in some
> > > cases.
> > >
> > > (There are analogies to the above model in FCP & SRP, which
> authenticate
> > > port name/identifier on login, allowing I/O resumption after session
> > > termination and re-establishment.)
> > >
> > >
> > > > To prevent such authentication issues, iscsi can send the iscsi
> target
> > > > port identifier (portal group tag) explicitly in the login request,
> and
> > > > the login can be rejected by the target on a portal group tag
> > mis-match.
> > > > (if the network portal does not belong to the addressed portal group
> > > > tag).
> > > > <JLH>
> > > > Did you mean for the initiator to send this TPGT?
> > > > </JLH>
> > >
> > > Yes. The initiator login request must include the target portal group
> > > tag, thus identifying the target port to which a session is being
> > > established.
> > >
> > > Login currently carries no addressing information, since the addressing
> > > is implicit, based on the TCP connection in use. The problem with this
> > > approach is that the login implicitly addresses a network portal, and
> > > not the target port. As seen in the earlier example, the association of
> > > network portal <-> target port can change, and such changes can be
> > > detected, if the initiator were to explicitly identify the target port
> > > being logged into.
> > >
> > > --
> > > ##################################
> > > Santosh Rao
> > > Software Design Engineer,
> > > HP-UX iSCSI Driver Team,
> > > Hewlett Packard, Cupertino.
> > > email : santoshr at cup.hp.com
> > > Phone : 408-447-3751
> > > ##################################
> > > *
> > > * For T10 Reflector information, send a message with
> > > * 'info t10' (no quotes) in the message body to majordomo at t10.org
> > >
> > *
> > * For T10 Reflector information, send a message with
> > * 'info t10' (no quotes) in the message body to majordomo at t10.org
> >
> >
> >
> >
*
* For T10 Reflector information, send a message with
* 'info t10' (no quotes) in the message body to majordomo at t10.org




More information about the T10 mailing list