My question has had to do with differentiating the
disaster clean up
case from the non-cooperating host
case.
How do I clean up from a disaster? If all my
"reserved" initiators
melt down, and there aren't any of them left anymore
(because of
a site disaster, or whatever), how does some other
node come along
and clean up so it can gain
access?
Would it require manual intervention? Or, is there a way in the
protocol
that
I can register and preempt the group reservation (does the
use
of
the SPEC_I_PT bit allow this as you have suggested Roger).
I
thought the SPEC_I_PT had to come from an already
registered
initiator (which in a disaster, none exist
anymore).
Fred Knight
Kevin,
I'm sorry, I don't think it's as cut and dried as you
make out. This gets into some of the corner cases that I listed in my first
response.
The point to be made in response to Fred's case is
that a third-party can create registrations for a downed initiator (via
the SPEC_I_PT) bit, so that when it comes up again it will be able to
participate in the reservation without having to register
itself.
Also, you say that "We have made provisions for
adding members once the reservation exists, but only one of the reservation
holders can add another entity." Two things in response to
that:
1) I didn't see any specific provision for adding
members in your proposal, so I presume you'd just issue another RESERVE with
the same type and the whole list of transport IDs to be included again, and
thus the Target would have a whole lot of work to do again to set up another
reservation.
2) I that really what you want, that an member of the
existing group can reissue the RESERVE with a whole bunch of different
TransportIDs, perhaps excluding some that were previously
there?
Regards,
Roger
From: owner-t10@t10.org
[mailto:owner-t10@t10.org] On Behalf Of Kevin D Butt
Sent:
Monday, December 17, 2007 3:54 PM
To: Knight,
Frederick
Cc: t10@t10.org; Christine R Knibloe
Subject:
RE: Persistent Reservation Proposal - Group Reservations
Fred,
This is being proposed for SPC.
There are multiple types of reservations. In an
environment where one node of a cluster must join later, one of the other
types can be used. Either that or have an existing node in your
cluster add the new node. The whole intent of this Group reservation
is to lock out everybody that is not explicitly specified during the
reserve. We have made provisions for adding members once the
reservation exists, but only one of the reservation holders can add another
entity. The new entity cannot add itself. This is the whole
point of reservations (i.e., lock out others from doing stuff while I think
I have exclusive rights).
To put
it in other word's, to allow somebody to join the reservation of their own
accord without permission is EXACTLY what I am trying to protect
against.
Thanks,
Kevin D. Butt
SCSI & Fibre
Channel Architect, Tape Firmware
MS 6TYA, 9000 S. Rita Rd., Tucson, AZ
85744
Tel: 520-799-2869 / 520-799-5280
Fax: 520-799-2723
(T/L:321)
Email address:
kdbutt@us.ibm.com
http://www-03.ibm.com/servers/storage/
| "Knight, Frederick"
<Frederick.Knight@netapp.com>
12/17/2007 01:38 PM
|
|
To
| Kevin D
Butt/Tucson/IBM@IBMUS
|
|
cc
|
|
|
Subject
| RE: Persistent Reservation
Proposal - Group Reservations |
|
Sorry, you can't require everyone to register
before the reserve.
That's like saying my whole cluster can't boot
because 1 node is down. You need
to have a way for a "down" initiator to join the fun after
the fact.
I helped write a host cluster product that used a shared
tape (failover model). The
backup application would write to the tape. If a system failure
ever happened, the
backup
application would failover to a different host. It would skip
backwards on
the tape for a
few records, recognize where it left off, and then resume operation.
BUT, for some protection, we used
reservations to make sure only 1 initiator at a
time could access the tape. The interesting point
however, is that we were in the
process of upgrading from old SCSI-2 RESERVE to using PR.
Because, we also
have
multiple HBAs in the host, and we wanted to be able to use more than 1
of
those HBAs (so we needed
multiple reservations - aka PR). Having this idea
(group reservations) would have been a real
nice addition.
As for the RA/AR differences. It seemed to be
timing. Registrants Only was fairly
early on (as I remember), and so implemented by several
O/S vendors. Later on,
some issues were found (which got complicated spec-ees added to
address), but also,
the All
Registrants was added (which didn't have those issues). But, since
there were
implementations, it
couldn't be removed like the other old PR types that no one ever
used. Anyway, I agree, they
offer basically the same capabilities, but RO is already
out there, and AR is probably what new
implementers are using (it's easier to understand
and implement from the host side). Most
of the differences are already documented,
so there wouldn't be that much extra for you to write to
have both types (which I think
would be better than bit somewhere - do it the same way all the
others are done). But,
you could also just do the AR version, and let someone else add the
RO version if they
want
it.
Are you proposing this for tape only? or SPC in general? I
assume SPC in general.
Fred Knight
From: Kevin D Butt
[mailto:kdbutt@us.ibm.com]
Sent: Monday, December 17, 2007 9:51
AM
To: Roger Cummings
Cc: t10@t10.org
Subject:
RE: Persistent Reservation Proposal - Group Reservations
Roger,
Thank you for your
feedback. I am certainly willing to entertain other methods for
accomplishing the end goal in an easier fashion. I am not sure I
understand how your proposed method makes it more backward compatible.
In my proposal PRin would show a different type of reservation and
hence the application clients would not try to join the reservation because
they don't know about the type. In your proposal, application clients
would not be allowed to register. This is a deviation from what they
can always do today - unless there is a resource issue. This seems
more disruptive to me. I would assume that there would be a new
additional sense code added for UNABLE TO REGISTER BECAUSE A GROUP
RESERVATION IS IN PLACE (or analogous). This would be a new thing for
failure to register and there would be pain at the register point.
Perhaps that is better than at the reserve point - but I would think
that it would be better handled as a reservation conflict since that is what
it is instead of something the application client does not
understand.
As for "all registrants" type vs. "registrants only" I didn't see
where the difference would be interesting, but I am not opposed to providing
a way to switch between which of these two types is done. Whether it
is additional types or some bit during registration etc.
As for some of the corner cases
mentioned below, if each I_T nexus that is supposed to be part of the group
reservation is required to be registered before the reservation is made, and
if the reservation is released when the last group reservation participant
is unregistered, then I think we don't have an issue.
I would prefer that we work
together to shape a mutually beneficial proposal as opposed to have
"competing" proposals. I am willing to modify my proposal where it can
be made easier and such. I am not sold that my proposed method is the
only way or even the best - it's just the way I thought of doing it. I
admit that I have always been very confused about the usefulness of RA and
AR types. They make absolutely no sense in the tape world.
Thanks,
Kevin D. Butt
SCSI
& Fibre Channel Architect, Tape Firmware
MS 6TYA, 9000 S. Rita Rd.,
Tucson, AZ 85744
Tel: 520-799-2869 / 520-799-5280
Fax: 520-799-2723
(T/L:321)
Email address:
kdbutt@us.ibm.com
http://www-03.ibm.com/servers/storage/
"Roger Cummings"
<roger_cummings@symantec.com>
Sent by:
owner-t10@t10.org
12/14/2007 12:10 PM
|
|
To
| Kevin D
Butt/Tucson/IBM@IBMUS
|
|
cc
| <t10@t10.org>
|
|
Subject
| RE: Persistent Reservation
Proposal - Group Reservations |
|
Kevin,
First of all, let me say that I
completely support what you're trying to do here. I think that providing a
method in persistent reservations (PRs) to support shared access between
ONLY a specifically-designated set of systems is a worthy goal, and
something we should do in SPC-4.
Adding a set of Transport IDs to Reserve as per your
document 08-024 & 08-025 is certainly possible, but it's a massive
change to the way that PRs work today, and it throws up a bunch of nasty
corner cases and backwards compatibility issues.
The massive change comes from
the fact that now the Target will have to remember which registrations are
in the Reservation, and which are not. It will probably have to preserve all
of the transport information for the life of the reservation.
The corner cases are
things like, what happens if there's no longer a registration that
corresponds to the transport ID in the Reserve? Does the Reserve succeed?
What happens if a registration comes in later, after the reservation has
been established - does that device it get access?
Backwards compatibility issues
may arise like this: An existing device registers, and finds it has no
access, so it does a PR In and finds out that a reservation is in place,
retries its access and still it has no access. What does it do next, preempt
the reservation because it assumes the Target is broken?
Reserve also has to be an
"atomic" command, and I've always thought that was why it's functionality is
as compact as it is today. Most of the complex operations related to
addresses and keys are done at registration time, and those operations don't
have to be atomic.
One more thing: you chose for your new "group" reservations to
follow the "all registrants" approach is terms of the definition of the
reservation holder. While that's fine by me (obviously), I suspect there are
also situations where group reservations that follow the "registrants only"
approach might be useful.
The bottom line from
my point of view is this: Your proposal is feasible and we can probably make
it work. But I wonder if there's an easier way to achieve the same goal that
is more compatible with existing practice and requires less of a change in
functionality on the Target side.
What if we didn't add
any new reservations types, but instead added some new functionality to the
registration process? What I'm thinking of a new Register feature that
causes the Target to kill all existing registrations, create the
registrations identified in the transport IDs in the Register command, and
not accept any future registrations. That way, we don't need any changes to
Reserve, and an Initiator with existing functionality would just not be able
to register and therefore would not be confused.
Does that make sense to you? Is
there a chance this is an easier approach? If so, I'll write up a detailed
proposal that's the equivalent of 08-025r0 and we can compare and contrast
at the next CAP.
Again, thanks for getting this started, I think it's a worthwhile
endeavor and I'll be glad to put some cycles towards defining this sort of
functionality for SPC-4.
Regards,
Roger
From: owner-t10@t10.org
[mailto:owner-t10@t10.org] On Behalf Of Kevin D Butt
Sent:
Monday, December 10, 2007 4:18 PM
To:
t10@t10.org
Subject: Persistent Reservation Proposal - Group
Reservations
I have posted two documents related to an additional
Persistent Reservation Type. The first document is a presentation on
where persistent reservations are today and where they fall short in the
scenarios covered by the proposal. It also covers the intent of the
proposal and what will be proposed. The second is the actual
proposal
Your PDF file will
be posted
at:
http://www.t10.org/ftp/t10/document.08/08-024r0.pdf
http://www.t10.org/ftp/t10/document.08/08-025r0.pdf
Normally,
the posting/archiving process takes about 30 minutes.
Kevin D. Butt
SCSI
& Fibre Channel Architect, Tape Firmware, IBM
MS 6TYA, 9000 S. Rita
Rd., Tucson, AZ 85744
Tel: 520-799-2869 / 520-799-5280
Fax:
520-799-2723 (T/L:321)
Email address:
kdbutt@us.ibm.com
http://www-03.ibm.com/servers/storage/