Persistent Reserve Document, X3T10/95-229r0

Bob Snively Bob.Snively at Eng.Sun.COM
Thu May 25 16:21:47 PDT 1995


The document is available by ftp from playground.sun.com in the
directory pub/SCSIDOCS/ in the file persrsrv.ps

The .mif version is also available.

The text in this letter includes the text of the cover letter, the text of the
introductory comments, and the text of the questions to be reviewed,
just to whet your appetite.

It reads best if you slightly widen your window.

Comments greatfully received.


-------------------------------------------------------------------------------

Bob Snively				         Phone:	   (415) 786-6694
Sun Microsystems Computer Company	         FAX:	   (415) 786-6458
Mail Stop UMPK 12-204
2550 Garcia Avenue			   	 E-mail:   bob.snively at sun.com
Mountain View, CA 94043-1100
-------------------------------------------------------------------------------
COVER LETTER:


John Lohmeyer
Chairperson, X3T10
Symbios Logic Inc.
1635 Aeroplaza Drive
Colorado Springs, Colorado 80916

Subject:	Proposal for Persistent Reservation

The following proposal has been extracted from work by Bill Dallas of DEC, 
Roger Cummings of Storage Technology, and many others. The proposal extends 
the definition of reservations to allow proper behavior in multi-initiator and 
multi-port environments. The proposal defines persistent reservations which 
remain valid across Target Reset and can only be cleared by power down or by a 
properly qualified persistent reservation from another initiator. Using the 
commands defined by this proposal, a host can protect the logical unit from 
improper behavior caused by another initiator on the same or other ports. At the 
same time, the host can determine from a logical unit which initiators share the 
logical unit, which initiator is presently reserving the logical unit, and can 
choose to displace the reservation of an initiator which is known to have failed.

Sincerely,


Robert N. Snively
------------------------------------------------------------------------------

INTRODUCTORY COMMENTS:



			PERSISTENT RESERVATION PROPOSAL


Description of function / Persistent Reservation model:


Two new commands are defined in SPC, PERSISTENT RESERVE IN (PRIN) and 
PERSISTENT RESERVE OUT (PROUT). The commands are used to create and release 
persistent reservations (including logical unit or Extent reservations), to provide reservation 
keys to logical units, and to force actions on tasks from other initiators.

The PRIN and PROUT commands have the following capabilities, many of which are optional:

	Create persistent reservations using PROUT.

		Extent (shared reservations defined)
		Exclusive logical unit
		Shared logical unit (shared reservations defined) 

	Release persistent reservations using PROUT. 

	Register an 8-byte Reservation Key for the sourcing initiator with the 
		attached device using PROUT. 

	Determine the present reservation's key and characteristics using PRIN.           

	Determine the key of other initiators that are attached to the peripheral device using PRIN. 

	Preempt a reservation with another initiator with new reservation from this using PROUT. 

	Optionally automatically clear all tasks related with the preempted reservation. 

	Generate a 32-bit number increasing with each reservation to warn of intervening actions.

A persistent reservation is formed between an initiator and a logical unit when the PROUT 
command containing the parameters requesting a persistent reservation is successfully 
executed. The parameters indicate the type of reservation that is formed. In addition, the 
parameters contain an 8-byte reservation key that is used by the target and by software on other 
initiators to identify the initiator holding the reservation. The persistent reservation cannot be 
released by a Target Reset, other reset activity, or by a RELEASE command. It can be released 
only by a PROUT from the same initiator containing the release parameters, by a power off, or 
by a PROUT from another initiator containing the proper preemptive reservation parameters 
and the reservation key of the initiator holding the reservation. While the persistent reservation 
is active, any conflicting persistent reservation or activity that conflicts with the reservation is 
rejected with RESERVATION CONFLICT status. This behavior allows any cooperating 
initiators having access to the logical unit through any port to execute carefully managed 
reservation protocols that will allow the logical unit to be safely shared among them. The 
reservation is safe from interruption by any initiators not participating in the persistent 
reservation protocol, even during booting and error recovery operations.

The use of a reservation key as part of the reservation process enables initiators to identify 
other ports holding reservations or sharing the logical unit. Hosts use the reservation key to 
perform locking and failover recovery operations. The communication for such algorithms is 
usually through an auxiliary port such as Ethernet or Fibre Channel. Targets with reservations 
held by failing hosts can be identified and preemptively reserved by hosts that are still 
operational. When preemptive reservations are performed, the reserving initiator can 
optionally invoke an automatic clearing of all tasks for the initiator port that is being 
preempted.

The structure of PROUT favors the generation of a single reservation for each PROUT 
command. Alternative implementations are discussed at the end of this proposal. 

The reservation key of other ports that have previously generated persistent reservations can be 
obtained from the logical unit to allow each initiator to monitor which initiator ports have 
participated in the sharing process.

The programming conventions typically used with RESERVE and RELEASE may conflict with 
the programming conventions that are used with PROUT and PRIN. Operating systems should 
only use one of the two reservation command sets at a time with a logical unit.


----------------------------------------------------------------------------------------------

DISCUSSION ITEMS:



Arbitrary decisions made developing this document, please review:

1) Single key per initiator

The definition of persistent reservation at present uses a single key for each initiator. The 
definition could use a single key for each reservation, but that would significantly increase the 
space required for keys in the logical unit. The use of multiple keys for an initiator holding 
multiple persistent reservations does not appear to add any useful function.

2) Modification of key

The key is presently specified as being modifiable during persistent reservations by the 
Register action. The key is not modifiable during the execution of additional reservations or 
overlapping reservations to a logical unit. Invalid keys during Reserve actions are 
RESERVATION CONFLICTS.

3) Management of queued tasks

Queued tasks from any initiator continue normally if a PROUT with a Preempt command is 
executed. The assumption is that they have been accepted as nonconflicting and if you really 
wanted to kill them off, you would do so with a Preempt and Clear.

However, a Reserve action can only be performed if there are no conflicting queued tasks. The 
assumption is that the initiators are cooperating and have agreed upon the reservations to be 
performed in such a manner that any conflicts are errors. Note that this may require targets to 
do a lot of task inspection to see if a conflict exists.

For a Preempt and Clear command, only those queued tasks for the specified initiator are 
cleared. Other queued tasks complete normally. Note that this allows the creation of conflicting 
tasks that will be completed normally, unlike the Reserve action. No new conflicting tasks are 
accepted.

4) Command linking

It is assumed that each command of a linked set of commands will be checked for conflicts 
independently, in spite of the fact that they are nominally all part of the same task. This is 
necessary because the initial command may be nonconflicting, but subsequent commands 
conflicting. This is also true for the completion of commands that have not yet been queued but 
are part of an on going task during preemption.

5) Number of reservations per PROUT

At present, the document specifies one reservation per PROUT. This is a requirement for all 
commands except PROUT with the Reserve action, so it seemed natural to make the same 
restriction for that case as well. Theoretically, multiple extents could be reserved at the same 
time, but the Scope and Type parameters are presently in the command, so all reservations 
would have to be identical except for extent. In addition, all reservation keys would have to be 
identical. For now, only one reservation at a time can be performed.

6) Reservation established by preempting initiator

There is no mechanism to preempt an initiator's reservations without establishing a reservation 
of some kind with the preempting initiator. A null reservation scope could be established to 
allow preemption without such a new reservation, but this does not seem to be a useful 
function.

7) Target Reset

As presently defined in SAM, TARGET RESET and CLEAR TASK SET clear all tasks for all 
initiators. This proposal retains that property and includes that definition in the proposed 
modifications to clause 5.4 of SPC. Some proposals have suggested that TARGET RESET 
should only reset the tasks for the initiator that generated the task management function. 
Unless the committee directs otherwise, I propose that we leave the definitions unchanged and 
accept the clear understanding that TARGET RESET will mess up all initiators and should not 
be used. At least it will not clear persistent reservations.

8)	Reservation Conflicts

At present, it appears that a RESERVE(6) or RESERVE(10) command does not conflict with a 
reservation from another initiator. I believe it should be required to conflict for all logical unit 
reservations by another initiator and should be required to conflict for overlapping and 
inconsistent extent reservations by another initiator. This will require modification to clause 
5.3 and to the definitions of the RESERVE commands.






More information about the T10 mailing list