XPWRITE data integrity (and 03-381r0)
RRose at exabyte.com
RRose at exabyte.com
Mon Dec 22 09:17:22 PST 2003
* From the T10 Reflector (t10 at t10.org), posted by:
* RRose at exabyte.com
*
This message is in MIME format. Since your mail reader does not understand
this format, some or all of this message may not be legible.
------_=_NextPart_001_01C3C8AF.7667EC20
Content-Type: text/plain;
charset="iso-8859-1"
On Read operations, most tape drives do support some means of
block-level addressing; however, the blocks are still variable sized.
The device typically implements some combination of indexing, heuristics
and counting blocks along a track to locate a specific block.
Tape drives are not normally block addressable on Write operations;
although, there are exceptions with varying restrictions. The
compromises to permit block-addressable writes may include a preset
fixed-block size, preformatted tapes, significant loss of capacity to
provide splice points and significantly decreased I/O throughput.
Also, the fact that a disk is fixed block doesn't imply that the backup
will be fixed block, since the blocks on tape consist data from multiple
disk blocks and possibly multiple disk drives. A disk block may contain
fragments from several files. On a file-by-file backup, these fragments
are gathered from wherever they reside and sent to the backup device.
There may also be header labels, trailer labels and
application-generated indexes which are likely to be of a different
size.
Until some form of an O/S-independent standard tape file system gains
broad acceptance, variable-length tape blocks are pretty much needed for
portability between O/S's and app's. People expect that data written
|from their proprietary-O/S mainframes be readable on their
different-proprietary-O/S pc's and workstations. There may be solutions
which don't involve a tape filesystem or variable-length blocks, but
some broadly-implemented, O/S-independent, standard method of encoding
the exact length of each item of data needs to exist.
-roger rose
Exabyte Corp.
> -----Original Message-----
> From: Gerry.Houlder at seagate.com [ mailto:Gerry.Houlder at seagate.com
]
> Sent: Friday, December 19, 2003 4:31 PM
> To: t10 at t10.org
> Subject: RE: XPWRITE data integrity (and 03-381r0)
>
>
> * From the T10 Reflector (t10 at t10.org), posted by:
> * Gerry.Houlder at seagate.com
> *
>
> By the way, Keith, the backup / restore operation you
> describe here that
> isn't restored to the same LBA it was backed up from (thus
> resulting in
> mis-matched reference tags) can be recovered by using the 32
> byte commands
> that allow for specifying a non-interlocked reference tag
> value. This is a
> new reason for those commands that was not brought up during earlier
> discussions.
>
> On the CRC seed value subject, I have heard some tape folks
> indicate that
> most new tape drives implement block addressing commands. I have heard
> statements that all new tape drives support this command
> mode. This makes
> the tape look more like a fixed block device (at least in
> terms of the data
> block that are read and written). Does this partly negate the
> concern about
> not knowing the length of each block, since it is usually
> known for such
> cases? Is the length concern mostly about older backups make
> with devices
> that don't support the fixed block mode and new backups that
> choose not to
> use the block mode even if it is available on the device?
> Could we expect
> that tapes that know about E2E feature will always use fixed block
> addressing? Also if a tape is used to back up a disk,
> wouldn't the fixed
> length blocks from the disk always result in the same fixed block size
> being present on the tape (unless compression is used)?
>
>
>
>
>
> "Holt, Keith"
>
> <kholt at lsil.com> To:
> "'George Penokie'" <gop at us.ibm.com>
> Sent by: cc: T10
> Reflector <t10 at t10.org>
> owner-t10 at t10.org Subject: RE:
> XPWRITE data integrity (and 03-381r0)
> No Phone Info
>
> Available
>
>
>
> 12/19/2003 02:21
>
> PM
>
>
>
>
>
>
>
>
>
> George,
>
> It was not my intent to imply that tape devices are lower
> forms of life
> that are not worthy of end-to-end data protection. Far from
> it. In fact,
> I would like to see some way to preserve the block storage data guard
> across backup/restore operations. As discussed in the November CAP
> meeting, it's not feasible to expect to preserve the
> Reference Tag, since
> the file will likely be restored to a different logical block address,
> perhaps even to a different logical unit altogether. The
> data guard, on
> the other hand, could reasonably be preserved while moved from disk to
> tape, then back again. I look forward to seeing some
> specific proposals
> regarding E2E protection for tape devices on how
> variable-length block tape
> E2E protection could be applied while preserving fixed-length
> block disk
> E2E protection.
>
> Since I obviously didn't get my point across the way I
> intended, I'll try
> again. Please find a way of adding E2E protection to tape
> command sets
> that doesn't require a dilution of E2E protection for RAID sets that
> utilize the block storage command set.
>
> Regards,
>
> Keith Holt
>
>
> --
> Keith Holt
> LSI Logic Storage Systems Inc.
> keith.holt at lsil.com
> 316-636-8665
>
>
> -----Original Message-----
> From: George Penokie [ mailto:gop at us.ibm.com
]
> Sent: Thursday, December 18, 2003 2:21 PM
> To: Holt, Keith
> Cc: owner-t10 at t10.org; T10 Reflector
> Subject: RE: XPWRITE data integrity (and 03-381r0)
>
>
> Keith,
>
> So is the next step to obsolete from SCSI all non-block devices
> because block devices don't need to communicate with
> those lowly tape
> devices.
>
> Bye for now,
> George Penokie
>
> Dept 2C6 114-2 N212
> E-Mail: gop at us.ibm.com
> Internal: 553-5208
> External: 507-253-5208 FAX: 507-253-2880
>
>
>
>
>
> "Holt, Keith"
>
> <kholt at lsil.com> To: T10
> Reflector
> Sent by: owner-t10 at t10.org <t10 at t10.org>
>
> cc:
>
> Subject: RE:
> XPWRITE data
> 12/18/2003 01:22 PM integrity (and 03-381r0)
>
>
>
>
>
>
>
>
> I want to call attention to an important characteristic
> of the simple
> function. As Jim pointed out, if you use a seed of
> zero, you can
> still check the data guard. By using a seed of zero,
> the XOR'd CRC
> of all the data drives is, by definition, equivalent to
> the CRC that
> is calculated from the data in the parity drive. You
> can still check
> the CRC without performing any additional operations.
>
>
> This benefit is realized if and only if we stick with
> our original
> plan of record to use zero for the seed for CRC
> calculations. A seed
> of zero was chosen for block storage data guards
> specifically for
> this reason. There is a proposal from Rob Elliott on
> the agenda for
> the January CAP meeting, 03-381r0, SBC-2 Data
> Protection CRC Seed,
> that proposes that the seed for block storage E2E protection be
> changed to 1-seeded and transmitted inverted, solely
> because it's
> beneficial for tape devices. In his proposal, Rob
> acknowledges that
> a benefit is lost to a RAID set implementing RAID 5 if
> the proposed
> change is implemented. This same benefit would be lost to the
> XPWRITE simple function, as well.
>
>
> I can't dispute that it would simplify things for the
> HBA if disk and
> tape E2E protection used identical CRC generation
> methods. Neither
> can I dispute that having a non-zero seed would be
> better for tape
> devices. However, no one has attempted to dispute the fact that
> zero-seeded is better for a block storage
> specification. It seems to
> me that requiring disk devices to use non-zero seeds
> vs. requiring
> tape devices to use a zero seed are roughly equivalent
> trade-offs.
> Being more interested in block storage devices, I am
> obviously going
> to argue that we shouldn't have to change the seed for
> block storage
> E2E protection because it offers some incremental
> benefit to tape
> devices, at the expense of losing protection on block
> storage RAID
> set commands.
>
>
> Keith
>
>
> --
> Keith Holt
> LSI Logic Storage Systems Inc.
> keith.holt at lsil.com
> 316-636-8665
>
>
> -----Original Message-----
> From: George Penokie [ mailto:gop at us.ibm.com
]
> Sent: Tuesday, December 16, 2003 8:58 AM
> To: Jim.Coomes at seagate.com
> Cc: owner-t10 at t10.org; T10 Reflector
> Subject: RE: XPWRITE data integrity
>
>
> Jim,
>
> I vote for simple. The end-to-end protection is primarily for
> end-to-end protection not side-to-side protection
> (e.g., RAID-5). If
> it happens to help out on side-to-side protection then
> that's nice
> but I see no reason to add in complexity to accomplish that.
>
> Bye for now,
> George Penokie
>
> Dept 2C6 114-2 N212
> E-Mail: gop at us.ibm.com
> Internal: 553-5208
> External: 507-253-5208 FAX: 507-253-2880
>
>
>
>
> Jim.Coomes at seagate.com
>
> Sent by: owner-t10 at t10.org To: T10
> Reflector
> <t10 at t10.org>
>
> cc:
>
> 12/15/2003 04:28 PM Subject: RE:
> XPWRITE data
> integrity
>
>
>
>
>
>
>
>
>
> * From the T10 Reflector (t10 at t10.org), posted by:
> * Jim.Coomes at seagate.com
> *
> The incorporation of the data integrity proposal into
> SBC-2 raises a
> question: how does XPWRITE function with data
> integrity? Below are
> several
> possible approaches. Please voice your opinions to the
> reflector or
> to me
> offline.
>
> Simple function
>
> The simple approach would be to require the protection
> information on
> the
> parity drive be the XOR product of the protection
> information on the
> data
> drives. In this case, little checking of the protection
> information
> is
> possible. The only checking may be the guard field if
> the CRC is not
> seeded.
>
> With this XPWRITE function the protection information
> for a failed
> data
> drive would be regenerated the same way as the user
> data, XOR product
> of
> the parity drive and the remaining data drives.
>
> Robust function
>
> To validate the transfer of data to and from the parity
> drive, the
> same
> functionality as for normal writes and reads could be
> required. The 3
> bit
> WRPROTECT field could be added to the XPWRITE command.
> The parity
> drive
> would check the protection information as specified by
> the WRPROTECT
> field.
> The parity drive would write the protection information
> to the media
> without XORing with the previous protection information. A read
> access to
> the parity drive could check the protection information
> as specified
> by the
> RDPROTECT field in the read command.
>
> With this XPWRITE function the protection information
> for a failed
> data
> drive would have to be recalculated.
>
> Jim Coomes
>
> 952-402-2665
>
>
>
> *
> * For T10 Reflector information, send a message with
> * 'info t10' (no quotes) in the message body to
> majordomo at t10.org
>
>
>
>
>
>
>
> *
> * For T10 Reflector information, send a message with
> * 'info t10' (no quotes) in the message body to majordomo at t10.org
>
------_=_NextPart_001_01C3C8AF.7667EC20
Content-Type: text/html;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
RE: XPWRITE data integrity (and 03-381r0) On Read operations, most tape drives do support some = means of block-level addressing; however, the blocks are still variable = sized. The device typically implements some combination of = indexing, heuristics and counting blocks along a track to locate a = specific block. Tape drives are not normally block addressable on = Write operations; although, there are exceptions with varying = restrictions. The compromises to permit block-addressable writes = may include a preset fixed-block size, preformatted tapes, significant = loss of capacity to provide splice points and significantly decreased = I/O throughput. Also, the fact that a disk is fixed block doesn't = imply that the backup will be fixed block, since the blocks on tape = consist data from multiple disk blocks and possibly multiple disk = drives. A disk block may contain fragments from several = files. On a file-by-file backup, these fragments are gathered = from wherever they reside and sent to the backup device. There = may also be header labels, trailer labels and application-generated = indexes which are likely to be of a different size. Until some form of an O/S-independent standard tape = file system gains broad acceptance, variable-length tape blocks are = pretty much needed for portability between O/S's and app's. = People expect that data written from their proprietary-O/S mainframes = be readable on their different-proprietary-O/S pc's and = workstations. There may be solutions which don't involve a tape = filesystem or variable-length blocks, but some broadly-implemented, = O/S-independent, standard method of encoding the exact length of each = item of data needs to exist. -roger rose
Exabyte Corp. > -----Original Message-----
> From: Gerry.Houlder at seagate.com [mailto:Gerry.Houlder at seagate.c= om]
> Sent: Friday, December 19, 2003 4:31 PM
> To: t10 at t10.org
> Subject: RE: XPWRITE data integrity (and = 03-381r0)
>
>
> * From the T10 Reflector (t10 at t10.org), posted = by:
> * Gerry.Houlder at seagate.com
> *
>
> By the way, Keith, the backup / restore = operation you
> describe here that
> isn't restored to the same LBA it was backed up = from (thus
> resulting in
> mis-matched reference tags) can be recovered by = using the 32
> byte commands
> that allow for specifying a non-interlocked = reference tag
> value. This is a
> new reason for those commands that was not = brought up during earlier
> discussions.
>
> On the CRC seed value subject, I have heard = some tape folks
> indicate that
> most new tape drives implement block addressing = commands. I have heard
> statements that all new tape drives support = this command
> mode. This makes
> the tape look more like a fixed block device = (at least in
> terms of the data
> block that are read and written). Does this = partly negate the
> concern about
> not knowing the length of each block, since it = is usually
> known for such
> cases? Is the length concern mostly about older = backups make
> with devices
> that don't support the fixed block mode and new = backups that
> choose not to
> use the block mode even if it is available on = the device?
> Could we expect
> that tapes that know about E2E feature will = always use fixed block
> addressing? Also if a tape is used to back up a = disk,
> wouldn't the fixed
> length blocks from the disk always result in = the same fixed block size
> being present on the tape (unless compression = is used)?
>
>
>
> ; ; ; ; ;
> ; ; ; ; =
> ; ; ;Holt, = Keith; nbsp; nbsp;
> ; ; ; ; =
> ; ; = <kholt at lsil.com> = To:
> ;'George Penokie'; = <gop at us.ibm.com> bsp; bsp;
> ; ; Sent = by: sp; cc: T10 =
> Reflector = <t10 at t10.org> ; ; ;
> ; ; owner-t10 at t10.org = Subject: RE:
> XPWRITE data integrity (and = 03-381r0) sp;
> ; ; No Phone = Info bsp; bsp;
> ; ; ; ; =
> ; ; = Available sp; sp;
> ; ; ; ; =
> ; ; ; ; ;
> ; ; ; ; =
> ; ; 12/19/2003 = 02:21 nbsp; =
> ; ; ; ; =
> ; ; = PM p; p; p;
> ; ; ; ; =
> ; ; ; ; ;
> ; ; ; ; =
> ; ; ; ; ;
> ; ; ; ; =
>
>
>
>
> George,
>
> It was not my intent to imply that tape devices = are lower
> forms of life
> that are not worthy of end-to-end data = protection. Far from
> it. In fact,
> I would like to see some way to preserve the = block storage data guard
> across backup/restore operations. As = discussed in the November CAP
> meeting, it's not feasible to expect to = preserve the
> Reference Tag, since
> the file will likely be restored to a different = logical block address,
> perhaps even to a different logical unit = altogether. The
> data guard, on
> the other hand, could reasonably be preserved = while moved from disk to
> tape, then back again. I look forward to = seeing some
> specific proposals
> regarding E2E protection for tape devices on = how
> variable-length block tape
> E2E protection could be applied while = preserving fixed-length
> block disk
> E2E protection.
>
> Since I obviously didn't get my point across = the way I
> intended, I'll try
> again. Please find a way of adding E2E = protection to tape
> command sets
> that doesn't require a dilution of E2E = protection for RAID sets that
> utilize the block storage command set.
>
> Regards,
>
> Keith Holt
>
>
> --
> Keith Holt
> LSI Logic Storage Systems Inc.
> keith.holt at lsil.com
> 316-636-8665
>
>
> = -----Original Message-----
> From: = George Penokie [mailto:gop at us.ibm.com]
> Sent: = Thursday, December 18, 2003 2:21 PM
> To: Holt, = Keith
> Cc: = owner-t10 at t10.org; T10 Reflector
> Subject: = RE: XPWRITE data integrity (and 03-381r0)
>
>
> = Keith,
>
> So is the = next step to obsolete from SCSI all non-block devices
> because = block devices don't need to communicate with
> those lowly tape
> = devices.
>
> Bye for = now,
> George = Penokie
>
> Dept = 2C6 114-2 N212
> = E-Mail: gop at us.ibm.com
> = Internal: 553-5208
> External: = 507-253-5208 FAX: 507-253-2880
>
>
>
> ; ; ; ; ;
> ;
> ;Holt, = Keith; nbsp; nbsp; nbsp;
> ;
> = <kholt at lsil.com> bsp; = To: T10
> = Reflector sp;
> Sent by: = owner-t10 at t10.org = <t10 at t10.org> ;
> ;
> ; ; ; = cc: sp;
> ;
> nbsp; nbsp; nbsp; = Subject: RE:
> XPWRITE data
> 12/18/2003 01:22 = PM integrity (and = 03-381r0)
> ;
> ; ; ; ; ;
> ;
>
>
>
>
>
> I want to = call attention to an important characteristic
> of the simple
> = function. As Jim pointed out, if you use a seed of
> zero, you can
> still check = the data guard. By using a seed of zero,
> the XOR'd CRC
> of all the = data drives is, by definition, equivalent to
> the CRC that
> is = calculated from the data in the parity drive. You
> can still check
> the CRC = without performing any additional operations.
>
>
> This = benefit is realized if and only if we stick with
> our original
> plan of = record to use zero for the seed for CRC
> calculations. A seed
> of zero was = chosen for block storage data guards
> specifically for
> this = reason. There is a proposal from Rob Elliott on
> the agenda for
> the January = CAP meeting, 03-381r0, SBC-2 Data
> Protection CRC Seed,
> that = proposes that the seed for block storage E2E protection be
> changed to = 1-seeded and transmitted inverted, solely
> because it's
> beneficial = for tape devices. In his proposal, Rob
> acknowledges that
> a benefit = is lost to a RAID set implementing RAID 5 if
> the proposed
> change is = implemented. This same benefit would be lost to the
> XPWRITE = simple function, as well.
>
>
> I can't = dispute that it would simplify things for the
> HBA if disk and
> tape E2E = protection used identical CRC generation
> methods. Neither
> can I = dispute that having a non-zero seed would be
> better for tape
> = devices. However, no one has attempted to dispute the fact = that
> zero-seeded = is better for a block storage
> specification. It seems to
> me that = requiring disk devices to use non-zero seeds
> vs. requiring
> tape = devices to use a zero seed are roughly equivalent
> trade-offs.
> Being more = interested in block storage devices, I am
> obviously going
> to argue = that we shouldn't have to change the seed for
> block storage
> E2E = protection because it offers some incremental
> benefit to tape
> devices, at = the expense of losing protection on block
> storage RAID
> set = commands.
>
>
> = Keith
>
>
> --
> Keith = Holt
> LSI Logic = Storage Systems Inc.
> = keith.holt at lsil.com
> = 316-636-8665
>
>
> = -----Original Message-----
> From: = George Penokie [mailto:gop at us.ibm.com]
> Sent: = Tuesday, December 16, 2003 8:58 AM
> To: = Jim.Coomes at seagate.com
> Cc: = owner-t10 at t10.org; T10 Reflector
> Subject: = RE: XPWRITE data integrity
>
>
> Jim,
>
> I vote for = simple. The end-to-end protection is primarily for
> end-to-end = protection not side-to-side protection
> (e.g., RAID-5). If
> it happens = to help out on side-to-side protection then
> that's nice
> but I see = no reason to add in complexity to accomplish that.
>
> Bye for = now,
> George = Penokie
>
> Dept = 2C6 114-2 N212
> = E-Mail: gop at us.ibm.com
> = Internal: 553-5208
> External: = 507-253-5208 FAX: 507-253-2880
>
>
> ; ; ; ; ;
> ;
> = Jim.Coomes at seagate.com bsp; bsp; bsp;
> ;
> Sent by: = owner-t10 at t10.org = To: T10
> = Reflector
> ; ; = <t10 at t10.org> ;
> ;
> ; ; ; = cc: sp;
> ;
> 12/15/2003 04:28 = PM p; = Subject: RE:
> XPWRITE data
> ; ; = integrity sp;
> ;
> ; ; ; ; ;
> ;
>
>
>
>
>
>
> * From the = T10 Reflector (t10 at t10.org), posted by:
> * = Jim.Coomes at seagate.com
> *
> The = incorporation of the data integrity proposal into
> SBC-2 raises a
> question: = how does XPWRITE function with data
> integrity? Below are
> = several
> possible = approaches. Please voice your opinions to the
> reflector or
> to = me
> = offline.
>
> Simple = function
>
> The simple = approach would be to require the protection
> information on
> the
> parity = drive be the XOR product of the protection
> information on the
> data
> drives. In = this case, little checking of the protection
> information
> is
> possible. = The only checking may be the guard field if
> the CRC is not
> = seeded.
>
> With this = XPWRITE function the protection information
> for a failed
> data
> drive would = be regenerated the same way as the user
> data, XOR product
> of
> the parity = drive and the remaining data drives.
>
> Robust = function
>
> To validate = the transfer of data to and from the parity
> drive, the
> same
> = functionality as for normal writes and reads could be
> required. The 3
> bit
> WRPROTECT = field could be added to the XPWRITE command.
> The parity
> = drive
> would check = the protection information as specified by
> the WRPROTECT
> = field.
> The parity = drive would write the protection information
> to the media
> without = XORing with the previous protection information. A read
> access = to
> the parity = drive could check the protection information
> as specified
> by = the
> RDPROTECT = field in the read command.
>
> With this = XPWRITE function the protection information
> for a failed
> data
> drive would = have to be recalculated.
>
> Jim = Coomes
>
> = 952-402-2665
>
>
>
> *
> * For T10 = Reflector information, send a message with
> * 'info = t10' (no quotes) in the message body to
> majordomo at t10.org
>
>
>
>
>
>
>
> *
> * For T10 Reflector information, send a message = with
> * 'info t10' (no quotes) in the message body to = majordomo at t10.org
>
------_=_NextPart_001_01C3C8AF.7667EC20--
More information about the T10
mailing list