T10 DIF for Error correction
shivaram.u at quadstor.com
Wed May 15 03:03:01 PDT 2013
Formatted message: <a href="http://www.t10.org/cgi-bin/ac.pl?t=r&f=r1305150_f.htm">HTML-formatted message</a>
I am looking a way for automatic error correction (self healing) using T10
protection information. I have written down a possible solution below.
a) will it work ?
b) Is such a thing already supported by implementations
c) Easier, obvious alternatives
Feedback, pointers and criticism will be very helpful.
Note: A copy can be got from
Many filesystems now incorporate self healing when a corrupt data block is
detected. However self healing requires RAID within the filesystem/volume
manager layer or at least functionality to retrieve an alternate copy for
verification and writing back the good copy. Many filesystems/applications
would need to change significantly to incorporate such techniques which
Guard checksums (T10 DIF) already allows for verifying the integrity of the
data received and stored. A device can (if it supports) automatically do
handling if guard checking is enabled. For example on a READ
1. Compute the guard for the data block
2. Verify that the guard stored is the guard which is computed
3. If not, retrieve an alternate block if available do steps 1, 2 etc and
found to be good, return that data. Additionally if possible, fix the
Retrieving an alternate block could be as simple as reading a redundant
|from an alternate disk in a mirrored configuration. In the case of for
a triple mirrored configuration, the redundant copy can be more than one.
In the case of a RAID configuration where parity is used, retrieving an
alternate block would amount recreating the block from the parity block and
all other blocks other than the incorrect block.
However there does exist the possibility (in theory) that the final block
received by the application is not what it expected. T10 DIF allows an
application to store a tag with the data and receive the same on a READ. If
during a READ there is a tag mismatch, the application can assume an
Using Application Tags for error correction
Lets suppose due to stale data on one of the disks or probably due a CRC 32
hash collision, the guard checksum matches, and the data is sent back to
Assuming that the application is capable of determining that the data it
received is not what it expected (application tag mismatch), the
tag can be useful then to inform the device to look for redundant copies.
However the application tag verification can be complicated to use.
Application tag verification can be enabled in the control mode page and
device can be notified of the expected application tag for a certain range
logical blocks using the application tag mode page. This applies to all
and WRITE commands except for the 32 byte CDB variants
In the case of the 32 byte CDB variants the expected tag is passed along
the CDB itself.
Now this is fine if the application tags used are a small number of ranges.
However if the application wishes to use a different tag for each logical
block (or a very small range of logical blocks, say per filesystem block)
will be cumbersome to set mode pages for all the application tags. And
is no reason for an application to know the expected tag for a block in
advance. Due to this limitation the following will work only with
type 2 (??? or is it possible otherwise)
One good way to use application tag would be
1. Send the application tags during WRITE 32 but disable application tag
checking. Guard check will ensure that the data received is order and the
application tag will be stored along with the block
2. Receive the data without an application tag checking. The application
are sent along in the READ 32 response without any checks.
3. If the application receives the tag it expects, then fine.
4. If not reissue the read command, this time with application tag check
enabled. This would be by Issuing a READ 32 command per logical block or
of blocks with the same application tags, with the expected application tag
specified in the CDB itself.
5. Now when the device reads the blocks and finds a mismatch in the
application tag it could then do the appropriate error handling, which
a. Do nothing in which case the application has unexpected data as before
b. Retrieve an alternate copy for which the guard checks fine and the
application tag matches. In this case the application has a good copy of
c. Retrieve an alternate copy for which the guard check fails or the
application tag still has a mismatch. This is equivalent to (a) from an
d. In the case of (b), the device can transparently write back the good
data, guard and application tag to the incorrect block
When things may not work
1. SBC3 r35c mentions data deduplication and states "De-duplication shall
affect protection information, if any.". There is a definite possibility of
losing an Application Tag over here. How are application tags and reference
tags handled here ?
One possibility is that the implementation deduplicates the application
but still has the ability to retain the original application tag elsewhere.
However this would mean that when returning the application data, the
will return the retained original application tag with the incorrect data.
This would mean that the application would see the correct application tag
incorrect data. But this is really left to the device implementation to
this situation in the first place.
2. Unless the application tag used are a small number, only Type 2
More information about the T10