Pat's many comments about SBP-2

Eric Anderson ewa at apple.com
Thu Jul 5 17:27:08 PDT 2001


* From the T10 Reflector (t10 at t10.org), posted by:
* "Eric Anderson" <ewa at apple.com>
*
Hi Pat,

It's great to see someone so enthusiastic about SBP.  But you have
read far between (and perhaps outside) the lines of what I wrote.
Many of your questions and statements don't seem to follow from what
I said.  I have a few short comments to points you have raised, but
I will probably not be able to answer any other questions.

The performance loss from double-buffering depends on many factors
and will vary greatly, but it is always measurable and usually
quite noticeable.  Otherwise I would not have complained about it.

I can not identify ("celebrate") the silicon that is bug-free.  That
would identify the others by exclusion.  Sorry.  If you require details
of what kinds of I/O will fail, and how to work around them, you will
have to work with the bridge vendors.  They have generally been quite
cooperative about debugging these problems, and will provide such
help to anyone who will buy their product (or support it in an OS).
Consequently I have no desire to drag them through the mud on a
public mailing list.

For historical information about SBP-2 you might want to look at the
drafts archived at www.t10.org.  There might even be minutes from
those meetings.

If you have specific suggestions for clarifications or improvements
to SBP, the SBP-3 mailing list would be a good place to post them.

Eric


====================================================
On Wed, Jul 4, 2001 6:23 AM, Pat wrote:

> <ewa at apple.com> 07/04/01 01:00 AM
> t10 at t10.org

> Apple uses the "unrestricted page table"
> exclusively (except for the occasional
> ORB with no page table at all)

Ah, another chink of empirical character.  Helpful, thank you.

> Several commercially available
> 1394-ATA bridges have bugs in this area,
> and we painfully double-buffer some I/O
> to them as a result,
> at a great loss in performance.  

Holding to my focus on empirical reality ...

Can you confirm that by a "great loss" you mean something measurable?

You have a bridge to a drive, you change only the bridge by only fixing
this bug, and you see human-perceived speed increase notably?  You change
the bridge back and you see speed decrease notably?

> ... some I/O ... 

Or is "some" the key word here?  Do you mean to say something less direct -
less scientific - something more open to conflicting interpretation?

Since most of the unrestricted page tables I see are in fact normalised to
physical page size, me, I would expect to see the potential of losing much
speed to double-buffering become actual only in the rare case where the
page table is in fact not normalised.  (I hear the Windows version of
QuickTime is good at producing such cases.)

Aye, personally, I'm pleased to see people start demanding not only
integrity but also minimum sustained transfer rates from storage devices. 
The broader the understanding of merit, the more measures on which I can
hope to compete well.

But I'd guess we'll see double-buffering for alignment as issue in the real
world only as a rare pause while streaming.  People who move only
not-real-time data wouldn't much care.

How wrong-headed am I here?

> an unconstrained scatter/gathr list,
> [Apple] expose this in ... API,
> and allow drivers to specify ...

Empirically, then, once a given driver agrees to double-buffer, the
device's expensive specified ability to cope with strange page tables is
then never used?

Ouch.  I hate to see a spec published but then little regarded.  I hate to
see generic cost a lot more than device-specific.  Kind of like an
unequally enforced law: people's vision of what the law is grows fuzzy -
inescapably.

> Several commercially available 1394-ATA bridges
> have bugs in this area,
> and we painfully double-buffer
> some I/O to them as a result,
> at a great loss in performance.

> I will not identify them;

Understood your personal obligations preclude you from publically shaming
broken bridges and the broken devices built out of those bridges.

But can we somehow celebrate the bridges and devices known to work?

Especially the bridges known to work when segment byte size modulo 4 is 1,
2, or 3?  (Me, I'm aware of bridges that work for 2 but not for 1 and 3.)

> ask your silicon suppliers
> very clearly if they have this problem or not.

Is there any quasi-public document posted somewhere that draws attention to
this fruitful concentration of misunderstanding?

Sure, me, I'd like to think I would ask about why unrestricted page tables
are unrestricted from day one.

But I'm aware of - self-evidently junior - silicon designers who some time
ago - erroneously and arrogantly - concluded that the English of the spec
can't have possibly meant what it said.  1394 is a quad-based bus,
therefore quad-alignment is "of course" guaranteed.  Flatly wrong, I know. 
But not obviously wrong enough to some people prematurely given more
responsibility than we might now wish.

I'm also aware of people who bought silicon without thinking to ask this
question.

I'm also aware of people who think everything on a 16-bit ATA bus involves
pairs of bytes.  Intensely influential people, like the author of
Microsoft's Win95B Esdi506.pdr Atapi.  he worl

====================================================
On Wed, Jul 4, 2001 6:26 AM, Pat wrote:

> <ewa at apple.com> 07/04/01 01:00 AM
> t10 at t10.org

> start addresses
> that are also not a multiple of four,
> nor the same non-multiple of four
> as the previous start address.

Ah.  You mean to say you know of devices that choke over a nonzero
remainder in byte size divided by four, but only when that remainder
varies?

Thanks again in advance.    Pat LaVarre <p.lavarre at ieee.org>

====================================================
On Wed, Jul 4, 2001 6:31 AM, Pat wrote:

> we could not identify any reason
> to bother with the "normalized page table".
> since support for unrestricted page tables
> is mandatory.

> As a target, you should fully support
> unrestricted page tables,
> and make no assumptions ...

I'd like to carefully distinguish failing to supply mandatory support for a
rare case - the evil into which some silicon vendors have apparently fallen
- from failing to optimise a comparatively rare case.

For example, if I try accessing the blocks of disk drive backwards,
suddenly I see that the people who designed the so-called random-access
disk drive (and maybe the software above it) optimised for forward access. 
Myself, on the bus, I've seen 120X degradation in speed, under this stress.

Being an ex-gate-level designer myself, I can see clearly enough that the
simplest silicon works only with normalized page tables.  This preference
arises from the same causes that lead many RISC CPUs to choke over memory
accesses aligned to less than the word size of the processor.

> failing to optimise a comparatively rare case

Silicon that divides a page table segment of x9E0 into transfers of x800 +
x1E0 is likely to be making two misaligned accesses rather than one
misaligned access followed by an aligned access.

This too is a failure to optimise a common case.

Myself, I'd like to think my first thought would be to not to do it that
way.  Maybe this specific failure to optimise rarely matters?

> we could not identify any reason
> to bother with the "normalized page table".

Somebody somewhere once upon a time invented the normalized page table? 
That person then must have had a reason?  And I've been told unrestricted
page tables came later?  Anybody have a clue who when thought normalize
page tables were a Good Idea (TM)?

Lately I've heard unrestricted page tables "were added for Win98".  Also
added for Apple, I take it?  Did any real hosts ever favour "normalized
page table"s?

> [nonzero page_size]
> offers no advantage to either the OS
> or the target, as far as I can tell,

Can you elaborate why politely setting page_size nonzero to admit the use
of a normalized page table is so obviously useless to you?

Me, coming from Usb, I was surprised to see an ORB with a page table
nowhere tells the target the count of bytes that the initiator expects to
move.  The target only discovers the expected total transfer size
implicitly - by way of summing all the byte sizes of segments, a task it
can complete only by fetching all the elements of the page table.

In that context, I'd say making page_size nonzero to tell the device to
expect a normalized page table is advance, redundant info.

And yes, in general, advance redundant info is useful.  (It's also
perniciously treacherous, in the case where the advance info is wrong -
where it doesn't agree with the info that follows.)

> Apple ... could not identify any reason
> to bother with the "normalized page table".

Then I wish this feature was absent from the spec.

If I won't give offence, may I ask, was Apple present & voting against it? 
(I know I wasn't.)

Ironic to find us now speaking with pain of silicon vendors who could not
identify any reason to bother with the unrestricted page tables.

Thanks again in advance.    Pat LaVarre <p.lavarre at ieee.org>


====================================================
On Wed, Jul 4, 2001 6:54 AM, Pat wrote:

My mail client tells me it quietly truncated my mail titled "sbp2
scatter/gather - "great loss" of speed".  Continuing therefore where I left
off ...

> <ewa at apple.com> 07/04/01 01:00 AM
> t10 at t10.org

I'm also aware of people who think everything on a 16-bit ATA bus involves
pairs of bytes.  Intensely influential people, like the author of
Microsoft's Win95B Esdi506.pdr Atapi.  The world is too full of normal
people.  Soon enough, we're going to find people building stuff that chokes
unless aligned to 64 bit boundaries.

What's the process we have in place to keep the newbie customer from buying
and shipping excessively cheap silicon?  How do we stop this problem from
growing?

> Apple ... painfully double-buffer some I/O ...

I hear the Apple FireWire mass storage driver double-buffers as needed to
align data to x10 byte boundaries?

This is an interesting choice.  x10 is often the size of a cache line on
the motherboard.  As we've been discussing, we know we see trouble if for
all devices we guarantee no alignment.  I wonder if we'll see trouble if
for all devices we guarantee no more than 4 byte alignment?

Thanks again in advance.    Pat LaVarre <p.lavarre at ieee.org>









*
* For T10 Reflector information, send a message with
* 'info t10' (no quotes) in the message body to majordomo at t10.org




More information about the T10 mailing list