X3T9.2/89-020 January 4, 1989 TO: X3T9.3 Fiber Optic Study Group Members FROM: Roger Cummings SUBJECT: FIBER OPTIC CHANNEL WORKING GROUP MINUTES Please find attached a draft of the minutes of the ANSI X3T9.3 Fiber Optic Channel Working Group that was hosted by Jim Smith of Tandem Computers at their facility in Cupertino, CA on December 1 and 2, 1988. Note that there are also nine Attachments to the minutes that relate to presentations at the meeting. The working group mailing list is continuing to grow, and at the last count has 102 names. At this size it becomes an expensive proposition to issue a mailing, especially if the presentations given at the meetings are included along with the minutes. This subject had been discussed previously, and it had been decided to adopt a policy of one mailing per organization, with the person receiving the mailing (the "primary" contact) having the responsibility to distribute it internally. However after further discussion, both at the meeting and with the officers of the X3T9.2 and X3T9.3 committees, this decision has been rescinded. Instead both the minutes and the full text of the presentations will be included in the regular bimonthly X3T9.2 and X3T9.3 mailings, and a separate mailing will not be made. The only exception to this will be that notices of future meetings will be mailed separately if the timing of the bimonthly mailings is not appropriate. THEREFORE ALL MEMBERS OF THE FIBER OPTIC WORKING GROUP WHO DO NOT ALREADY SUBSCRIBE TO EITHER THE X3T9.2 OR X3T9.3 MAILINGS ARE ENCOURAGED TO DO SO NOW IN ORDER THAT THEY MAY RECEIVE THE RESULTS OF FUTURE WORKING GROUP MEETINGS. A subscription form is attached for your convenience. The next meeting of the Fiber Channel Working Group will be held at the Sunnyvale Hilton, 1250 Lakeside Drive, Sunnyvale CA on January 30 and 31. As this is a no-host meeting an attendance fee of $20 per person will be charged to cover the costs of the meeting room. The phone number of the hotel is (408) 738-4888. Further details may be obtained from either Dal Allan of ENDL at (408) 867-6630 or myself. The intention is to agree a schedule and location for all of the Fiber Channel Working Groups for 1989 at this meeting, so all attenders are asked to investigate in advance the possibility of their hosting a meeting. If there are any corrections required to, or omissions noted from, the minutes I can be reached as follows: Phone: Business (416) 826-8640 x3332 Home (416) 625-4074 (ans machine) Telex/MCI Mail: 650-289-5060 (USA) Fax: (416) 821-6363 Regards Roger Cummings Principal Engineer, I/O and Peripherals Systems and Strategies Group Control Data Canada Ltd. 1855 Minnesota Court Mississauga, Ontario L5N 1K7 Canada #ww/rc MINUTES OF THE FOURTH FIBER OPTIC WORKING GROUP MEETING The fourth meeting of the ANSI X3T9.3 Fiber Optic Working Group was hosted by Jim Smith of Tandem Computers at their facility at 10555 Ridgeview Court, Cupertino, CA on December 1 and 2, 1988. Jim also distributed the meeting notice. A total of 34 people attended, as follows: AMD Paul Scott Wayne Wong AMDAHL Masanori Motegi AMP Charles Brill ANCOR COMMUNICATION Terry Anderson AT&T Ming-lai Kao Philip Puglisi AT&T BELL LABS Steve Siegel CANSTAR Karl Lue Shing Kumar Malavalli CDC Wayne Sanderson CDC CANADA Roger Cummings CIPRICO Bill Winterstein CONTROL DATA Frank Holland CRAY RESEARCH INC. Eric Fromm DATA GENERAL David Hartig ENDL I Dal Allan FUJITSU AMERICA Bob Driscal Koji Mori GAZELLE MICROCIRCUITS Chris Popat HEWLETT PACKARD Del Hanson IBM Henry Brandt Ron Soderstrom Horst L Truestedt IBM GENERAL PRODUCTS DIVISION Curtis Wong IMPRIMIS Tom Leland LOS ALAMOS NATIONAL LAB Don Tolmie PILKINGTON PLC Simon Honey PRIME COMPUTER INC. Mike Fitzpatrick SIEMENS Schelto Van Doorn TANDEM COMPUTERS Armando Pauker Duc Pham Phil Sinykin Jim Smith December 1&2 Fiber Optic Working Group Minutes Page 2 The meeting was opened by the Chairman of the Working Group, Dal Allan of ENDL Consulting, who distributed a sheet containing both the agenda for the meeting and a shortform of the functional requirements derived at an earlier meeting. A copy of the sheet is Attachment 1. Dal noted that none of the presenters for the three items identified in the agenda were present at the beginning of the meeting, although he expected the AT&T presentation on Parallel Fiber Applications to take place on the second day of the meeting. Therefore he asked for other presentations from the attenders to commence the business of the meeting. IBM volunteered two presentations which had been prepared in response to the general call for information about standards related to office environment and fiber cabling that had been issued in the last minutes. The first presentation was given by Horst Truestedt of IBM Rochester on the subject of the National Electrical Code (NEC). A copy of Horst's slides is Attachment 2. He described in detail the stringent new requirements that have been mandated by the 1987 NEC revision. Adoption of this revision means that all cables installed in a building that are over 10 feet in length must used UL-listed cable (as opposed to today where the cables merely have to be UL-recognized). Section 725-38 B 1 deines two types of cable, designated CL2 and CL2P respectively, with the difference that the CL2P cables are able to be directly installed in plenums. Because pvc-jacketed polyethylene cables cables cannot pass the cable tray fire test defined by the NEC other materials such as teflon have to be used in plenum cables with a consequent increase in cable diameter, stiffness (bend radius) and cost. The alternative is to use CL2 cables enclosed in a separate, approved cable tray. Horst stated that the cabling space under a false floor may be regarded as a plenum unless it is vented completely separately from the rest of the building. Thus all IPI and SCSI cables used today, which typically meet the CL2 requirements, may require additional protection if they are to be run in such spaces. Horst also identified the equivalent classes for Fiber Optic Cable. These are OFC (Optical Fiber Conducting) and OFN (Optical Fiber Nonconducting), which are equivalent to CL2, and the corressponding plenum-qualified classes designated OFCP and OFNP. December 1&2 Fiber Optic Working Group Minutes Page 3 The next presentation was given by Ron Soderstrom, also of IBM Rochester, on the subject of safety standards for fiber optic systems. A copy of Ron's presentation is Attachment 3. He began with a detailed comparison of the safety standards mandated by CDRH (Center for Devices and Radiological Health), ANSI and IEC. This comparison is not simple because the standards use different measurement parameters. The IEC standard, which has the force of law in Europe, is generally regarded as being the most restrictive because of its large aperture diameter (50 mm). Each of the standards describes different class levels (ANSI calls them service groups) with the class being determined by the power level and frequency. This frequency sensitivity means that care must be taken with splices and connectors to not introduce propagation modes for which the power level exceeds the definition. Ron emphasized that the safety problems with a optical fiber link only occur during maintenance and service operations. Whereas he believes that no maintenance is required, service is definitely an issue - as is the "curious customer". The safety precautions mandated vary with the class and include interlocks to shut down the source if the fiber is disconnected and red/block starburst Danger labels. Only in Class 1 are there no requirements for interlocks or labels. Most of the installed optical fiber links are of course in long distance telecom applications, and the majority of these are defined to meet the requirements of ANSI Service Group 3B. This defines that the light be able to be viewed safely even using optical instruments (Group 3A defines viewing with the naked eye only). Ron was questioned closely about the likely class of requirements that the Fiber Channel will have to meet. His view is that Class 1 will not be possible with lasers which means that some form of interlock will be required. However he noted that an interlock at both ends does still not protect against a disconnection at an intermediate point which, given the operating distances of the fiber channel, may well be outside of any controlled area. Ron recommended that anyone wishing to research the subject of fiber optic standards start with the ANSI Z136.3 document, as he believed it to be the easiest to read and a new revision is just becoming available. December 1&2 Fiber Optic Working Group Minutes Page 4 After the lunch break, Wayne Sanderson of Control Data presented a definition of the scope of the Fiber Channel project by drawing a protocol stack diagram that had originally been sketched by Dal Allan. A representation of the diagram is Attachment 4. The key point of this presentation was that the SD3 for the Fiber Channel limits the scope to the definition of a new "Physical Interface" (lowest two layers) to support the existing SCSI and IPI command sets and a future HSC upper layer that may be largely vendor unique. Clearly other protocols could, at least in theory, use the same Physical Interface but such considerations, and the resulting impacts of network addressability etc., are by definition outside the scope of the present SD3. Roger Cummings of Control Data Canada then presented a overview of possible Fiber Channel topologies. A copy of Roger's slide is Attachment 5. He identified four topolgies, namely a Simple Star (broadcast, point-to-point), a Complex Star (point-to-point supporting multiple channels with non-blocking switching), a Simple Ring and a Dual Counter-Rotating Ring configuration. Roger expressed some concern over using the Complex Star configuration given the fact that the control over pathing to peripherals is usually an intimate and complex part of an operating system which would therefore be very difficult to change. Henry Brandt of IBM noted in response that just such a configuration was used to allow peripherals to be accessed from multiple large IBM mainframes. Apparently, though, the control of the switch is performed by a separate entity containing two RISC-type processors and not within any of the accessors operating systems. Roger identified a problem with the Simple Ring configuration, namely that a single fault renders the entire channel inoperative. Clearly the Dual Ring configuration overcomes this limitation, but Don Tolmie of Los Alamos National Labs and others seriously questioned wether the cost impact of requiring two transceivers per unit for connection to one channel was justified. A major limitation of all ring configurations was also identified, namely that a long return cable is required from the last peripheral in the channel to the mainframe. A number of people pointed out that this long return path would present a major problem in upgrading systems as it would have to be replaced and rerouted as each additional peripheral is added to the channel. Given that the existing SCSI and IPI configurations are daisy-chains, and that sites have been laid out with this in mind, departing from a daisy-chain configuration might easily cause severe routing problems. December 1&2 Fiber Optic Working Group Minutes Page 5 Schelto Van Doorn of Siemens then described the manufacture of passive stars by fusing multiple fibers, and he noted that active stars are available today to interconnect Ethernet devices. Schelto came up with an ingenious method of configuring a Simple Ring as a physical daisy-chain. A representation of this configuration is Attachment 6. It involves having two pairs of connectors per unit but only one pair are connected to transceivers and the others are directly interconnected. If the cable containing the two cores is then twisted between each unit a physical daisy-chain is required. This configuration received general support as being worth very serious consideration for the Fiber Channel. The second day of the meeting opened with a presentation by Steve Siegel of AT@T Bell Labs on Multichannel Optical Data Links. A copy of Steve's presentation is Attachment 7. He began by describing a project to create a 200 MB/s, eight fiber, 1 kilometer link that has been undertaken at Bell Labs. Steve identified skew (i.e. the difference in time delay between the fibers) as a major design challenge, and stated that a skew as high as 10 ns per km can be experienced if fibers are not selected. A skew of 3 ns per km can be achieved with simple selection, however. Packaging was also identified as being of concern, both in terms of optical and electrical crosstalk between the parallel paths and in thermal management. Steve showed a photograph of a twelve element led array and stated that for a power output of -18dBm per channel the array dissipates 10 Watts. Clearly this has to be efficiently removed if high reliability of the array is to be achieved. Wayne asked if the power obtainable from each led was limited by the fact that it was packaged in an array and Schelto noted that FDDI uses a launch power of -17dBm so that it does not seem to be. Steve then moved on to consider the subject of receiver design. A dc-coupled design was required because the data was not encoded, and a transimpedance type with a sensitivity of -30dBm (after allowing for dc losses) was used. With the margin thus established he then described a system power budget which for a practical system indicated a maximum operating distance of approximately one kilometer. Much discussion resulted from the power budget. Steve had allocated a figure of 0.5 dBm per connector and in response to a question from Dal identified that this was a mean value for a mated pair of connectors measured immediately after the fiber is cut and the connector applied. However Schelto warned that a December 1&2 Fiber Optic Working Group Minutes Page 6 real world value may be considerably higher, especially if the fibers are different and each connector is made by a different manufacturer. He said that he knew of one installation where the losses incurred in a cable plant installed by multiple vendors were so large that the entire plant had to be recabled using fiber and connectors from a single source. Clearly this means that a careful and specific budget will be required in the Fiber Channel standard to ensure interoperability. Steve described some eye pattern testing that had been performed, and stated that he believed that the maximum rate that could be achieved using leds is 200 Mbits/s. He thought that this would be more than adequate, but this view was challenged by Henry Brandt, who stated that IBM has a requirement for a 200 Megabyte/s, 10 KM link to allow remote backups for large mainframe sites. Steve closed by considering methods of dynamic deskewing using tapped delay lines or another form of elastic store, and noted that this is the subject of much research. He knew of an IBM project to create a parallel link on a single fiber using wavelength division multiplexing in which a calibration pulse was sent down the fiber at system startup and used to adjust for the measured skew. AT@T is also looking at was to reduce the skew by additional controls during fiber manufacture. Steve and Phil Puglisi (also AT@T) gave some details and approximate costs for the ribbon fiber cable. This consists of standard 62.5 um multimode fiber and the costs are: $3.25/meter for unsheathed cable with 7 ns skew maximum, $7.50/meter for sheathed cable with 7 ns skew maximum, and $15.00/meter for sheathed cable with 3 ns skew maximum. They noted that these costs do not include connectors and that the connector are not presently field-installable. Don Tolmie asked why the 2 Gigabyte serial links presently used in telecom application could not be parallelled, and Phil indicated that again skew was a problem. However Wayne Sanderson noted that a parallel link in which each fiber contained its own clock and was synchronized separately would not be so effected by skew or other component variations. This approach would be expensive in terms of the amount of support silicon required, but may be viable if the cost of the entire link is considered due to the decreasing cost of VLSI. Dal noted that these tradeoffs are a key part of the definition of the Fiber Channel, and asked for a paper on the subject to be prepared for a future meeting. Don Tolmie noted that a part of this tradeoff would be the availability of VLSI, and suggested that presentation be solicited from companies such as Vitesse and Gazelle, who are known to be working on high speed VLSI for this area. Phil noted that the Multichannel Link project was still in the research stage, and agreed to continue to make available the results of the project to the working group. December 1&2 Fiber Optic Working Group Minutes Page 7 Don Tolmie then presented an overview of the High Speed Channel, and the reasons for its development at Los Alamos. A copy of Don's presentation is Attachment 8. He stated that HSC was intended to allow the visualization of numerical processes by making possible the display of high quality animated displays. The minimum acceptable was a 512x512, eight color image and it was essential that the display be smooth. It was this last feature that ruled out the use of shared resources with contention delays such as networks, and forced the development of a point-to-point link. It was also essential that the system be very interactive with zoom, pan etc. As an example of the value of this approach he quoted an example of a German scientist looking at a airflow problem who discovered both a physics problem and a numerical instabillity in five year old data on the first day that he was able to display the data visually using such as system. Don then showed a videotape of a gasjet simulation, and noted that it took 10 hours of calculation on a Cray X/MP to produce 30 seconds of the tape. Don then gave an overview of the HSC features, signals and waveforms. The meeting then returned to a task that had been begun the previous day, namely the consideration of a Fiber Channel Description document that had been generated by Dal Allan. The list of requirements generated by previous meetings was also considered. Because there was much discussion about, and major changes made to, Dal's document only the final form is included as Attachment 9. This is also the document that was presented at the X3T9.3 plenary in San Diego, CA. One of the key concepts of the Fiber Channel is Control Streaming. This is an extension to the data streaming concept of IPI and SCSI. It involves designing a protocol that allows control information to be transmitted without requiring a round trip delay for each control sequence in much the same way as multiple data words are transmitted without waiting a round trip delay for acknowledgement in data streaming. As an example of the concept only, Dal produced examples of how the present IPI and SCSI protocols could be converted to a control streaming concept. These were useful as a stimulant for discussion, but it was agreed that they could cause confusion and they were therefore deleted from the document. December 1&2 Fiber Optic Working Group Minutes Page 8 There was also much discussion on the subject of defining a parallel copper version of the Fiber Channel. The view was expressed that again this could lead to confusion, but Dal defended the concept strongly on the grounds that it was required to avoid a future version of SCSI including a new and incompatible physical interface which would thus defeat the unification goals of the Fiber Channel. The parallel copper version was therefore retained in the document, and thus it will have to be considered in the definition of the protocol. Frank Holland of Control Data asked that the Burst prefix be defined to have a length that is as a minimum a multiple of 32 bits, and as a preference a multiple of 64 bits, to simplify the design of a dma channel in systems with wide memory word widths.