Whence IDENTIFY DRIVE?

As most everyone knows, the AT Attachment standard (informally known as IDE) started by literally bolting the previously standalone AT disk controller onto a MFM drive with a ST506 interface and connecting the assembly to the host system with a 40-pin ribbon cable.

As it often happens, the standardization process was far behind the technology and the first ATA standard officially became ANSI standard X3.221-1994 in (obviously) 1994. The standard had been in the works for several years by that time, but even so the first ATA standard drafts appeared in mid-1990, when at least half a dozen vendors have been shipping ATA drives for some time.

From a software perspective, an IDE drive is almost indistinguishable from an ST506 drive plus an AT controller—in fact that was the whole point of IDE. One crucial difference is the IDENTIFY DRIVE command.

The IDENTIFY DRIVE command (soon renamed to IDENTIFY DEVICE, in order to reflect the broadened scope of ATA) makes it possible to build self-configuring systems, also known as Plug and Play, even though apparently no one thought of calling it ATA PnP until about 1995. But where did it come from and why?

Hardware

We know that IDENTIFY DRIVE was specified in the early ATA drafts in summer 1990. But we also know that at that point, there were numerous IDE drives implementing the command, for example Seagate ST-157A from early 1989, the Quantum ProDrive 40 from mid-1988, the Miniscribe 8051A of similar vintage, or the even older Conner CP-342 (in all cases confirmed by examining actual drives).

Western Digital WD1003-IWH (1986), non-integrated drive electronics

The very fact that the early ATA drafts already included the IDENTIFY DRIVE command (with command code ECh) indicates that it was widespread; at the same time, those drafts (and the final ATA-1 standard) mark IDENTIFY DRIVE as optional, which suggests there were IDE drives with no IDENTIFY DRIVE support.

Document Drought

Researching the history of the IDENTIFY DRIVE command prior to 1990s turned out to be remarkably difficult. Wikipedia claims the first ATA draft was “Common Access Method AT Bus Attachment, Rev 1, April 1, 1989”, document CAM/89-002, created by the CAM Committee, which is plausible but there’s no sourcing.

For whatever reason, the ATA standardization effort started in the CAM Committee, which was part of the X3T9.2 working group. The primary focus of X3T9.2 was SCSI, although it also standardized ESDI. CAM (or Common Access Method) was an ultimately unsuccessful attempt to create a standardized software interface to SCSI hardware, and its draft documents do not appear to have survived. In 1990, the ATA Working Group was folded back directly into X3T9.2, and several ATA drafts from that period did survive.

The next source of information would be technical references or OEM manuals for pre-1990 IDE drives. But… there’s nothing. By the end of 1989, Conner, Seagate, Quantum, IBM, Micropolis, CDC, Western Digital, and several others had IDE drives on the market. But the manuals are not available online anywhere, even though there is plenty of archived documentation for SCSI and ESDI drives from the same period.

Part of the problem is no doubt that for the vast majority of users, including technical users, those manuals weren’t all that interesting. For a basic IDE drive, there was very little they wouldn’t already know from the PC/AT Technical Reference—again, the compatibility was the point. The one exception would have been the IDENTIFY DRIVE command.

Software

Drawing a blank on the hardware side, I thought I’d perhaps have more a bit more luck tracking down which software used the command. What I found surprised me a little: Even though the IDENTIFY DRIVE command had been around since well before 1990, major operating systems do not appear to have used it until 1992-1993:

• The WDCTRL.386 disk driver in Window 3.1 (1992) did not use IDENTIFY DRIVE
• Windows NT 3.1 used IDENTIFY DRIVE in the atdisk.sys driver in the final July 1993 release, but not in the March 1993 beta
• Novell introduced the IDE.DSK driver for NetWare in 1993
• OS/2 2.0 did use IDENTIFY DRIVE in its poorly named IBM1S506.ADD driver in 1992
• Microsoft OS/2 1.3 used IDENTIFY DRIVE in ESDI-506.BID on installation diskette A but not in the AT disk driver on diskette B (1991)
• Microsoft OS/2 1.21 used IDENTIFY DRIVE in BASEDD01.SYS (1990)

OS/2 clearly used IDENTIFY DRIVE relatively early on, but the details were complicated. While Microsoft’s OS/2 1.2 and 1.3 used IDENTIFY DRIVE, IBM’s did not. What’s more, the betas of OS/2 2.0 did not use IDENTIFY DRIVE until 1992, most likely because the disk driver was forked from the OS/2 1.x code base before the support was added.

IBM used the DISK01.SYS driver in OS/2 2.0 pre-releases throughout 1991, but the final beta in early 1992 switched to IBM’s brand new modular storage stack with ADD and DMD drivers.

Reviewing the 1992 source code of the IBM OS/2 2.0 storage stack (which fortunately did survive) it’s clear that the information about IDENTIFY DRIVE command must have come from the IBM drive division.

The OS/2 2.0 driver uses IDENTIFY DRIVE to query the default disk geometry, which might seem like a good idea but probably isn’t, because it may not match the geometry the BIOS setup programmed into the drive. The IBM1S506.ADD driver also uses IDENTIFY DRIVE to query and (if present) enable multi-sector transfer support, a feature which may have brought a noticeable performance boost when available.

I don’t have the source code for the MS OS/2 1.21 BASEDD01.SYS driver, but at this point June 1990 is the earliest known usage of IDENTIFY DRIVE by an operating system. Similar to IBM’s OS/2 2.0, the Microsoft driver uses IDENTIFY DRIVE to detect whether READ/WRITE MULTIPLE commands are supported, and if so, enables multi-sector transfers.

Okay then, operating systems probably didn’t really use IDENTIFY DRIVE before 1990. But surely someone must have used it, right? Why else would the command be there in the first place? Drives definitely supported the IDENTIFY command well before 1990.

A BIOS could have, in fact should have used it… but I couldn’t find any evidence of common BIOSes using IDENTIFY DRIVE before 1993 or 1992.

Then again, the most likely user of IDENTIFY DRIVE would have been Compaq, since Compaq (together with Western Digital) was the driving force behind IDE and Compaq machines used IDE drives before anyone else.

Yet checking a 1988 DeskPro 386 BIOS revealed no trace of IDENTIFY DRIVE usage. Still, if anyone used IDENTIFY DRIVE in the 1980s, it should have been Compaq.

Deskpro 386 Tech Ref to the Rescue

Jeff Parsons pointed me to old Technical References for the Compaq DeskPro machines. That turned out to be extremely helpful.

Volume 2 of the DeskPro 386 Technical Reference from 1986 (the original DeskPro, the first 386 PC) shows that the 40 MB drive option did in fact use a 40-wire ribbon cable nearly identical to “modern” parallel ATA. The 40 MB drive was a half-height 5.25″ CDC Wren with a custom Western Digital PCB; the PCB included the equivalent of the PC/AT hard disk controller (conveniently also a WD product) and was connected to a multi-I/O board inside the DeskPro 386 with the above mentioned 40-wire ribbon cable.

As an aside: The Wren HH with a WD adapter board is often claimed to be the first IDE drive, but that’s only half of the story—Compaq used a 3.5″ Miniscribe drive with a separate WD adapter board and 40-wire ribbon cable in the Portable II a few months before the DeskPro 386 came out. There are even photos (scroll down a bit). The WD adapter board was model WD1003-IWH, pictured near the beginning of this article. It is likely that the Compaq-only Wren was the first actually integrated IDE drive, but the DeskPro 386 where the drive appeared was not Compaq’s first machine using IDE cabling.

Okay, so the DeskPro 386 definitely had an IDE drive of some sort of another. Even better, the DeskPro 386 Tech Ref also documents a rudimentary “Identify” command with code ECh. That means the command already existed in 1986! Here’s what it looked like:

Needless to say, this is only a small subset of the IDENTIFY DRIVE command specified by the original ATA standard, but it is entirely compatible with it.

But wait! A note clarifies that the Identify command (plus two others) is only available on the 130 MB drive, not on the 40 MB IDE drive! What was the 130 MB drive then? It was an ESDI drive, and the controller was almost certainly a Western Digital WD1005-WAH. So the IDENTIFY DRIVE command was implemented by an ESDI controller before it showed up in IDE drives! That must be why the DeskPro Tech Ref labels the contents of IDENTIFY command results as “ESDI Parameter Words”.

Back to Hardware

What’s ESDI, Anyway?

Unfortunately I could not find detailed documentation of the WD1005 or WD1007 ESDI controllers. There are detailed datasheets available for the WD1002 and WD1003 series of controllers, but not for the ESDI ones.

The only solid information (thanks to Compaq) is that the WD1005-WAH implemented the Identify command, but what does it have to do with ESDI drives? And how did the WD1005-WAH work?

From extant documentation it is apparent that the Western Digital WD1005-WAH was a 16-bit ISA adapter similar to the PC/AT WD1003-WAH, but instead of a ST506 style drive it could control an ESDI drive (or two). At the same time, it provided a register and command interface fully compatible with the WD1003-WAH, and thus did not require its own BIOS.

In other words, an ESDI drive connected to a host through the WD1005-WAH controller looked to the host system just like a standard PC/AT drive. There were possible issues with geometry translations because the higher-end ESDI drivers were already running into the BIOS limits, but aside from that, a WD1005-WAH with an ESDI drive was, as far as software was concerned, exactly the same as a PC/AT fixed disk controller with a ST506 style drive.

Except, again, for that IDENTIFY DRIVE command. (And possibly the READ/WRITE BUFFER commands.)

But the ESDI connection provided the vital clue that let me piece together the answer to the question where the IDENTIFY DRIVE command came from, or at least form a very informed guess.

ESDI Drive Interface

ESDI drives were an outgrowth of ST506 drives commonly used for higher-end hard disks. ESDI was faster—while ST506 was limited to 5 Mbps, ESDI could handle 10 Mbps and over time, more than 20 Mbps. ESDI drives had a data separator on the drive, and not on the controller; while ST506 controllers essentially moved an analog signal over the data lines, ESDI uses NRZ encoding synchronized with a clock signal produced by the drive. ESDI drives could also handle larger capacity, upwards of 500 MB and theoretically into many gigabytes.

ESDI drives also accept a small set of commands that do things like select a head, seek, or return drive configuration.

The people designing ESDI were presumably aware that the complete lack of ST506 drive intelligence had one nasty consequence: Requiring users to tell the controller what the drive geometry is. The controller had no way to tell but had to know, and the only means available was the least reliable one: Humans. ESDI drives know what geometry they have and a controller can ask them.

The ESDI command interface is very simple: Write a 16-bit command word to the drive, get a 16-bit result back. Here’s what ESDI drives could tell the controller about themselves, from an ESDI draft specification published by CDC in 1984:

Surely it is no coincidence that ESDI drives return 16-bit words in response to the REQUEST CONFIGURATION commands, and that the first nine words line up exactly with the Compaq Tech Reference. Ever wondered why the third word of IDENTIFY DRIVE is reserved in ATA? Because it reported the number of cylinders for removable-media drives, something ATA did not support.

Here’s what the first word (general configuration) looked in the 1984 ESDI spec:

ESDI general configuration bits (1984)

For comparison, here’s the same table in the final ATA draft from late 1993:

General configuration bit-significant information:
15 0 reserved for non-magnetic drives
14 1=format speed tolerance gap required
13 1=track offset option available
12 1=data strobe offset option available
11 1=rotational speed tolerance is > 0,5%
10 1=disk transfer rate > 10 Mbs
 9 1=disk transfer rate > 5Mbs but <= 10Mbs
 8 1=disk transfer rate <= 5Mbs
 7 1=removable cartridge drive
 6 1=fixed drive
 5 1=spindle motor control option implemented
 4 1=head switch time > 15 usec
 3 1=not MFM encoded
 2 1=soft sectored
 1 1=hard sectored
 0 0=reserved

It’s very obviously the same table… and again, the ESDI origins explain why half the bits don’t even make any real sense for IDE. Who cares if an IDE drive is hard- or soft-sectored, or if the internal transfer rate is 5, 10, or 15 Mbps? Certainly not the software using it. But an ESDI controller would very much care about such parameters when driving an ESDI disk.

In fact does ECh, the command code of IDENTIFY DRIVE, stand for ‘ESDI Configuration’? It very well might.

As an aside, I was not able to confirm if my Western Digital WD1007V-SE2 controller supports the IDENTIFY DRIVE command because I have no working ESDI drive (help, anyone?). I am fairly sure that the controller’s BIOS does not use IDENTIFY DRIVE, but it uses a different command, E0h. That command clashes with STANDBY IMMEDIATE specified in the ATA standard, but the WD1007V clearly uses it to send a command word to the drive and read back the 16-bit reply. The WD1007V BIOS uses the E0h command to query the same information that IDENTIFY DRIVE would return, but piecemeal. One possible reason for that could be the difficulty of safely allocating a buffer for the IDENTIFY DRIVE results in the controller POST routine.

Deskpro 286 Confusion

We can also examine the Compaq DeskPro 286 Technical Reference from December 1987, roughly a year newer than the original DeskPro 386 Tech Ref. It’s similar, but different, and it’s different in odd ways. Here’s what it says about the IDENTIFY command:

IDENTIFY (ECh) command from DeskPro 286 Tech Ref (December 1987)

Note that while the older 1986 Tech Ref defined words 7 and 8, those are gone from the late 1987 Tech Ref. And the missing two also happen to be words which are somewhat specific to ESDI and are not defined in ATA (left as vendor-specific). Most IDE drives do not report any values there, although some—notably old Conner drives—do. Also note that the newer Tech Ref removed references to ESDI, presumably because the command was no longer specific to ESDI drives.

Which is why it gets rather confusing when the Tech Ref states that “The 20-MB and 40-MB integrated drives do not support the IDENTIFY, READ SECTOR BUFFER, or WRITE SECTOR BUFFER commands.” The Dec ’87 DeskPro 286 Tech Ref covers IDE (aka “COMPAQ 16”), ST506, and ESDI drives. If the statement were true, it would imply that both ESDI and ST506 drives/controllers support the IDENTIFY command, and IDE drives do not, which is extremely implausible. It is much more likely that the Tech Ref is wrong and it is the ST506 drives which do not support those commands, whereas ESDI and IDE drives do.

Whodunnit?

So… the IDENTIFY DRIVE command (ECh) existed in 1986 and must have been implemented by the Western Digital WD1005 ESDI controller. It was, at least initially, obviously intended as a way to conveniently return all of an ESDI drive’s configuration information to software.

The question is, what software? As we’ve seen above, OS/2 started using IDENTIFY DRIVE in 1990, but many other operating systems waited another couple of years. It’s hard to say if OS/2 was first, but it may well have been, at least when considering generic drivers.

At the same time, the command had already existed for at least four years, and by 1990 it was supported by numerous IDE drives and at least some ESDI controllers.

On the ESDI side, WD was not the only vendor implementing IDENTIFY DRIVE in their controllers. Documentation is scarce, but at least the OMTI 8640 documentation from 1989 describes a “Read Parameters” (ECh) command, as well as an “Initiate ESDI” (E0h) command that looks extremely similar to what the BIOS of my WD1007V is internally using.

On the IDE side, it’s difficult to say who supported what without examining the actual drives, but the IDENTIFY DRIVE command was certainly out there. I have been able to confirm that a Conner CP-341i from 1988 supports IDENTIFY DRIVE and not only that, it even supports the READ/WRITE MULTIPLE commands.

I have also examined a somewhat odd Conner CP-342 drive and found that it also supports IDENTIFY DRIVE, though not READ/WRITE MULTIPLE; the CP-342 was likely the first IDE drive available to OEMs other than Compaq. Why is my CP-342 odd? The drive was made in 1989, but the PCB is full of components made in 1987. Yet the sticker on the ROM clearly says “CP 1989” and the revision on the sticker (5.85) matches what IDENTIFY DRIVE reports. The upshot is that while Conner’s 1989 firmware provably supported IDENTIFY DRIVE, that is not exactly new information and I can’t conclusively say that the 1987 firmware did too, although it is extremely likely. Curiously, the CP-342 calls itself CP-341 in the IDENTIFY DRIVE output (didn’t expect that!), the CP-341 being an earlier Compaq-only model.

After most of this article was written, Tom Gardner kindly provided another valuable piece of information. The Conner CP3022 Product Specification from February 1988 does document the IDENTIFY DRIVE command; it is the fourth revision of the document and there is no reason to suspect that the IDENTIFY DRIVE command was new at that point; Conner almost certainly implemented it in 1987, possibly earlier. The Conner CP3022 spec is notable for calling the command IDENTIFY DRIVE, rather than just IDENTIFY as it was named in the Compaq technical references.

Here is what the Conner documentation shows for IDENTIFY DRIVE:

IDENTIFY DRIVE command from Conner CP3022 specification (February 1988)

It is notable that Conner documents words 7 and 8 which Compaq un-documented. It is likewise notable that Conner’s IDENTIFY DRIVE definition is significantly extended compared to the Compaq Tech Refs, and it is in fact a fairly large subset of what later appeared in the first ATA standard (it is actually almost the same as 1990 ATA standard drafts).

At this point it is impossible to tell whether Conner extended the IDENTIFY DRIVE information on its own initiative or if someone else (Compaq?) asked for it. At any rate, Conner supplied the model number, firmware revision, and serial number. There is already a provision to support the READ/WRITE MULTIPLE commands—not all Conner drives did that, but Conner IDE drives probably implemented this capability before anyone else. There is also the mysterious “double word transfer flag”, as well as information about the drive’s buffer.

And there is information about the number of ECC bytes passed on READ/WRITE LONG commands. That may well have been important to Compaq’s diagnostics: ST506 and early IDE drives used 4 bytes, while ESDI drives used 7 bytes of ECC.

It is apparent that the IDENTIFY DRIVE command was somewhat widespread before the first ATA draft showed up, even though it does not appear to have been widely (if at all) used by operating systems.

A retired WD firmware engineer confirmed my suspicion that the driving force behind this command were PC OEMs, notably Compaq, but likely also Dell and others. OEMs had a vested interest in drives returning identifying information in a compatible manner, and had enough clout to force drive manufacturers to make it so.

Still, that leaves the question what those OEMs actually used the IDENTIFY DRIVE command for, and it is almost a given that they must have—otherwise why ask for it?

Who Needs It?

The most obvious user of IDENTIFY DRIVE would be Compaq’s setup and/or diagnostic utilities. I have not been able to find any use of IDENTIFY DRIVE in Compaq’s utilities from 1987 or earlier, even though Compaq documented IDENTIFY DRIVE as far back as 1986. Compaq diagnostics version 5.02 from May 1987 do not appear to use IDENTIFY DRIVE.

Finally I struck gold while examining the Compaq Portable II diagnostic floppy from this page. This is Compaq diagnostics 5.08, with files dated January 29, 1988. They don’t look like much:

But despite their humdrum appearance, both the Compaq diagnostics (TEST.COM) and the setup utility (SETUP.EXE) issue an IDENTIFY DRIVE command when starting up.

I’m not sure what exactly the utilities do with it and the executables are obfuscated in a silly way that makes analysis harder, but IDENTIFY DRIVE is there.

This narrows the software timeline considerably: Compaq presumably added IDENTIFY DRIVE to their diagnostic floppies sometime between May 1987 and January 1988. Probably not coincidentally, this was the time period when Compaq started moving away from ST506 drives to IDE drives in their then-current systems.

Remaining Questions

We’ve clearly established that the IDENTIFY DRIVE command (however it was originally name) was first implemented in Western Digital ESDI controllers (not drives), and was documented by Compaq as early as 1986. Not much later, IDENTIFY DRIVE showed up in IDE drives, and was documented by Conner no later than February 1988, almost certainly earlier. The command was used in Microsoft OS/2 in 1990 and several years later in numerous other PC operating systems.

Some of the remaining questions are:

• Did all of Conner’s IDE drives support IDENTIFY DRIVE? If not, which was the earliest Conner drive with IDENTIFY DRIVE support?
• When exactly did Compaq’s utilities start using IDENTIFY DRIVE?
• Did any operating system use IDENTIFY DRIVE before MS OS/2 1.21 did?
• When did BIOSes start using IDENTIFY DRIVE to implement automatic drive detection and which one was first?

If any reader has an IDE drive manufactured in 1988 or earlier, it would be great to get a dump of the IDENTIFY DRIVE data. Seagate’s FIND-ATA is one option, but there are others.

Another source of information would be old IDE drive documentation; there must be more out there than the Conner CP3022 product spec. Finding more Compaq Technical References would be great too, and possibly references from other vendors if they mention IDENTIFY DRIVE.