"G=C800:5"

Only a few of you will understand that title, but that's OK

In the dark days of the early 1980's hard drives were mysterious, largely analogue devices  If you look at image above the connections are very different to recent technologies, and the cables carried a mixture of analogue data and digital control signals.

It's not super clear in the image, but the white label towards the back of the drive was a list of "bad sectors".  Literally, these were manufacturing defects on the surface of the drive where data could not be read or written reliably.

Much earlier in my career, I had to guide customers through the process of installing very expensive ten and twenty megabyte hard disks into PC-XT and PC-AT style machines.  The XT style machines didn't actually understand hard disks, and relied on a BIOS extension on the controller card to do the work.  BIOS extensions were one of the most brilliant aspects of the IBM PC architecture in my opinion, but that's a story for another day.

Anyway, back to the disk controller.  We first had to set jumper positions or switches to define the physical characteristics of the disk drive because the drive essentially had no on-board intelligence that you could query. On the DTC controllers, I told so many customers "Switch 2 and 4 on, everything else off" that it's burned into my mind after forty years!

After that, we used to boot DOS from a floppy and then run the "debug" program to launch a routine contained in the controller BIOS.  Early Western Digital and DTC controllers had this at address "C800:5", and the firmware-based formatter would start.  We called this a "low-level" format, and it's purpose was to prepare the drive for whatever operating system you were going to install later.

But why was this even necessary?

In the good old days, hard disk drives were not the intelligent devices they are today.  They were not much more than precision mechanical components, a lot of analogue circuitry to read and write from the spinning metal platters (coated with iron oxide) and most likely a stepper motor or linear actuator to position the heads.

They also had the aforementioned bad blocks.  The operating systems of the day just blindly assumed the media was perfect and the controller had to protect it.  I wanted to understand, simply out of curiosity how this really worked, so I reverse-engineered the firmware from some controllers stored in the shed.

Please note that I am not going to show any of the code I extracted out of respect for the manufacturer's copyrights, but what I found out revealed a complete hidden world that most of us need never know existed.  Read on if you dare:

The mystery deepens

The disk arrives from the manufacturer and it is quite literally blank.  Neither it, nor the controller had any understanding of what the operating system was going to need, and there was no memory on-board to store the defect information.  What I wanted to understand was how and more importantly where was this data recorded?

When is 512 bytes not 512 bytes?

The majority of disk controllers presented a block of 512 bytes in response to a command to read a sector at a specific disk/cylinder/head/sector location; but there is actually considerably more data written to the disk.  It is almost a miniature file system below the filesystem we see.

The little documentation which exists on this stuff (for the PC world, anyway) refers to three types of blocks on the disk:

• An inter-sector gap
• A sector header
• A sector data block

The gaps are pretty much void space between sectors.  These are required on rotating media because it takes time to position the actuator at the cylinder and select the correct head.  Then the electronics need to wait for an index signal, and then start counting sectors as the pass under the head.

Two things make it obvious to the electronics that a sector is beginning.  The first part is the "Preamble", 12 bytes of zeroes (0x00) followed by the "Address Mark", which was three missing clock pulses (a deliberately illegal condition) followed by a byte of ones (0xff)

The next three blocks of data make up the "header" we mentioned before.  Four bytes of data form the Header ID (two bytes of cylinder address, one byte for the head and one byte for the sector).  All this was followed by a two-byte CRC.  This combination allowed the electronics to locate and confirm the desired sector,  because that calculation time the designers built in another gap (22 bytes) so that the controller would be able to read the sector data when it arrived.

After another sync block (16 bytes) we get the actual data, followed by a CRC value for the data.  It was this CRC value which allowed the controller to flag a bad read to the operating system.

Including a variable inter-sector gap (40-84 bytes, depending on the device) this means your 512 byte sector was actually 650-660 bytes.

But what about those bad sectors?

Ah yes, here is where the story gets even more interesting.  Remember the ID Header?  Well no disk at the time (or pretty much ever) had 256 heads.  The hardware specification allowed for only sixteen heads, and that was stored in the bottom half of the "head" byte.  The top four bits had very interesting uses.

In the controllers I reverse-engineered, bit 7 (MSB) was the "Bad Block" bit.  Normally this would be set to zero, but a "one" value meant this block was bad and can't be used.  The next two bits stored the sector size."00" meant 256 bytes, and "01" meant 512 bytes.  "10" and "11" don't seem to be defined.  Finally, there is a single bit for a "disk id".  It always seems to be zero, but there may well be cases I haven't seen yet.

What happens when DOS or OS/2 or Xenix tried to use a marked sector?

In short, they won't try.  Because that sector is inaccessible, the controller returned error 0x0A (sector marked bad) and the OS would just mark the sector (or cluster containing that sector) as bad in the filesystem and it would never get used.

And the result of all this is?

If you made it this far, welcome to the rabbit hole.  But seriously, now you know why old machines could not process more than 16 heads and 256 sectors/track.  The operating systems could have supported up to 64K cylinders, but were limited to 1024 for reasons I haven't discovered yet.

These days we don't worry about any of this.  Microprocessors on the drives handle all this magic for us, and we now address storage by blocks rather than C/H/S.  That way it doesn't matter what the underlying technology is, it all works the same.

But it is fascinating to look behind the veil and get an understanding of what the electronics engineers had to do just to make these amazing devices even work at all.

Till next time!