Write Bits Directly Onto a Hard Drive Platter? 578
kidcharles writes "I'm working on a project that requires writing bits to a magnetic hard drive platter in a completely controlled fashion. I need to be able to control exactly where 1s and 0s will appear physically on the platter. Normally when data is written to a drive the actual bits that get written are determined by the file system being used, as modified by whatever kind of error handling the drive itself is using (e.g. Reed-Solomon). All of the modern innovations in file systems and error handling are great for reliable and efficient data storage, but they are making my particular task quite daunting. My question for Slashdot: is there a way to get down to the 'bare metal' and write these bits? Any good utilities out there to do this? Obviously a free and open source solution would be preferable, but I'm open to anything at this point."
Re:Sure (Score:2, Interesting)
Agreed. I would find is difficult to believe there is a practical use for this, but if there is an actual requirement to control zero and ones in specific tracks/sectors, you might as well build your own chip for a drive. :)
If the solution sucks that maybe they're trying to solve the wrong problem. :)
Re:Data Recovery (Score:1, Interesting)
Your best bet is going to be going through data recovery guys. If this is a company project that your company demands to have done entirely in house without contracting out any of the work prepare for things to get very expensive. If you outsource this though or use some data recovery people to get you equipped and set up for this project it might have a cost, but no where near what it would be to building this yourself.
Re:You Da Man!!! (Score:3, Interesting)
First thought was - WoW! Somebody wants to defeat disk caching, ECC and make life bloody difficult to boot. The best way, if you really want to do direct disk RMW operations - is to use Flash. Its fast, reasonably portable, and won't fail you if you use it properly.
You still won't know where the data is actually written. Flash does wear leveling.
Re:Talk to Steve Gibson author of Spinrite (Score:5, Interesting)
The stuff Spinrite does stopped working 15 years ago. On modern drives you're pretty much doomed. Considering Gibson has been known to rip off SYN cookies and spout such nonsense as "Windows RAW sockets are going to destroy the internet", I wouldn't go anywhere near him for advice.
DRM? (Score:5, Interesting)
The optimist in me wonders if he is trying to defeat such a scheme.
Re:You Da Man!!! (Score:1, Interesting)
. Once again, OP: I'd love to know what you're up to, if it's not trolling Ask Slashdot.
My first thought was steganography using bit position. My second was a defeating mechanism for NSA-level data-recovery methods which use bit position to help infer overwritten data. Those seem the only reasons anyone would ever want to worry about their bit position on a track.
Perpendicular or normal? (Score:1, Interesting)
I guess you want to use modern drive technology to write fine magnetic structures to a plate. Probably producing the equivalent of a magnetic image or a hologram. What you need to do is keep the disk's platter, motor, head, etc. and dump all the electronics. Draw a line on the disk, radial to produce an index and use an optical sensor to read it. Drive the disk at a reasonable speed and then draw your patterns one scan line at a time, using the optical as a trigger to your circuit. You can step inwards from the edge after you are sure that each line is drawn correctly, perhaps with a second head linked to an o.s.
Have you considered the difference between previous tech and perpendicular recording? I suggest perp. is better for you.
Mel, is that you? (Score:5, Interesting)
We don't need to work like that any more. [jargon.net]
Re:You're doing it wrong! (Score:2, Interesting)
You can calculate this by finding out C/H/S parameters (and the physical dimensions of the platter(s)), calculating the LBA, reading the sector where the bit should 'appear' and rewriting it with the 'one' bit changed (ATA controllers read/write 512 bytes at a time (sector size)).
Simple, LBA [wikipedia.org] .
Re:Slashdot trolled (Score:5, Interesting)
The submitter appears to know enough about what he's asking to know that it's also impossible / completely impractical. Recording ones and zeros directly isn't done for a reason, submitter appears to understand this.
Keep in mind that the question is not just why the submitter wants to write directly to the magnetic domains of a hard disk, but why the submitter asked the question of how to write directly to a hard disk.
I can think of a number of reasons for the latter:
- The submitter could be an author writing some cyberpunk techno-thriller, and wanted some technical feedback from the 'experts' on Slashdot.
- The submitter could be a CIA drone trying to write a report on how it's impossible for the terrorists to somehow 'hide' data on hard drives that even 'dd' can't read.
- The submitter could be a troll.
That's just off the top of my head.
On the other hand, I seriously can't think of a good reason why anyone would want to actually damage a hard drive that way. If all you wanted to do is very precisely create a surface with a complex pattern of specifically oriented magnetic domains, there's easier ways. For example, there are equipment suppliers that can provide high-precision two axis steppers with ready to use computer control interfaces. It's not hard to get equipment that'll go down to atomic precision [wikipedia.org].
Re:Slashdot trolled (Score:3, Interesting)
http://slashdot.org/article.pl?sid=01/02/22/0622249&mode=thread [slashdot.org]
Did any of the old black box recorder like devices write in a loop directly?
Re:Slashdot trolled (Score:4, Interesting)
The submitter doesn't appear to know enough to say why he wants to do this. If this were a consulting job, I would say, what are you really trying to do ? My guess is that, whatever that is, there is a better way to do it.
Re:I tried it once (Score:3, Interesting)
I had a drive fail last year with some personally valuable information on (during the 3 hour window of time I had repurposed its backup drive for a data transfer - sods law, should have just gone and bought another drive).
I ended up sending it to a data recovery company who told me that the first sectors were corrupt, which is where the "softest" part of the firmware was stored and consequently there was no possibility of recovering the data, even with a controller swap or extracting the platters.
A friend of mine who works for the UK police data forensics team (paedo-investigator) said that other options that exist were mind-numbingly expensive.
I guess this is what you did.
He may mean something a bit different. (Score:3, Interesting)
Some operating systems have volume management tools that do let you get a great deal more specific about where data is written. Normally, I'm a Linux or Wintel guy, but I have managed some AIX boxes a few times and from setting up a large one once I recall that it's logical volume manager allowed you to create volumes on the SCSI drive systems that were very specific about how and what part of the drive was used. You could specific, for example, that a particular volume use only the outer tracks (or inner or middle I suppose) of the drives -- in addition to a great many options for raid striping and transaction logs and so on.
The idea is that the outer edges of the platters travel faster under the read/write heads than the inner and so performed differently. Also, you could keep the head from having to jump around as much by keeping all the data that tended to be used at the same time on the same tracks -- reducing your average seek time when reading randomly from that pool of data.
In practice, I think this mattered a great deal more when us old guys were dealing with 80+ millisecond random seek times on 5 1/4" wide full height (what would now be considered two bays) drives -- or larger disk-pack based drives (aka washing machines) with the massive physical movement necessary for those read/write heads.
Today, I think the admin is better focused on distributing the data load better across different drives/arrays to even the load out and also focused on reducing overall disk i/o in creating database schema and applications. Focusing on very fine details like this is likely to have only marginal benefit next to those key areas --- but I can't presume to know for sure what this project has in mind.
My First Hard Drive (Score:2, Interesting)
I'm still very thankful that I didn't go with my alternate plan of getting the Lt. Kernel 20 MB hard drive for my Commodore 128. I imagine I wouldn't quite have the career I have today had I gone in that direction.
Re:I tried it once (Score:3, Interesting)
It's a Hitachi 2.5" 160GB manufactured in APR-07 labelled as having "Apple HDD Firmware 2006". It was from a white MacBook.
The guys that looked at it (ABC Data Recovery in Sheffield) gave me the explanation above.
My friend said that if that was the case, if the data could be extracted, it would have to be reconstituted by hand which would be expensive.
He recommended MJM Data Recovery in Somerset to do this.
Re:Sure (Score:5, Interesting)
A whole chip? Really? That sounds hard. Just Ebay an old 20MB Seagate ST-225 MFM drive, and write whatever bits you want. It doesn't know any better.
Or, the submitter could contact Seagate or another drive manufacturer and ask what it would take to get a drive with special firmware that let him write 1's and 0's directly to the drive wherever he wanted. Basically, remove the intelligence on the drive.
It's not that impossible for drive manufacturers to do things like this for you, if you have the $$ to pay for it. I don't know what the cost would be, especially since he's really only [probably] looking for a few drives for this project. If it's grant-funded research, the grant would pay for it. If it's an independent project of some kind, he's in for a surprise.
I took a behind-doors tour of a major drive manufacturer a few years ago. During our visit, we were able to visit with engineers - one of whom was head of the firmware engineering team. He told us lots of stories about the firmware requests they've fulfilled. One example was a customer who supported lots of old PBX systems. These PBX systems ran software from a hard drive, but due to the age the system only supported drives up to (around) 200MB. Nobody made drives that small anymore, so this drive manufacturer re-wrote the firmware for them ($$) so a 120GB drive (the smallest they made at the time) would only recognize & address the first 200MB.
So yeah, I'm sure a drive manufacturer like Seagate could write custom firmware for him that would meet his project needs.
Re:Slashdot trolled (Score:5, Interesting)
- Submitter wants (for whatever weird project of theirs) a surface with magnetic image on it (a'la LightScribe, only magnetically).
- Submitter missed the actual point, and wants to write raw data to drive (some embedded project) circumventing partitioning and filesystem overhead - quite frequent.
- Submitter wants to record a lot of analog/unreliable data - skipping error checking and using the extra space to store more data (errors = acceptable noise)
- Submitter doesn't care about data storage and wants to use disk heads as a very fast magnetic field sensor/generator
This is quite doable with ANCIENT pre-ATA drives. The computer controlled everything then, the head position, the moment of writing and so on.
It is doable with old ATA drives that run PIO modes, probably with a bit of electronics tinkering.
It is about impossible with modern drives. You'd have to replace the whole electronics with your own and drive the head and the motors with your own.
Re:Sure (Score:3, Interesting)
Does is have to be a hard drive? The Apple IIe was a wretched computer with awful software, but it had a floppy controller that was possibly the cutest electronic device ever made. It's been an awfully long time, but my recollection is that it was built from a handful -- six or eight -- of TTL chips and that not only COULD one control exactly what was written where in software, one HAD to control what was written where in software.
Re:Headlines are superfluous (Score:2, Interesting)
except for "If you are looking for disk I/O without a filesystem..." i agree to all your arguments
in linux though, you have the /dev/(sd[abcde...|012345...]|hd[abcde..|012345...]) block device files, which you can directly access through normal filedescriptors like a normal file.
These give you linear access to the whole harddisk, accessable through open, read, write, seek and ioctl. Windows surely has something similar, else partitioning programs like fdisk or partition magic would have a real hard life...
I guess though, that this kid really meant the physical bits, and without building his very own harddrive with its very own controller he is totally out of luck, not even the firmware is capable of that! (rle(yes i am old) and other encodings are done in hardware from what i remember, while the firmware takes over the logical structure of the data positioning etc)
The bits you send to the harddisk are not what is written to the platter, but a special encoding representing them.
You might have luck with some 1950's style harddisks, which might not havent gotten to those features yet.. but you will have to find them in a museum
No kind of harddisk (controller) on any ibm compatible ever gave you access to that part.
btw harddisk recovery from broken drives is seldom done with special controllers, but through replacing the broken controller board on the harddisks themselves through a similar model
now back to the part about why he is wanting to do this.... while it is possible this would be used for drm, this would never work realiable without taking care of what the used filesystem wants to do... you need to allocate the used blocks as a file, else they will be overwritten at random times.....
Even the good old protection style to make use of unused bytes in filesystem blocks at the end of files wont work on most filesystems today anymore, as they can put more than one (small) file into single blocks nowadays to save space
Re:Tell us your project? (Score:2, Interesting)
And I want a pony. Point is, nobody at all can understand a good reason for doing it, so that tends to make people think he's jumped prematurely to a diagnosis - fussing about "how" before he's really sure "what".
He's stating a half-assed solution, rather than expressing what the problem is - for which the best solution might be entirely different.
http://catb.org/~esr/faqs/smart-questions.html#goal [catb.org]
Re:Wrong. (Score:4, Interesting)
It would be nice to know why the poster thinks this feature is important. The only things I can think of is the following...
He is using parts from a hard drive to make a robot. So he wants to control a Robot using the hard logic board, to control the mechanisms.
He is using the high speed nature of the drive and the noses it makes to make a speaker out of it by moving writing bits in the right spot.
Trying to make a full real-time application.
Researching better algorithms for drive storage.
Attempting to make some sort of digital watermark on the drive.
The first two are just silly hacks which would get geek creds but not overly useful.
The Real Time System You probably need more then off the shelf and free software to do the trick
And the Research you could probably program a simulator to simulate the results.
The Watermark idea will work until someone fills up the drive.
So I would agree with the poster unless you come up with a good reason where there isn't an other approach I don't see the need to do this.
Re:Headlines are superfluous (Score:5, Interesting)
Since the intricacies of the drive's physical address space are not accessible from the outside, there will never never never be a reason to try and fiddle with it directly. Because you can't.
Careful there. I know you're addressing a typical user of the hard drives, but there ARE ways of -- and reasons for -- getting at the "bits themselves". One of my colleagues, as a fresh student at one of the National Labs in the US, was tasked with recovering data from a crashed (literally crashed, so that the controller was destroyed and platters bent, with lots of data loss) disk. Of course there are ways and means of doing so, usually involving fantastically specialized equipment and lots of expense (and, often, the complicity or orders at the Federal level).
He ended up building a sweet little system which could scan the surface (in a low-level clean-room, of course), albeit slowly, and interact directly with the bits.
I'll have to ask him how they figured out the filesystem, the error-correction, and so on, without any of the usual partition tables and so forth.
Not a Stupid Question - And a suggestion (Score:5, Interesting)
I'm surprised by the number of people who seem hostile to this question, mostly because they can't imagine any circumstances where they would want to do this. I can think of several. In fact, I was thinking about something similar with optical disks just earlier today. (I'm curious whether small holograms could be created by writing an interference pattern directly onto a CD or DVD, but I would need exact control over how the tracks lined up to achieve this.)
Here's some concepts just off the top of my head:
* He's come up with a new disk encoding scheme, and wants to try it out.
* He's doing research into how long disks retain data, and is questioning basic assumptions like whether surface bits permanently magnetize the platter underneath, affecting later bits in the same location. Or how far the domains spread.
* He's working with self-assembling molecules and needs to give them a patterned magnetic surface to build on.
* He wants to 'print' a 2-D picture onto a small portion of the hard drive, and bounce a laser off it. (The magnetic alignment of the surface domains would polarize the photons slightly.)
* He's making a high-resolution rotational position encoder, so by reading across 32 tracks knows the rotation of the platter down to a few nanoarcminutes.
* He wants to totally destroy the contents of a disk. (I assume this is what most of the hostile people think is the intention, presumably as the payload of some virus)
Those are just the ones off the top of my head.
However, the ranting people do have a point that without knowing WHY you want to do this, we can't really suggest the best solutions. A lot of people have recommended going back to super-old MFM hard drives that allowed this, but we don't know if you require the density of modern hard drives.
To do this with a modern drive, you're basically going to have to rebuild the controller. Either totally remove the controller board (leaving handy raw connections to the stepper motors and drive heads) or cut the connections between the microcontroller and the low-level electrical functions of the drive, and substitute your own. Here's where knowing your accuracy requirements could have help, because if you want relatively large bits, you can get away with fairly low-frequency components. A 20Mhz microcontroller can, with say an external high-speed shift register, push out an 80mbit/s serial stream, which equates to >120,000 bits around the track of a 6,000RPM drive. Not quite the same density as the manufacturer, but better than the old MFM drives.
Your next problem is going to be this: It's really, really hard to tell where the head is on a platter. I've no idea how modern drives do it, but it used to be done with 'marker' bits either in the track, or on a nearby 'index' track, plus a little timing. Preserving these location marker bits is actually the most important job of the drive controller.
It's not impossible, merely very difficult. I could probably make one with only several weeks of hard work.
Alternately, you could try getting into the firmware and re-writing it to your specifications, but that might take longer. You would have to reverse-engineer a lot of stuff that is specifically hardened against this, but at least the hardware would be stable.
Re:Slashdot trolled (Score:3, Interesting)
I was stumped as to why one would want direct access to the drive like that, until I read your post. Thank you.
But I also agree that this is just about impossible, depending on what he's trying to do. There's no way he can control the heads of the disk: too many layers of abstraction exist between the user and the hardware. But it should be possible to control where data is placed on the disk: this is what disk defragmentation does, after all.
But even then I'm not sure that what the device reports is necessarily what happens on the platters. Flash memory drives, for example, relocate information to balance the number of writes and extend the life of the drive.
I would recommend contacting the drive manufacturer and then making my own block device driver for Linux based on their recommendations. But even then I'd question the core reasoning: is this really, really necessary?
Re:Slashdot trolled (Score:4, Interesting)
What you say is all well and good, but you don't think that knowledge of that purpose might help with the solutions?
It's nothing like that at all. It's more like that-won't-work-so-what-are-you-trying-to-achieve-maybe-something-else-will-work, which is fair enough.
A lot are asking "Why do you want to do that?", which sure looks like curiosity to me.
Re:Slashdot trolled (Score:1, Interesting)
I guess this is a good example of the crowd around and how they see/face problems. Great stuff for a next slashdot poll.