Write Bits Directly Onto a Hard Drive Platter? 578
kidcharles writes "I'm working on a project that requires writing bits to a magnetic hard drive platter in a completely controlled fashion. I need to be able to control exactly where 1s and 0s will appear physically on the platter. Normally when data is written to a drive the actual bits that get written are determined by the file system being used, as modified by whatever kind of error handling the drive itself is using (e.g. Reed-Solomon). All of the modern innovations in file systems and error handling are great for reliable and efficient data storage, but they are making my particular task quite daunting. My question for Slashdot: is there a way to get down to the 'bare metal' and write these bits? Any good utilities out there to do this? Obviously a free and open source solution would be preferable, but I'm open to anything at this point."
Hard disk firmware hacking (Score:1, Insightful)
I'm fairly sure you would have to go fairly deep and directly hack/modify hard disk firmware to do what you describe. Such hack would be non-trivial and most likely model-specific.
That is gonna be hard (Score:5, Insightful)
You would need to replace the firmware inside the drive or use an undocumented manufacturer mode. Whatever they use to write the servo tracks would be interesting to you. You will be in the situation of the firmware writer: There will be problems all the way. Be prepared to find a way to position the heads (ever tried to find a servo track?). Most likely you also need to at least parametrize the amplifiers in the DSP part of the firmware that does the analog-to-something-to-digital so you can have direct influence on the "bits". Good luck
Your own Disk controller chipset? (Score:3, Insightful)
If you don't need lots of bytes, you should use an old disk. I mean very old, maybe going back to those 300-400MB disks we had on 486 computers or even older.
Recent disks have embedded error-handling mechanisms which will prevent you from writing what you want, where you want (physically speaking of course). The disk controller translates your "physical" disk addresses into its own internal mappings. The only way to bypass this would be to remove the disk controller's chip and put your own in place.
Maybe your question would be: "how hard is it to create my own disk controller's chipset?"
Slashdot trolled (Score:5, Insightful)
Wrong. (Score:5, Insightful)
Yes, I don't what they are. Still wrong.
Hm ... (Score:5, Insightful)
This kind of copy protection has been outdated for quite some while and should have died with floppy disks ...
Tell us your project? (Score:5, Insightful)
These kinds of questions are stupid: "I need to do XYZ for a project, how do I do XYZ?", where XYZ is one or more of complicated, ridiculous, vague, nonsensical, etc. Try telling us what your project is, and then we might be able to suggest a useful solution, possibly not involving XYZ at all, or involving a very particular/practical version of XYZ.
If you have to ask, it's hopeless (Score:5, Insightful)
"I'm working on a project to build a nuclear powerplant. Is there documentation on how to do it? Obviously a free and open source solution would be preferable, but I'm open to anything at this point."
It seems to me that if you're involved in such a project and have to ask how to do it, it's doomed.
Also, the whole idea of it seems rather impossible. Why would you want to do this in the first place? Have in mind that hard disks don't write bits as is, not just because of ECC. They use an encoding to ensure that there are no long strings of 1s or 0s. If you just wrote some data without regards to that you'd run into a sequence you'd be unable to read later, due to not having a clock signal to figure out where a bit ends and the next begins.
Modern hard disks require using special encodings and servo data in order to be readable at those densities. It just makes little sense to me to want to bypass it. Unless you're working for a disk manufacturer this just seems very odd, but in that case you'd have access to the required equipment and information.
Re:DD (Score:5, Insightful)
I'm actually more interested in why the fuck he wants to do this, much less how it would be achieved.
Re:You Da Man!!! (Score:3, Insightful)
Actually that's the best suggestion yet. After he's spent a week banging bits into the 82072A, he'll either have gone insane, or will have realised he's well and truly barking up the wrong tree. Once again, OP: I'd love to know what you're up to, if it's not trolling Ask Slashdot.
You're doing it wrong! (Score:2, Insightful)
No you don't. Rethink what it is that you're actually trying to achieve and try again.
Hmmm. The twisted, it is strong in this one. (Score:5, Insightful)
Short answer: no, it's not possible to do that. It's not impossible, simply incredibly hard to do. Unless you happen to be very handy with a soldering iron and go dumpster-diving in the backyard of an HDD company (as many others have pointed out).
Longer answer: how about creating a 'virtual' hard drive? There are utilities out there that probably let you create (a) a virtual interface (let's say IDE) and (b) a virtual hard drive, attached to said interface. The next step, of course, is to hack these (VirtualBox?) in order to make them do whatever it is you want them to do. This being said, I strongly suspect it involves non-trivial virtual machine hacking and that it probably does not respond to whatever your needs are.
Other than this (very twisted) idea, sorry, bare metal writing has been disabled a long time ago, and for good reasons, too.
Re:Sure (Score:5, Insightful)
A whole chip? Really? That sounds hard.
Just Ebay an old 20MB Seagate ST-225 MFM drive, and write whatever bits you want.
It doesn't know any better.
Re:DD (Score:1, Insightful)
I guess the dd utility is not enough for your needs, otherwise you would have already tried it.
Right?
I was just thinking that the submitter might be an idiot. Then I read your post. You're definitely an idiot.
Re:Sure (Score:5, Insightful)
The custom controller would REPLACE the the on-drive controller, not sit on top of it.
=Smidge=
Re:Tell us your project? (Score:5, Insightful)
These kinds of questions are stupid: "I need to do XYZ for a project, how do I do XYZ?", where XYZ is one or more of complicated, ridiculous, vague, nonsensical, etc. Try telling us what your project is, and then we might be able to suggest a useful solution, possibly not involving XYZ at all, or involving a very particular/practical version of XYZ.
Except he doesn't want your alternate solution. He wants the solution he requested. If you don't know of a way to do it, then move on.
Re:You're doing it wrong! (Score:5, Insightful)
And if he's trying to see if he can read data patterns off platters by hand from a dismantled drive, and needs a known test pattern to calibrate his equipment with?
Headlines are superfluous (Score:5, Insightful)
Don't you hate it when people refuse to accept the premise of a technical question and write long monologues why the submitter is working with false assumptions even though they don't know what exactly they are dealing with? Yeah, me too. Makes them look arrogant, ignorant and smug. I'm going to go ahead though and reject the submitter's premise: there is no chance in hell that you're on the right track with whatever project you're attempting to do. But instead of merely dissing you for incompetence, I'll lay out a few scenarios (might as well, since you didn't supply any of your own).
If the actual physical bits matter to you...
you're either a hard drive manufacturer or a clueless person who should read up on how drives actually work. And we both know you're not working for a manufacturer. What you need to know is that there are several layers of indirection between the write call from within an OS down to the actual magnetic platter. These layers are there for a reason. At the very least, the onboard controller of the drive abstracts away the physical block allocation, and the drive won't work without the controller at all. Since the intricacies of the drive's physical address space are not accessible from the outside, there will never never never be a reason to try and fiddle with it directly. Because you can't.
If you are looking for disk I/O without a filesystem...
we're finally in saner territory. There are valid reasons to do this, e.g. speed and overhead considerations. Some database vendors actually have features like these. In this scenario, you're using the entire drive as one big addressable blob. A good starting point would be to have a look at the source code of a simple filesystem, such as ext2. Strip away all the actual file handling stuff and learn what you can from the disk I/O routines. On the other hand, if you didn't arrive at this conclusion yourself, that's not a very encouraging sign.
If you simply want a drive without error correction...
you're not developing software that will run on any modern system. If you accept this caveat, you can buy an ancient drive off ebay and use that. However, keep the first scenario firmly in mind: there is simply no reason to control the exact placement of every single byte if you don't plan on literally putting the drive under a microscope afterwards. Otherwise, this has no practical implications and, again, you are on the wrong track.
If you're a DRM/malware/virus developer...
I will sleep very comfortably tonight, because you had to ask about this on Slashdot, signaling once more that you're doing it wrong.
Re:That is gonna be hard (Score:3, Insightful)
Yet to someone who is not a troll and has technical knowledge it makes perfect sense! :-p
Re:Slashdot trolled (Score:2, Insightful)
Or maybe the submitter is a developer who has been told to do something impossible/completely impractical by someone in management. Management doesn't believe that what they are asking for is impossible/completely impractical so this is the submitters attempt to get an informed body of people to say the task is impossible/completely impractical so management will believe him.
Re:Data Recovery (Score:3, Insightful)
Wow, nice contribution. You totally miss the point, which is that writing arbitrary magnetic fields to an arbitrary physical location on the platter is the very last thing you want to do during data recovery. Now, if you just want to write arbitrary data at arbitrary logical locations on the disk, then you can already do that with the disk in it's current state.
Neither of these scenarios are remotely data recovery, since by definition, you are overwriting data.
For what it's worth, I'd disagree that once you can read a human language, that you can also write it since there are visual and linguistic cues in written language that you have to provide when writing. In most people, the two are closely linked, but I'd be willing to bet that while learning to read/write, your reading aptitude probably has a slight lead over your writing aptitude.
Re:Tell us your project? (Score:3, Insightful)
I bet you're one of those people who force the rest of us to post extremely long and verbose questions on forums, IRC and mailing lists because you always come up with completely irrelevant answers if you don't get a wall of text explaining in detail why someone is looking for a specific type of solution (and even then you probably try to sneak in your irrelevant suggestions).
An example of what I meant by the above: Let's say you have a motherboard where you need to update the BIOS, the computer has no optical drive, no floppy drive and you have no USB memory sticks available, the machine is using grub2 and you're looking for a way to flash the BIOS anyway to get around some silly bug without having to wait until monday morning so you can buy extra hardware just to flash the BIOS. The correct answer would then be something like "Check out flashrom, you should be able to flash the BIOS directly from Linux without even rebooting then". The answers that are likely to show up on any random *nix forum or IRC channel are:
In fact, even if you explained all the above chances are some people would still give stupid advice and sometimes it's even beneficial to not explain yourself because you know from experience that 90% of those who read your question don't know enough about the subject matter to understand it and will instead give irrelevant answers.
/Mikael
Re:Tell us your project? (Score:5, Insightful)
What kind of answer do you expect (Score:3, Insightful)
if you describe your requirements so abstractly? It sounds to me like you've come up with a solution ("Let's write the bits directly to the hard disk") and now you want pointers on how to make it work without explaining why.
If you just want to get around the file system, then use raw block I/O to the device. However it *sounds* to me like for some reason you've decided the *actual physical* layout of bits on the drive is important, in which case you are going to have to write your own disk driver -- if the drive electronics themselves don't defeat your attempt to know where bit is physically written.
It's inconceivable to me that you'd actually need to know this. Since you only *read* the data through an interface, it should make no difference if you *write* though the same interface, as long as it's consistent. In other words, unless you are going to disassemble the drive and examine it with an atomic force microscope, you have no way of telling the difference between a physical layout and an equivalent *model* of a "physical layout".
If you can't say *why* you need to do this, at least explain the parameters (how much storage, how fast, what kind of retreival etc.). My first reaction was that you should not use a magnetic disk at all, but an MTD flash device. Even so, you're dealing with an abstraction. You have no idea whether the device itself has mapped a bad bit to a different location at the hardware level. However short of tearing the device apart and putting the flash chips in a special circuit, you'd have no way of telling.
Re:A real (but expensive) solution: (Score:3, Insightful)
I like your reply because it is one of the only to suggest a real sollution.
In a way, the provided answers show the current technically "watered down" slashdot community. I guess 10 years ago this questions would have given really interesting answers.
I myself had a similar question about 2 or 3 years ago. What I wanted to do was to use the LCD from an HP laptop for some DIY project. The problem was how to interface VGA RGB output with the propietary input of the LCD (made by samsung IIRC). I did a lot of research, got schematics/specs of the particular LCD I was working for and got into the point where I knew I had to create a RGB-to-something decoder/controller (I just let it go so now I dont remember). I even realized that creating the decoder/controller was a really challenging task (mostly time consuming, time I didn't had) or I could buy a 3rd party controller which was quite expensive.
Although I have very little knowledge of hard disks, I would suggest the OP to first focus on one specific type of drive, second, get the schematics of the drive controller and then depart from there.
I specially hate these post crying DRM or whatnot. Even though I "am new here" (see my /. ID) I really wish I could find a slashdot like page frequented by real tech geeks /nerds.
Re:If you have to ask, it's hopeless (Score:5, Insightful)
It seems to me that if you're involved in such a project and have to ask how to do it, it's doomed.
Agreed, except that he may only need that initial kick to find terms to google.
For example, I know that to do what he wants, he needs to learn about the MFM vs RLL interfaces from the mid 80s, not because he will emulate either, but if he doesn't understand both and how they worked and relate to each other, he's has no chance. He is more or less trying to make a hard drive version of the catweasel floppy controller. If he never heard of a catweasel he would have a hard time figuring out he needs to learn about it.
Image scans from the very detailed technical manuals for DEC's minicomputer hard drives are available online. This is from the era of individual TTL chips, he's basically going to replicate / emulate / reverse engineer that hardware into his microcontroller in order to write individual 1/0, after all they had a solution that worked 40-50 years ago. If he didn't already know that, he would have a hard time figuring out he needed to know that.
Now if he already knows that stuff, and is really asking how to get MPLAB working so he can program his PIC, well yeah then he's well and truely lost.
But given those first steps/hints, I think a reasonably experienced EE/CE type could probably figure out the rest of it.
Adding to your nuclear analogy, if someone went back in time and told folks after 1920 to play with neutrons and U-235, they would get quite the head start on the rest of the world that didn't get that idea to try that until much later. They had all the prerequisites by 1920 but it took many years to get the right combination of ideas...
Re:Sure (Score:5, Insightful)
That was my first idea too "Find an old MFM/RLL drive, and have at it" - then I realized, "You know, I'll bet that more than 1/2 of the readers of /. don't even remember them"
Re:You Da Man!!! (Score:3, Insightful)
Re:Sure (Score:3, Insightful)
but if there is an actual requirement to control zero and ones in specific tracks/sectors, you might as well build your own chip for a drive.
Not to mention that disks haven't actually written "zeros and ones" as such for at least 15 years, which is why a single pass of /dev/zero will wipe a recentish disk beyond recovery.
Re:Headlines are superfluous (Score:5, Insightful)
In fact no, I hate it more when people do not state their actual purpose, especially in cases where what they are asking combined with the fact that they *need* to ask, cannot prevent one from wondering whether they really understand what they're doing, or whether they are addressing the right problem from the right angle (which, in this case, appears extremely unlikely).
The problem with this topic is *exactly* that we don't know what we're dealing with, and the first thing any decent engineer would do is to try and figure that out (in fact, you started out to do just that). Surely the stated goal doesn't stop at writing bits in exact locations just for the sake of it, right?
In fact, if I asked this to a 100 engineers, I'd really expect that at least 99 of them will immediately ask "why the fuck would you want to do that!?". And the one who doesn't probably works at a harddisk manufacturer.
Re:Tell us your project? (Score:4, Insightful)
This [bash.org] bash quote is, I think, appropriate:
When someone comes in with a question like this, it is entirely worth asking what he's actually trying to accomplish - because writing ones and zeros directly to a hard disk platter is the deepest sort of black magic, and there's absolutely no reason why this is the only way to accomplish what he wants to do.
Re:Slashdot trolled (Score:5, Insightful)
Re:Slashdot trolled (Score:3, Insightful)
The submitter doesn't appear to know enough....
Or the submitter is a consultant who knows enough about his non-disclosure agreement to reveal only the bare minimum about his goals without revealing the purpose of the project and violating his NDA.
Well, since we're dancing around the short answer (Score:3, Insightful)
I'll give the direct/short answer to the question you're asking.
No.
The reason is that hard drives only write data in precise locations so they can find it later. You can't write anywhere other than those locations because the drive won't do it... not even with new firmware. The read/write heads may not even be capable of addressing the locations you want to write. The only way to write in an arbitrary location is to remove the disks from the drive in a clean room, and use a very precise CNC read/write head to address the locations you want. Disk manufacturers have machines that do this.
If you confine your request to only writing arbitrary locations within the physically addressable areas of the disk, then you can do what you want if and only if you write new firmware for the drive... it used to be firmware was in EPROM and couldn't be altered, but you can flash it nowadays. So you A) Buy a drive then B) Re-flash the eprom with code to do what you want (custom developed after reverse engineering the original firmware) and then you can write wherever, without worrying about niceties like the end of sector marks the drive uses to keep track of data locations. You'll have to do that sort of thing yourself.
So, confining our discussion to disk areas that the default drive firmware will write to, If you're looking for a way to ignore/override the OS I/O code, the disk controller firmware and the disk firmware in order to "talk to the bare metal" it may be possible depending on the particular combination of OS, controller, and disk, and assuming you have the right privileges in the OS. But there's no standard way to do it, nor API. You have to know exactly how the hardware in question works, down to the chip level and in some cases below. If the computer you're interested in has a different drive, controller or OS I/O code than you wrote your program for, then you have to re-write it.
I'm sure a lot of us here would be curious to know what you're trying to do... accessing non data sectors on a disk hasn't been done commonly for years, and when it was it was used for some awful copy protection methods (awful as in they created compatibility issues, even with "standard" PC hardware and also they could still be broken).
Erik
Re:Sure (Score:2, Insightful)
You know, I'm far from an Apple fanboy, but I used the Apple IIe and by the standards of the day, it was clearly not a "wretched computer with awful software". It had good graphics, a built-in ML monitor and BASIC, few quirks, and a low enough price that it dominated the USA public school system. The C64 was a better value, but it had its own problems (like the glacially slow floppy drive).
Why ask us if you already know the answer? (Score:3, Insightful)
Remember this next time you are struggling getting requirements out of a non-technical manager or user. The submitter obviously has the technical background, but is making a common mistake.
What is it you want to do? "I want to write bits directly..."
Really? That's your final goal? Just to write bits? No, there's some other task you want to complete, and you've determined the best way to complete that task is to write bits directly to the platter.
But if you can't write bits directly to the platter, or you don't know what additional issues may arise when you do so, how can you determine that is the best course to take?
So slow down, back up a step, let us know what your real goal is. You want X, and you think the best way to X is to write bits to the platter.
Re:No. (Score:3, Insightful)
You would need to completely bypass the drive firmware in a way that would make the drive quite unusable afterwards
"Afterwards" wasn't part of the spec. He just wants to write the 1s and 0s. Reading them wasn't part of the spec that he gave us.
Seriously, if he just wanted to use a HD for data storage and retrieval, he already knows how to do that. Maybe he just wants a magnetic platter decorated with his logo printed in magnetic domains, a la lightscribe. Or something entirely different. Since he didn't tell us, but yet wants to do something so unusual, we can't make assumptions.
Tell us *what* you are trying to do, not *how* (Score:2, Insightful)
Almost every time someone asks a question where they obviously have made an implementation decision that depends on "doing it differently than everyone else on the planet" while providing no information about what they are trying to actually accomplish, the problem can usually be solved in a much simpler way. While it is possible you are doing something exotic like trying to turn a hard drive platter into a meta-material by patterning high density magnetic patterns on it and so you really do need to be able to control the bits at the hardware level, odds are low.
You provided no information about what you are trying to do. There are pretty good odds that if you provide information about what you are trying to do instead of trying to get people to come up with a way for you to do it 'the hard way' you, will get an answer that will work for you.
Re:Slashdot trolled (Score:3, Insightful)
He was smart enough to consult with his peers.
The problem with doing something that hasn't been done before is that you have to figure out how to do it.
The first step is research. Slashdot seems a likely place to find that research starting point or a glimmer of wisdom that might point in the right direction.
Re:Sure (Score:3, Insightful)