Write Bits Directly Onto a Hard Drive Platter? 578
kidcharles writes "I'm working on a project that requires writing bits to a magnetic hard drive platter in a completely controlled fashion. I need to be able to control exactly where 1s and 0s will appear physically on the platter. Normally when data is written to a drive the actual bits that get written are determined by the file system being used, as modified by whatever kind of error handling the drive itself is using (e.g. Reed-Solomon). All of the modern innovations in file systems and error handling are great for reliable and efficient data storage, but they are making my particular task quite daunting. My question for Slashdot: is there a way to get down to the 'bare metal' and write these bits? Any good utilities out there to do this? Obviously a free and open source solution would be preferable, but I'm open to anything at this point."
magic marker (Score:5, Funny)
I tried it once (Score:5, Funny)
Got myself a disk editor, and wrote something on the very first sector.
After that I can't use the HD anymore.
The computer can't even recognize it anymore.
Re: (Score:3, Interesting)
I had a drive fail last year with some personally valuable information on (during the 3 hour window of time I had repurposed its backup drive for a data transfer - sods law, should have just gone and bought another drive).
I ended up sending it to a data recovery company who told me that the first sectors were corrupt, which is where the "softest" part of the firmware was stored and consequently there was no possibility of recovering the data, even with a controller swap or extracting the platters.
A friend o
Re: (Score:3, Interesting)
It's a Hitachi 2.5" 160GB manufactured in APR-07 labelled as having "Apple HDD Firmware 2006". It was from a white MacBook.
The guys that looked at it (ABC Data Recovery in Sheffield) gave me the explanation above.
My friend said that if that was the case, if the data could be extracted, it would have to be reconstituted by hand which would be expensive.
He recommended MJM Data Recovery in Somerset to do this.
May I suggest (Score:2, Funny)
Slashdot trolled (Score:5, Insightful)
Re:Slashdot trolled (Score:5, Interesting)
The submitter appears to know enough about what he's asking to know that it's also impossible / completely impractical. Recording ones and zeros directly isn't done for a reason, submitter appears to understand this.
Keep in mind that the question is not just why the submitter wants to write directly to the magnetic domains of a hard disk, but why the submitter asked the question of how to write directly to a hard disk.
I can think of a number of reasons for the latter:
- The submitter could be an author writing some cyberpunk techno-thriller, and wanted some technical feedback from the 'experts' on Slashdot.
- The submitter could be a CIA drone trying to write a report on how it's impossible for the terrorists to somehow 'hide' data on hard drives that even 'dd' can't read.
- The submitter could be a troll.
That's just off the top of my head.
On the other hand, I seriously can't think of a good reason why anyone would want to actually damage a hard drive that way. If all you wanted to do is very precisely create a surface with a complex pattern of specifically oriented magnetic domains, there's easier ways. For example, there are equipment suppliers that can provide high-precision two axis steppers with ready to use computer control interfaces. It's not hard to get equipment that'll go down to atomic precision [wikipedia.org].
Re: (Score:3, Interesting)
http://slashdot.org/article.pl?sid=01/02/22/0622249&mode=thread [slashdot.org]
Did any of the old black box recorder like devices write in a loop directly?
One more possibility... (Score:3, Funny)
Re: (Score:3, Funny)
Re:Slashdot trolled (Score:5, Interesting)
- Submitter wants (for whatever weird project of theirs) a surface with magnetic image on it (a'la LightScribe, only magnetically).
- Submitter missed the actual point, and wants to write raw data to drive (some embedded project) circumventing partitioning and filesystem overhead - quite frequent.
- Submitter wants to record a lot of analog/unreliable data - skipping error checking and using the extra space to store more data (errors = acceptable noise)
- Submitter doesn't care about data storage and wants to use disk heads as a very fast magnetic field sensor/generator
This is quite doable with ANCIENT pre-ATA drives. The computer controlled everything then, the head position, the moment of writing and so on.
It is doable with old ATA drives that run PIO modes, probably with a bit of electronics tinkering.
It is about impossible with modern drives. You'd have to replace the whole electronics with your own and drive the head and the motors with your own.
Re: (Score:3, Interesting)
I was stumped as to why one would want direct access to the drive like that, until I read your post. Thank you.
But I also agree that this is just about impossible, depending on what he's trying to do. There's no way he can control the heads of the disk: too many layers of abstraction exist between the user and the hardware. But it should be possible to control where data is placed on the disk: this is what disk defragmentation does, after all.
But even then I'm not sure that what the device reports is nec
Re:Slashdot trolled (Score:5, Funny)
Find a drive with characteristics similar to what you want, and contract with the manufacturer to modify the drive to do what you want. The manufacturer has the tools and engineers to alter the existing firmware. It won't be cheap. If the manufacturer refuses the proposal, buy the company.
Re:Slashdot trolled (Score:4, Interesting)
The submitter doesn't appear to know enough to say why he wants to do this. If this were a consulting job, I would say, what are you really trying to do ? My guess is that, whatever that is, there is a better way to do it.
Re: (Score:3, Insightful)
The submitter doesn't appear to know enough....
Or the submitter is a consultant who knows enough about his non-disclosure agreement to reveal only the bare minimum about his goals without revealing the purpose of the project and violating his NDA.
Re: (Score:3, Insightful)
He was smart enough to consult with his peers.
The problem with doing something that hasn't been done before is that you have to figure out how to do it.
The first step is research. Slashdot seems a likely place to find that research starting point or a glimmer of wisdom that might point in the right direction.
It CAN be done! (Score:4, Funny)
Re:Slashdot trolled (Score:5, Insightful)
Re:Slashdot trolled (Score:4, Interesting)
What you say is all well and good, but you don't think that knowledge of that purpose might help with the solutions?
It's nothing like that at all. It's more like that-won't-work-so-what-are-you-trying-to-achieve-maybe-something-else-will-work, which is fair enough.
A lot are asking "Why do you want to do that?", which sure looks like curiosity to me.
Re: (Score:3, Informative)
Re: (Score:3, Informative)
Re: (Score:3, Informative)
So even aside from boot sectors and so on, the data areas of the drive are separated by areas written by the drive's control firmware, so the la
Re:Slashdot trolled (Score:5, Informative)
The difference between the way you learned hard disks worked long ago and the way we use them now is due to IDE technology. Internally, the drive is still commanding cylinder, head and sector. Externally (to the PC), the interface is very different.
Way back when we used to have the computer telling the drive controller board to operate the physical drive mechanisms, with the controller telling the drive which cylinder, head and sector to retrieve. This was called Cylinder-Head-Sector (CHS) addressing. If you ever worked on an original IBM PC-AT, you might remember having to set the "heads and cylinders" in the BIOS settings. You, the human, had to select from a list in the BIOS how many heads and cylinders the disk drive had. (And the older the PC, the fewer the choices. That meant every drive you could buy already had to be supported in the BIOS, but since there were so few originally, it didn't matter too much. As I recall, the max size drive supported by the original AT BIOS was 33MB, even though the largest drive IBM offered at the time was something like 20MB, if you had all the money.) The protocol was called ST-506, named after the first commercially available 5-1/4" hard drive.
Integrated Drive Electronics (IDE) changed that. The controller board was moved from the PC to the drive itself. So now the controller board was set up by the drive manufacturer to know exactly how many heads and cylinders the drive had. To talk to an IDE drive, ATA was developed as a protocol. The computer would no longer send commands for a specific physical drive geometry, but would instead ask for a "logical block number", and the IDE would do a translation (looking it up in a map) to command the hardware to move to the correct physical cylinder, head and sector to retrieve the data. This is called Logical Block Addressing (LBA).
So unless the original story author knows exactly what drive he's dealing with, and can download the lookup table from the drive, what he's asking for is no longer realistic. Each individual drive has its own unique mapping from LBA to CHS, because each drive maintains its own map of bad sectors. A computer asking for logical block #52 might physically get it from head#0, cylinder#3, sector#21 on one drive, but due to bad block mapping on a different drive it might get it from head#0, cylinder#4, sector#35.
SCSI works on a similar principle, and Serial ATA has made further improvements on the scheme for performance and size changes. But all modern drive interfaces use some form of logical block addressing to access the data, and none use direct cylinder-head-sector addressing.
Re:Slashdot trolled (Score:4, Informative)
Spinrite may "write" 00000000 to a specific location on the disk, but that 00000000 is actually encoded as something else when it is written to the disk to maintain a minimum number of bit transitions. In the Telecom world, B8ZS is commonly used for encoding. AFAIK, the HDD manufacturer's algorithms are proprietary.
It is also worth mentioning that a hard drive head does not read a binary signal from the platter. It reads an analog waveform and converts it to digital via the encoding and ECC algorithms the OP is trying to bypass.
-ellie
Scott Moulton of http://myharddrivedied.com/ [myharddrivedied.com] is the smartest HDD guy I know. If you absolutely have to have a solution, talk to him.
Re: (Score:3, Funny)
gives new meaning to "my 2 cents worth" [google.com]
Sure (Score:5, Informative)
Just make your own controller chip for the drive.
Re: (Score:2, Interesting)
Agreed. I would find is difficult to believe there is a practical use for this, but if there is an actual requirement to control zero and ones in specific tracks/sectors, you might as well build your own chip for a drive. :)
If the solution sucks that maybe they're trying to solve the wrong problem. :)
Re: (Score:3, Insightful)
but if there is an actual requirement to control zero and ones in specific tracks/sectors, you might as well build your own chip for a drive.
Not to mention that disks haven't actually written "zeros and ones" as such for at least 15 years, which is why a single pass of /dev/zero will wipe a recentish disk beyond recovery.
Re: (Score:2)
Anybody actually does this or is it mostly about using, say, an ARM controller; almost off the shelf?
Re:Sure (Score:5, Informative)
Modern drives come with their own on-drive controllers, as far as I know. Chaining another controller on top will not give you any power to override the on-drive controller behavior. If the on-drive controller makes its own decisions on where and how to write data, your custom controller will not be able to override these decisions, I am afraid.
Re:Sure (Score:5, Insightful)
The custom controller would REPLACE the the on-drive controller, not sit on top of it.
=Smidge=
Re:Sure (Score:5, Insightful)
A whole chip? Really? That sounds hard.
Just Ebay an old 20MB Seagate ST-225 MFM drive, and write whatever bits you want.
It doesn't know any better.
Re:Sure (Score:5, Interesting)
A whole chip? Really? That sounds hard. Just Ebay an old 20MB Seagate ST-225 MFM drive, and write whatever bits you want. It doesn't know any better.
Or, the submitter could contact Seagate or another drive manufacturer and ask what it would take to get a drive with special firmware that let him write 1's and 0's directly to the drive wherever he wanted. Basically, remove the intelligence on the drive.
It's not that impossible for drive manufacturers to do things like this for you, if you have the $$ to pay for it. I don't know what the cost would be, especially since he's really only [probably] looking for a few drives for this project. If it's grant-funded research, the grant would pay for it. If it's an independent project of some kind, he's in for a surprise.
I took a behind-doors tour of a major drive manufacturer a few years ago. During our visit, we were able to visit with engineers - one of whom was head of the firmware engineering team. He told us lots of stories about the firmware requests they've fulfilled. One example was a customer who supported lots of old PBX systems. These PBX systems ran software from a hard drive, but due to the age the system only supported drives up to (around) 200MB. Nobody made drives that small anymore, so this drive manufacturer re-wrote the firmware for them ($$) so a 120GB drive (the smallest they made at the time) would only recognize & address the first 200MB.
So yeah, I'm sure a drive manufacturer like Seagate could write custom firmware for him that would meet his project needs.
Re: (Score:3, Insightful)
Re:Sure (Score:5, Insightful)
That was my first idea too "Find an old MFM/RLL drive, and have at it" - then I realized, "You know, I'll bet that more than 1/2 of the readers of /. don't even remember them"
Re: (Score:3, Informative)
You have a 30 MB HDD until you realize that your MFM HDD wasn't being sold as an RLL HDD because it failed RLL density checks at the factory, and your bits start decaying within hours or days of writing them.
Yes, this is the bitter voice of experience. An ST-255 wasn't always the equivalent ST-238.
Re: (Score:3, Interesting)
Does is have to be a hard drive? The Apple IIe was a wretched computer with awful software, but it had a floppy controller that was possibly the cutest electronic device ever made. It's been an awfully long time, but my recollection is that it was built from a handful -- six or eight -- of TTL chips and that not only COULD one control exactly what was written where in software, one HAD to control what was written where in software.
Re:Sure (Score:4, Informative)
IIRC the spec was you could not expect to _reliably_ write a bit stream that has more than two zeroes in a row. In practice you could usually get away with three zeroes in a row. You had to do stuff like write sync streams e.g. 11111111001111111100111111110011111111001111111100. Which are basically 5 x 10 bits of 1111111100.
You could control the drive head too. You could even move the head in sub-track increments - e.g. half track. However, tracks have to be at least a full track apart to be reliably read/written.
All these is why I find the submitter's request rather strange. When you get down to the low level details, there is no such thing as exact.
Even if you are writing one bit on the entire platter with a huge magnet
So I'm curious on what he's really trying to achieve.
That is gonna be hard (Score:5, Insightful)
You would need to replace the firmware inside the drive or use an undocumented manufacturer mode. Whatever they use to write the servo tracks would be interesting to you. You will be in the situation of the firmware writer: There will be problems all the way. Be prepared to find a way to position the heads (ever tried to find a servo track?). Most likely you also need to at least parametrize the amplifiers in the DSP part of the firmware that does the analog-to-something-to-digital so you can have direct influence on the "bits". Good luck
Re: (Score:3, Insightful)
Yet to someone who is not a troll and has technical knowledge it makes perfect sense! :-p
Your own Disk controller chipset? (Score:3, Insightful)
If you don't need lots of bytes, you should use an old disk. I mean very old, maybe going back to those 300-400MB disks we had on 486 computers or even older.
Recent disks have embedded error-handling mechanisms which will prevent you from writing what you want, where you want (physically speaking of course). The disk controller translates your "physical" disk addresses into its own internal mappings. The only way to bypass this would be to remove the disk controller's chip and put your own in place.
Maybe your question would be: "how hard is it to create my own disk controller's chipset?"
Re: (Score:3, Informative)
Hahaha. 300-400MB isn't "very old". It's only 20 years since I bought a 80MB hard disk for £150 and that was considered massive as a couple of years earlier 20MB was cutting edge and cost a few hundred pounds.
Re:Your own Disk controller chipset? (Score:5, Informative)
Read before write (Score:2)
Wrong. (Score:5, Insightful)
Yes, I don't what they are. Still wrong.
Re:Wrong. (Score:4, Interesting)
It would be nice to know why the poster thinks this feature is important. The only things I can think of is the following...
He is using parts from a hard drive to make a robot. So he wants to control a Robot using the hard logic board, to control the mechanisms.
He is using the high speed nature of the drive and the noses it makes to make a speaker out of it by moving writing bits in the right spot.
Trying to make a full real-time application.
Researching better algorithms for drive storage.
Attempting to make some sort of digital watermark on the drive.
The first two are just silly hacks which would get geek creds but not overly useful.
The Real Time System You probably need more then off the shelf and free software to do the trick
And the Research you could probably program a simulator to simulate the results.
The Watermark idea will work until someone fills up the drive.
So I would agree with the poster unless you come up with a good reason where there isn't an other approach I don't see the need to do this.
Be careful! (Score:5, Funny)
I can't help directly, but can give one important advice - careful how you distribute the bits! If too many ones get on the same side of the platter this will destabilize it, causing it to wobble due to the weight difference ( a one weighs quite a bit more than a zero, you know!) and potentially tearing the platter in two!
Re:Be careful! (Score:5, Funny)
I can attest that improper bit stacking is a very real danger - one that almost cost me my life.
It was in 1999 in Sierra Leone, and my last assignment before I left for Sydney was to punt three hundred reams of bit-printed A4 four kilometres up the Moa to a nearby monastery.
It had been a hectic week and the crew the boss had assigned to load the raft had already worked a double shift. Subsequently, they failed to read the job sheet properly and loaded the raft with the all the paper with the zero-bit set on one side and the one-bit set on the other.
I got about three hundred meters from the dock when I was startled by a large black snake swimming past. I quickly moved from where I was sitting on the zero-bit stack to the one-bit stack, and of course the raft immediately capsized.
I was told when I reached the shore that the snake had most probably been a mamba - one of the deadliest snakes in Africa. If I had been bitten I would have been dead within minutes.
I hope this serves as a warning to anyone involved in any kind of data stacking. In short, always properly disperse your bits - your life may depend on it!
Re:Be careful! (Score:4, Funny)
Oh, hell yeah. Sometimes on a slow day, when there's little wind and a crowd outside the building, my colleagues will send around a bunch of emails full of ones. The cables outside twist and swing, and people look around for the source of wind.
Those recent communication satellite crashes? Too many ones got in their relay buffers, weighing them down and destabilizing their orbits.
Hm ... (Score:5, Insightful)
This kind of copy protection has been outdated for quite some while and should have died with floppy disks ...
Tell us your project? (Score:5, Insightful)
These kinds of questions are stupid: "I need to do XYZ for a project, how do I do XYZ?", where XYZ is one or more of complicated, ridiculous, vague, nonsensical, etc. Try telling us what your project is, and then we might be able to suggest a useful solution, possibly not involving XYZ at all, or involving a very particular/practical version of XYZ.
Re:Tell us your project? (Score:5, Insightful)
These kinds of questions are stupid: "I need to do XYZ for a project, how do I do XYZ?", where XYZ is one or more of complicated, ridiculous, vague, nonsensical, etc. Try telling us what your project is, and then we might be able to suggest a useful solution, possibly not involving XYZ at all, or involving a very particular/practical version of XYZ.
Except he doesn't want your alternate solution. He wants the solution he requested. If you don't know of a way to do it, then move on.
Re:Tell us your project? (Score:5, Insightful)
Re:Tell us your project? (Score:4, Insightful)
This [bash.org] bash quote is, I think, appropriate:
When someone comes in with a question like this, it is entirely worth asking what he's actually trying to accomplish - because writing ones and zeros directly to a hard disk platter is the deepest sort of black magic, and there's absolutely no reason why this is the only way to accomplish what he wants to do.
Re: (Score:3, Insightful)
I bet you're one of those people who force the rest of us to post extremely long and verbose questions on forums, IRC and mailing lists because you always come up with completely irrelevant answers if you don't get a wall of text explaining in detail why someone is looking for a specific type of solution (and even then you probably try to sneak in your irrelevant suggestions).
An example of what I meant by the above: Let's say you have a motherboard where you need to update the BIOS, the computer has no opti
If you have to ask, it's hopeless (Score:5, Insightful)
"I'm working on a project to build a nuclear powerplant. Is there documentation on how to do it? Obviously a free and open source solution would be preferable, but I'm open to anything at this point."
It seems to me that if you're involved in such a project and have to ask how to do it, it's doomed.
Also, the whole idea of it seems rather impossible. Why would you want to do this in the first place? Have in mind that hard disks don't write bits as is, not just because of ECC. They use an encoding to ensure that there are no long strings of 1s or 0s. If you just wrote some data without regards to that you'd run into a sequence you'd be unable to read later, due to not having a clock signal to figure out where a bit ends and the next begins.
Modern hard disks require using special encodings and servo data in order to be readable at those densities. It just makes little sense to me to want to bypass it. Unless you're working for a disk manufacturer this just seems very odd, but in that case you'd have access to the required equipment and information.
Re:If you have to ask, it's hopeless (Score:5, Insightful)
It seems to me that if you're involved in such a project and have to ask how to do it, it's doomed.
Agreed, except that he may only need that initial kick to find terms to google.
For example, I know that to do what he wants, he needs to learn about the MFM vs RLL interfaces from the mid 80s, not because he will emulate either, but if he doesn't understand both and how they worked and relate to each other, he's has no chance. He is more or less trying to make a hard drive version of the catweasel floppy controller. If he never heard of a catweasel he would have a hard time figuring out he needs to learn about it.
Image scans from the very detailed technical manuals for DEC's minicomputer hard drives are available online. This is from the era of individual TTL chips, he's basically going to replicate / emulate / reverse engineer that hardware into his microcontroller in order to write individual 1/0, after all they had a solution that worked 40-50 years ago. If he didn't already know that, he would have a hard time figuring out he needed to know that.
Now if he already knows that stuff, and is really asking how to get MPLAB working so he can program his PIC, well yeah then he's well and truely lost.
But given those first steps/hints, I think a reasonably experienced EE/CE type could probably figure out the rest of it.
Adding to your nuclear analogy, if someone went back in time and told folks after 1920 to play with neutrons and U-235, they would get quite the head start on the rest of the world that didn't get that idea to try that until much later. They had all the prerequisites by 1920 but it took many years to get the right combination of ideas...
Old IDE drives (Score:2)
Can you tell us what it's for, or is it secret? I assume whatever you're up to involves the destruction of the HD in question?
If you can find a working *very old* IDE drive (back from the era when they had bad blocks printed on the label that you had to map manually) I'd guess those use purely physical sector / block mapping.
Hmmm. The twisted, it is strong in this one. (Score:5, Insightful)
Short answer: no, it's not possible to do that. It's not impossible, simply incredibly hard to do. Unless you happen to be very handy with a soldering iron and go dumpster-diving in the backyard of an HDD company (as many others have pointed out).
Longer answer: how about creating a 'virtual' hard drive? There are utilities out there that probably let you create (a) a virtual interface (let's say IDE) and (b) a virtual hard drive, attached to said interface. The next step, of course, is to hack these (VirtualBox?) in order to make them do whatever it is you want them to do. This being said, I strongly suspect it involves non-trivial virtual machine hacking and that it probably does not respond to whatever your needs are.
Other than this (very twisted) idea, sorry, bare metal writing has been disabled a long time ago, and for good reasons, too.
Solution: (Score:5, Informative)
Old MFM hard drive should do the work. The entire device was controlled via software in dos. I have one lying around. A whopping 5 MB!
http://en.wikipedia.org/wiki/Modified_Frequency_Modulation [wikipedia.org]
Now all you need is a 8 bit ISA slot and you are set.
DRM? (Score:5, Interesting)
The optimist in me wonders if he is trying to defeat such a scheme.
Re:DRM? (Score:4, Funny)
The romantic in me wonders if he is trying to create a piece of art. Next week's front-page headlines are like "Hard-Drive Hacker Recreates 'Starry Night' Directly On Platter".
You can't do it. (Score:5, Informative)
It can not be done with off-the shelf equipment and software for many reasons, some of which are:
Line Codes [wikipedia.org]
Error Correcting Codes [wikipedia.org]
PRML [wikipedia.org]
SMART [wikipedia.org]
LBA [wikipedia.org]
etc.
Mel, is that you? (Score:5, Interesting)
We don't need to work like that any more. [jargon.net]
8-bit ST412/506 MFM + Linux circa 1994-5 (Score:5, Informative)
Get yourself an old, totally unintelligent S412/506 MFM controller from out of an IBM PC or PC/XT. These were fairly dumb devices (g=c800:5 in debug, anyone?) for which you were meant to enter a "bad sectors list" printed on the front of the drive after performing the low-level format yourself. The earlier the drive, the closer to bare metal, so you might want to look for any of the 2, 5, or 10MB (yes, MB) full-height (2 x 5.25" drive bays stacked on top of one another) drives that were floating around then. You'll also want to get yourself a set of ribbon cables.
You should be able to use a drive/controller combination like this with any machine with ISA bus slots up through about the 386/486 era, and that would let you also go back and grab an early Linux distro (say, kernel 1.2.13 days, like Slackware 3 or so) that included drivers for such a controller that were actually in use and known to work at the time, giving you a base on which to build more code.
If 10MB is too small, you might just have luck going up to the largest of the MFM (80MB) or even RLL drives (160-200MB, just get an RLL controller instead) drives. I don't remember whether there were any ESDI drives back in the day that didn't remap their own sectors, but if there were, these controllers were 16-bit ISA and somewhat smarter (also with Linux drivers from the period available) and went up to 680MB or so.
But if you're looking for the best chance of success for your purposes and don't need tons of storage, my educated guess would be that the MFM controller out of an IBM 5150 PC plugged into a 5MB ST506 hard drive and connected to a SIMM-based 80386DX mainboard with 8 SIMM slots (for 8MB ram) might be the easiest combo to find and get working in practical terms that has a chance of doing what you want.
Re: (Score:3, Informative)
Depending on where you had set the base address of the BIOS in memory (many controllers had jumpers that let you shift it around).
Some controllers also had multiple routines in addition to the lowlevel format: various kinds of analytics, verify passes, some had self-tests that you had to be careful with because you could tell it to seek to a cylinder way beyond the drive boundary and knock off the heads with a few hundred rapid bangs, which was actually sort of okay because you could literally replace head
That's nothing! (Score:5, Funny)
Re:That's nothing! (Score:5, Funny)
"transmute lead into gold in a cost-effective manner"
Rob a bank with a gun?
A real (but expensive) solution: (Score:5, Informative)
You know how data restoration companies do it?
They take out the spindle with the platters, and put it in their own reading device with its own controller. And with that you can read and write the exact bits (as long as quantum physics allow it). But the head has to be compatible (e.g. perpendicular recording needs entirely different heads).
I bet those devices can be bought, and I bet their controller is actually just software on the computer (for flexibility). I also bet they come with different head configurations.
But they are definitely not going to be cheap.
Hey, at least it is a real solution. :)
Re: (Score:3, Insightful)
I like your reply because it is one of the only to suggest a real sollution.
In a way, the provided answers show the current technically "watered down" slashdot community. I guess 10 years ago this questions would have given really interesting answers.
I myself had a similar question about 2 or 3 years ago. What I wanted to do was to use the LCD from an HP laptop for some DIY project. The problem was how to interface VGA RGB output with the propietary input of the LCD (made by samsung IIRC). I did a lot of re
Are you sure that's your requirement? (Score:5, Funny)
"I'm working on a project that requires writing bits to a magnetic hard drive platter in a completely controlled fashion."
Are you sure?
The reason I ask is I'm working on a project that requires me to move data faster than light. At least that's what I spent last Monday working up the math to prove that data replication between our different data centers has an upper bound enforced by the fabric of the universe and that it was impossible for me to achieve the project's stated goals without essentially inventing warp drive. As it turns out after a meeting it was determined that the goal was just a stated guideline. It also turns out the price of faster data transfer rates is prohibitive and after a further meeting the stated project goal was total baloney. Yes. Baloney. We had sandwiches. It was a nice meeting.
Re:Are you sure that's your requirement? (Score:4, Funny)
The reason I ask is I'm working on a project that requires me to move data faster than light. At least that's what I spent last Monday working up the math to prove that data replication between our different data centers has an upper bound enforced by the fabric of the universe and that it was impossible for me to achieve the project's stated goals without essentially inventing warp drive.
I have a solution for your problem:
1. Declare your main data center King. This requires a coronation ceremony and a crown.
2. Declare your backup data center Crown Prince. This does not require a ceremony.
3. Push the big red button that kills your King (the main data center).
Thus, by the laws of royal succession, your backup data center will instantly become King with all the knowledge of its predecessor. And that is how you to move data faster than light without violating the laws of physics.
Headlines are superfluous (Score:5, Insightful)
Don't you hate it when people refuse to accept the premise of a technical question and write long monologues why the submitter is working with false assumptions even though they don't know what exactly they are dealing with? Yeah, me too. Makes them look arrogant, ignorant and smug. I'm going to go ahead though and reject the submitter's premise: there is no chance in hell that you're on the right track with whatever project you're attempting to do. But instead of merely dissing you for incompetence, I'll lay out a few scenarios (might as well, since you didn't supply any of your own).
If the actual physical bits matter to you...
you're either a hard drive manufacturer or a clueless person who should read up on how drives actually work. And we both know you're not working for a manufacturer. What you need to know is that there are several layers of indirection between the write call from within an OS down to the actual magnetic platter. These layers are there for a reason. At the very least, the onboard controller of the drive abstracts away the physical block allocation, and the drive won't work without the controller at all. Since the intricacies of the drive's physical address space are not accessible from the outside, there will never never never be a reason to try and fiddle with it directly. Because you can't.
If you are looking for disk I/O without a filesystem...
we're finally in saner territory. There are valid reasons to do this, e.g. speed and overhead considerations. Some database vendors actually have features like these. In this scenario, you're using the entire drive as one big addressable blob. A good starting point would be to have a look at the source code of a simple filesystem, such as ext2. Strip away all the actual file handling stuff and learn what you can from the disk I/O routines. On the other hand, if you didn't arrive at this conclusion yourself, that's not a very encouraging sign.
If you simply want a drive without error correction...
you're not developing software that will run on any modern system. If you accept this caveat, you can buy an ancient drive off ebay and use that. However, keep the first scenario firmly in mind: there is simply no reason to control the exact placement of every single byte if you don't plan on literally putting the drive under a microscope afterwards. Otherwise, this has no practical implications and, again, you are on the wrong track.
If you're a DRM/malware/virus developer...
I will sleep very comfortably tonight, because you had to ask about this on Slashdot, signaling once more that you're doing it wrong.
Re:Headlines are superfluous (Score:5, Insightful)
In fact no, I hate it more when people do not state their actual purpose, especially in cases where what they are asking combined with the fact that they *need* to ask, cannot prevent one from wondering whether they really understand what they're doing, or whether they are addressing the right problem from the right angle (which, in this case, appears extremely unlikely).
The problem with this topic is *exactly* that we don't know what we're dealing with, and the first thing any decent engineer would do is to try and figure that out (in fact, you started out to do just that). Surely the stated goal doesn't stop at writing bits in exact locations just for the sake of it, right?
In fact, if I asked this to a 100 engineers, I'd really expect that at least 99 of them will immediately ask "why the fuck would you want to do that!?". And the one who doesn't probably works at a harddisk manufacturer.
Re:Headlines are superfluous (Score:5, Interesting)
Since the intricacies of the drive's physical address space are not accessible from the outside, there will never never never be a reason to try and fiddle with it directly. Because you can't.
Careful there. I know you're addressing a typical user of the hard drives, but there ARE ways of -- and reasons for -- getting at the "bits themselves". One of my colleagues, as a fresh student at one of the National Labs in the US, was tasked with recovering data from a crashed (literally crashed, so that the controller was destroyed and platters bent, with lots of data loss) disk. Of course there are ways and means of doing so, usually involving fantastically specialized equipment and lots of expense (and, often, the complicity or orders at the Federal level).
He ended up building a sweet little system which could scan the surface (in a low-level clean-room, of course), albeit slowly, and interact directly with the bits.
I'll have to ask him how they figured out the filesystem, the error-correction, and so on, without any of the usual partition tables and so forth.
Re: (Score:3)
With a channel program one could directly control the reading and wr
Really? Do NONE of us read? (Score:3, Informative)
He may mean something a bit different. (Score:3, Interesting)
Some operating systems have volume management tools that do let you get a great deal more specific about where data is written. Normally, I'm a Linux or Wintel guy, but I have managed some AIX boxes a few times and from setting up a large one once I recall that it's logical volume manager allowed you to create volumes on the SCSI drive systems that were very specific about how and what part of the drive was used. You could specific, for example, that a particular volume use only the outer tracks (or inner or middle I suppose) of the drives -- in addition to a great many options for raid striping and transaction logs and so on.
The idea is that the outer edges of the platters travel faster under the read/write heads than the inner and so performed differently. Also, you could keep the head from having to jump around as much by keeping all the data that tended to be used at the same time on the same tracks -- reducing your average seek time when reading randomly from that pool of data.
In practice, I think this mattered a great deal more when us old guys were dealing with 80+ millisecond random seek times on 5 1/4" wide full height (what would now be considered two bays) drives -- or larger disk-pack based drives (aka washing machines) with the massive physical movement necessary for those read/write heads.
Today, I think the admin is better focused on distributing the data load better across different drives/arrays to even the load out and also focused on reducing overall disk i/o in creating database schema and applications. Focusing on very fine details like this is likely to have only marginal benefit next to those key areas --- but I can't presume to know for sure what this project has in mind.
What kind of answer do you expect (Score:3, Insightful)
if you describe your requirements so abstractly? It sounds to me like you've come up with a solution ("Let's write the bits directly to the hard disk") and now you want pointers on how to make it work without explaining why.
If you just want to get around the file system, then use raw block I/O to the device. However it *sounds* to me like for some reason you've decided the *actual physical* layout of bits on the drive is important, in which case you are going to have to write your own disk driver -- if the drive electronics themselves don't defeat your attempt to know where bit is physically written.
It's inconceivable to me that you'd actually need to know this. Since you only *read* the data through an interface, it should make no difference if you *write* though the same interface, as long as it's consistent. In other words, unless you are going to disassemble the drive and examine it with an atomic force microscope, you have no way of telling the difference between a physical layout and an equivalent *model* of a "physical layout".
If you can't say *why* you need to do this, at least explain the parameters (how much storage, how fast, what kind of retreival etc.). My first reaction was that you should not use a magnetic disk at all, but an MTD flash device. Even so, you're dealing with an abstraction. You have no idea whether the device itself has mapped a bad bit to a different location at the hardware level. However short of tearing the device apart and putting the flash chips in a special circuit, you'd have no way of telling.
Not a Stupid Question - And a suggestion (Score:5, Interesting)
I'm surprised by the number of people who seem hostile to this question, mostly because they can't imagine any circumstances where they would want to do this. I can think of several. In fact, I was thinking about something similar with optical disks just earlier today. (I'm curious whether small holograms could be created by writing an interference pattern directly onto a CD or DVD, but I would need exact control over how the tracks lined up to achieve this.)
Here's some concepts just off the top of my head:
* He's come up with a new disk encoding scheme, and wants to try it out.
* He's doing research into how long disks retain data, and is questioning basic assumptions like whether surface bits permanently magnetize the platter underneath, affecting later bits in the same location. Or how far the domains spread.
* He's working with self-assembling molecules and needs to give them a patterned magnetic surface to build on.
* He wants to 'print' a 2-D picture onto a small portion of the hard drive, and bounce a laser off it. (The magnetic alignment of the surface domains would polarize the photons slightly.)
* He's making a high-resolution rotational position encoder, so by reading across 32 tracks knows the rotation of the platter down to a few nanoarcminutes.
* He wants to totally destroy the contents of a disk. (I assume this is what most of the hostile people think is the intention, presumably as the payload of some virus)
Those are just the ones off the top of my head.
However, the ranting people do have a point that without knowing WHY you want to do this, we can't really suggest the best solutions. A lot of people have recommended going back to super-old MFM hard drives that allowed this, but we don't know if you require the density of modern hard drives.
To do this with a modern drive, you're basically going to have to rebuild the controller. Either totally remove the controller board (leaving handy raw connections to the stepper motors and drive heads) or cut the connections between the microcontroller and the low-level electrical functions of the drive, and substitute your own. Here's where knowing your accuracy requirements could have help, because if you want relatively large bits, you can get away with fairly low-frequency components. A 20Mhz microcontroller can, with say an external high-speed shift register, push out an 80mbit/s serial stream, which equates to >120,000 bits around the track of a 6,000RPM drive. Not quite the same density as the manufacturer, but better than the old MFM drives.
Your next problem is going to be this: It's really, really hard to tell where the head is on a platter. I've no idea how modern drives do it, but it used to be done with 'marker' bits either in the track, or on a nearby 'index' track, plus a little timing. Preserving these location marker bits is actually the most important job of the drive controller.
It's not impossible, merely very difficult. I could probably make one with only several weeks of hard work.
Alternately, you could try getting into the firmware and re-writing it to your specifications, but that might take longer. You would have to reverse-engineer a lot of stuff that is specifically hardened against this, but at least the hardware would be stable.
Well, since we're dancing around the short answer (Score:3, Insightful)
I'll give the direct/short answer to the question you're asking.
No.
The reason is that hard drives only write data in precise locations so they can find it later. You can't write anywhere other than those locations because the drive won't do it... not even with new firmware. The read/write heads may not even be capable of addressing the locations you want to write. The only way to write in an arbitrary location is to remove the disks from the drive in a clean room, and use a very precise CNC read/write head to address the locations you want. Disk manufacturers have machines that do this.
If you confine your request to only writing arbitrary locations within the physically addressable areas of the disk, then you can do what you want if and only if you write new firmware for the drive... it used to be firmware was in EPROM and couldn't be altered, but you can flash it nowadays. So you A) Buy a drive then B) Re-flash the eprom with code to do what you want (custom developed after reverse engineering the original firmware) and then you can write wherever, without worrying about niceties like the end of sector marks the drive uses to keep track of data locations. You'll have to do that sort of thing yourself.
So, confining our discussion to disk areas that the default drive firmware will write to, If you're looking for a way to ignore/override the OS I/O code, the disk controller firmware and the disk firmware in order to "talk to the bare metal" it may be possible depending on the particular combination of OS, controller, and disk, and assuming you have the right privileges in the OS. But there's no standard way to do it, nor API. You have to know exactly how the hardware in question works, down to the chip level and in some cases below. If the computer you're interested in has a different drive, controller or OS I/O code than you wrote your program for, then you have to re-write it.
I'm sure a lot of us here would be curious to know what you're trying to do... accessing non data sectors on a disk hasn't been done commonly for years, and when it was it was used for some awful copy protection methods (awful as in they created compatibility issues, even with "standard" PC hardware and also they could still be broken).
Erik
Why ask us if you already know the answer? (Score:3, Insightful)
Remember this next time you are struggling getting requirements out of a non-technical manager or user. The submitter obviously has the technical background, but is making a common mistake.
What is it you want to do? "I want to write bits directly..."
Really? That's your final goal? Just to write bits? No, there's some other task you want to complete, and you've determined the best way to complete that task is to write bits directly to the platter.
But if you can't write bits directly to the platter, or you don't know what additional issues may arise when you do so, how can you determine that is the best course to take?
So slow down, back up a step, let us know what your real goal is. You want X, and you think the best way to X is to write bits to the platter.
Re: (Score:2)
This only skips the file system, not whatever the firmware does.
Re: (Score:3, Informative)
They (the bits you write to your typical /dev/sd* device) would still be wrapped up in sectors, the sectors be enhanced with error correction, then encoded (NRZ, Manchester or whatever is appropriate to the magnetic storage) and then written out. What OP wanted is to manipulate the bit that comes out of encoding. Given the usual stack of SCSI-like commands (and not direct access to the write head signal as early HDD controllers had) this does not seem feasible.
Re: (Score:3, Informative)
Besides the fact that Flash doesn't have a hard disk platter (his problem obviously isn't just to save data), Flash chips also contain internal logic to map out defective blocks, and in addition they remap in order to avoid writing single blocks too often in order to prevent them from failing early. So even if the HDD platter isn't really needed, he won't get exact control on Flash either.
If you had at least suggested a floppy. There you can indeed control exactly where your bits go, and it's even a magneti
Re: (Score:3, Insightful)
Actually that's the best suggestion yet. After he's spent a week banging bits into the 82072A, he'll either have gone insane, or will have realised he's well and truly barking up the wrong tree. Once again, OP: I'd love to know what you're up to, if it's not trolling Ask Slashdot.
Re: (Score:3, Insightful)
Re: (Score:3, Interesting)
First thought was - WoW! Somebody wants to defeat disk caching, ECC and make life bloody difficult to boot. The best way, if you really want to do direct disk RMW operations - is to use Flash. Its fast, reasonably portable, and won't fail you if you use it properly.
You still won't know where the data is actually written. Flash does wear leveling.
Re: (Score:3, Informative)
NOR flash doesn't require or use wear leveling, error correction, or bad sector management (as long as your wear pattern isn't ridiculously unbalanced) - use that if you want true "write raw bits" style functionality. Most microcontrollers use this type of flash, and chips are readily available, though they are expensive compared to NAND flash and only exist in smaller sizes.
NAND flash requires you do your own wear leveling, error correction, and bad sector management. You can still control all the bits, bu
Re:DD (Score:5, Insightful)
I'm actually more interested in why the fuck he wants to do this, much less how it would be achieved.
Re: (Score:2, Informative)
dd only bypasses the filesystem. It doesn't override the HDD firmware, which has to avoid bad sectors and write ECC information to the platter as well.
I would love to hear why the submitter thinks he needs to write directly to the platter.
Re: (Score:3, Informative)
RTFS
Knowing everything there is to know about dd isn't going to tell you where on a physical disk your data is stored. dd only sees a long string of zeros and ones that are a far abstruction from what physically happens on the drive.
There doesn't seem to be any way to be sure exactly where the data is written except by a deep understanding and maybe a replacement of the drive firmware.
Re:Talk to Steve Gibson author of Spinrite (Score:5, Interesting)
The stuff Spinrite does stopped working 15 years ago. On modern drives you're pretty much doomed. Considering Gibson has been known to rip off SYN cookies and spout such nonsense as "Windows RAW sockets are going to destroy the internet", I wouldn't go anywhere near him for advice.
Re:Talk to Steve Gibson author of Spinrite (Score:4, Informative)
No it doesn't.
If I read it correctly (it was a couple of years ago) all Spinrite does is move the head to different tracks before trying to read the data. The theory is that a seek from different distances might align the drive head slightly differently to the data.
As far as I know there's no way to position drive heads directly via IDE/SATA, much less write individual bits. IDE/SATA are high level protocols, all the low-level logic is inside the drive controller.
Re:You're doing it wrong! (Score:5, Insightful)
And if he's trying to see if he can read data patterns off platters by hand from a dismantled drive, and needs a known test pattern to calibrate his equipment with?
Re: (Score:3, Funny)
Good. Go write "Paulatz wuz here" next to the HOLLYWOOD sign.
Re: (Score:3, Insightful)
Wow, nice contribution. You totally miss the point, which is that writing arbitrary magnetic fields to an arbitrary physical location on the platter is the very last thing you want to do during data recovery. Now, if you just want to write arbitrary data at arbitrary logical locations on the disk, then you can already do that with the disk in it's current state.
Neither of these scenarios are remotely data recovery, since by definition, you are overwriting data.
For what it's worth, I'd disagree that once you
Re: (Score:3, Informative)
Re: (Score:3, Funny)
Re: (Score:3, Insightful)
You would need to completely bypass the drive firmware in a way that would make the drive quite unusable afterwards
"Afterwards" wasn't part of the spec. He just wants to write the 1s and 0s. Reading them wasn't part of the spec that he gave us.
Seriously, if he just wanted to use a HD for data storage and retrieval, he already knows how to do that. Maybe he just wants a magnetic platter decorated with his logo printed in magnetic domains, a la lightscribe. Or something entirely different. Since he didn't