Slashdot is powered by your submissions, so send in your scoop

 


Forgot your password?
Close
typodupeerror
×
Hardware

Specialized MP3 Compression Hardware? 26

Posted by Cliff from the the-next-stage-of-MP3-appliance dept.
Syn Ack asks: "Does there exist any specialized hardware for the sole purpose of compressing audio into MP3 format? Without going into too much detail, I'm putting together a platform that has to rip and compress as many songs as possible in the shortest amount of time (ie compressing several thousand CDs). I was considering a large ~20 machine *BSD or Linux cluster of high end x86 or Alpha hardware solely for compressing digital audio into MP3 format. Before I go ahead and look at other solutions, I wanted to check the distributed minds of the Slashdot readers to see if they know of any specialized hardware for this purpose. Price isn't really a concern. Something like the dedicated encryption boxes/cards used on Web servers is the type of thing I'm thinking of. Anyone have any ideas?"
This discussion has been archived. No new comments can be posted.

Specialized MP3 Compression Hardware? More

Specialized MP3 Compression Hardware?

Comments Filter:
  • use plextor 32x cdroms for ripping to get the best quality at the highest speed.

    dunno about hardware, but why not put up 10-20 boxes with the above drive, or maybe sharing a tower to make cd-changing easy?

  • It is possible by the sound of the above that you don't consider a DJ tired of dragging 1000s of CDs to every event/show/party that decides to rip the CDs so he/she can show up with a laptop and mixing board and the speakers and put on a show a legit reason to convert CD to mp3 format?

    Malk-a-mite

  • hardware, or targeted compile options? (Score:3)

    by Tumbleweed ( 3706 ) on Monday November 27, 2000 @01:41PM (#597877)
    I dunno if there's any hardware out there for that purpose, but if you can find an open source encoder and compile it with Intel's SSE or AMD's 3DNOW! (depending on machine type) you'd likely get a substantial boost.

    Also, if price really is no object, for efficiency's sake, rip them all to wav first, then encode later.

    If quality is a concern, use something like EAC (Exact Audio Copy - for Windows. There's a Linux/unix equivalent available, but I can't remember the name right now) when ripping. You'll get copies as accurate as possible before you start encoding.

    Make sure you use the right encoder, and make DAMN sure you use the right settings!

    LAME is considered the best encoder by most people, especially at higher bitrates. (I wouldn't bother with anything less than 128, but for my own collection, I do 256.)
  • Are there any soundcards around that include hardware to accelerate MP3 encoding? Cards like the soundblaster live include programmable chips which the manufacturer can give new functions by updating the drivers; perhaps they can help out here.

    Steve.

  • use DSP to offload server processor (Score:1)

    by Anonymous Coward
    If you are willing to do a little "roll your own" work, the best bet would be to use a DSP platform. I haven't seen any production platform for that, but the big three DSP companies (Motorola, Analog Devices, and Texas Instruments) all make evaluation cards which place the DSP on a PCI bus. Additionally, third party suppliers make cards, some with multiple DSP's. The latest Fraunhoffer encoder is claimed to run at 10x real time on a PIII 500, if that is fast enough. If not, use multiple DSP cards, and the server processor will have the job of submitting data for encoding, and keeping track of which cards have finished their encoding work and need more data. Fraunhoffer already have implementations for all three of the DSP's mentioned above, but didn't mention any hardware that was ready to go. Of course, by the time you do all that driver work, you could probably be finished with software encoding. --caudle
  • Imagine a beowulf cluster of.... oh yeah, we're talking about a beowulf cluster.. ;)

    I haven't heard of anything that would be directly applicable to your situation.

    I'm sure that there are DSPs that work well for encoding MP3s in real time, intended for MP3 walkmen and stuff.

    But you are talking about really cranking them out at faster-than-real-time. Which probably means you want a bank of drives and a traditional PC cluster with whatever CPU will encode fastest. My feeling is that someting with 3DNow/KNI would work better, but I'm not sure.

    Besides, general purpose computing is better. If you don't need all of the machines, you can put them to work for other tasks.
  • I doubt this is it - in this case, price probably would be a concern and 20 boxes would be overkill. I'm sure he's trying to build an encoding system for a commercial Internet radio service or other type of online music service.

  • The SoundBlaster Live! mp3+ was touted as having onboard mp3 encoding capability (through the firmware...it's the same card as the others). Specifically, it claimed a 5x acceleration up to 320kbps. As far as I can tell (I've got one), the acceleration is only used by the included mp3 encoding software.



    The thing is, with that and a p3-500 I can turn an album's worth of wav's into mp3s in about, oh, 10 minutes (vs. about 50 minutes in software using BladeEnc).



    I've got a friend with an 850MHz Athlon and a Kenwood TrueX 72X drive. With the Xing encoder he can do an album in 5 minutes flat...from the CD! The 72X drives are a MUST for mp3 encoding...the drive speed will almost certainly be your bottleneck.

  • lame, and distributed encoding.... (Score:1)

    by Anonymous Coward
    Using my K6/266 and lame, I can encode MP3's in better-than-realtime. I'd suggest setting up a fileserver (NFS) with a few SCSI cd-roms (Plextors of course).. give it say 1 hdd per client encoder machine, write yourself a shoddy little perl something or other using cdparanoia to rip them... then setup 2 or 3 or whatever athlon's(or durons, a 1200MHz athlon should be able to encode a 6 minute mp3 in 1 minute or so, though i could be a little off on that) with a shoddy little perl something or other encoder script to encode them and delete the wav's when it's done all automagically.. so you just get to sit there swapping CD's. The tricky part would be figuring out how many cd-roms to use for ripping versus how many clients you have and how fast they are. so as everything stays busy all the time. I think something along those lines would be really easy to implement as well as cost effective. especially if you have some machines laying around already that could be used.

  • Some automation (Score:1)

    by Anonymous Coward
    I don't know about the computer hardware, but one way to automate this whole thing might be to make a Lego Mindstorms-based CD Changer. Build up some sort of caddy that you can fill (with dozens, or hundreds of CDs), in some manner that the robot can grab them, and have the robot switch the CDs on you. That way you can keep the encoding running all night without having to resort to having someone watching dozens of machines 24/7.

    Perhaps you can have a dedicated CD tower with 15 or so drives (SCSI, of course) and have the robot drop a new CD into whichever drive tray happens to be open, and press the drive closed.

    Have the CD eject when done, the robot's touch sensor rigged to 'see' this and pop a new CD in. I think it might be worth the effort - you'll only have to keep the robot 'fed' to keep the encoding going!

    Naturally this would help - if you want to do the encoding quickly you'll have to use your hardware 24/7. This is one way to keep it going. At the very least, it's a neat excuse to play with Lego!
  • There is quite a lot of specialized hardware out there, but you'll likely need to write your own drivers in most cases. Sigma does offer some kind of Linux compatibility - they have a workstation/reference design that claims to support Linux.

    Sigma Designs [sigmadesigns.com]
    Darim [darvision.com]
    Optibase [optibase.com]


    Good luck.
  • Seriously.. Use commodity PC hardware. Even given specialized compression hardware, the price point you have to beat is so horribly low as to make it moot. I can encode 3 average CD's per hour on a dual PII-300, using LAME, at good bitrates.

    1000 CD's would take two weeks, granted, but extrapolate the numbers. For $200 you can purchase a board/cpu combo 1.5 times as fast, and handle that load in just over nine days. By the time you spend $1000, you have enough processing power to handle all those CDs in 45 hours. What about ripping? Well, using the best CD ripper for Linux gives me about 1/4-1/2 the rated max speed of the drive. So, to feed each 'node' (This is batch processing, so the term is in quotes) at the rate it consumes wavs you would require at least one 40x CDROM drive. Another $80 should cover that. Need Ethernet and cases, so toss on some $25 Tulip cards and a cheapo $40 case.

    We've spent $1,725.

    Okay, to cover the final costs; Three 40G IDE drives should handle the compressed audio and the temp space required by the compression farm. Say $600 for those drives, another $500 for the server they reside in, and $200 in associated 100T network cost.

    We're up to $3,025.

    Whether to place the CDROM drives in the server or in the nodes is left as an exercise for you; Putting them in the server is more convenient, but you'd require $100 in SCSI hardware. Doubling the number of CDROM drives doing the rip will decrease the number of trips you have to make to change discs to every half hour, but at that point it is vastly less of a headache to go with a pair of 14 disc cabinets and change every 1.5 hours. Really pricey to do CD cabinets tho..

    So, $3K for 500 CDs/day the cheap way, add $600 for some extra CDROM drives and SCSI hardware to make your life easier.

    Oh. Everything assumes you've got someone changing discs 24/7. Tweak for the desired work day.

    Someone else mentioned EAC for Windows; The equivalent for Linux/*BSD is called CD-Paranoia.
  • Screw the Plextor ... use the Kenwood 72x True CD-ROM. Under windows 98 I can get rip speeds of about 60x-65x. The CD-ROM has a buffer of 2MB and it blocks if it's empty -- I've ripped about 300 or so CDs with it and not one of the rips has skips in it (even my scratchy old Led Zeppelin CDS).

    Using this CD-ROM, the bottleneck on my p3-800 system is the encoding itself. It takes only 4 or 5 seconds to rip a 20 minute song, but more than a minute to encode it. This was your original question, though ... I think the most specialized hardware you'll find for MP3 encoding (at this point in the technological timeline) is a hefty processor. Speed really counts, since it's just number crunching. I liken this to the good ole days of software compilation on 386-SX machines -- the faster your CPU the faster a program compiled. I'm sure you know, but the better quality encoding you do, the longer it takes to encode a given song. I usually use moderate/high VBR for all my songs (since I like post-processing using my equalizer) and have found that the only way to speed up the encoding is to lower the quality of the resulting MP3.

    I've been using the AudioCatalyst product for about two years now, and it has been (and continues to be) the fastest and best sounding (don't bore me with comparisons of audio plots -- it's not the data that's in the MP3 that matters, it's how it sounds) MP3 ripper/encoder that I've found. CDex (http://www.surf.to/cdex [http]) is a close second since it fits nicely on top of any encoder that you want. I've gotten comprable speed out of Cdex as from AudioCatalyst (http://www.xingtech.com [xingtech.com])
  • The ripper is called cdparanoia, available at xiph.org [xiph.org]

    -----
  • He's not a DJ. If he was he wouldn't buy a cluster of machines to do something he only needs to do once. I bet he's trying to set up some kind of business that needs encoded CDs for such things as streaming. If you buy a 20 machine cluster, once all the major ripping is done you can convert half or 75% of them to streaming machines and leave the other half or 25% for future ripping purposes and general stuff.

    anacron.
  • You've just described a CD jukebox. Typically these have one or more SCSI drives plus a SCSI changer device (a robot arm). You can get a Linux driver for the robot arm here [strusel007.de].

  • Looks like DSP chips are in (Score:3)

    by pjrc ( 134994 ) <paul@pjrc.com> on Monday November 27, 2000 @04:58PM (#597891) Homepage Journal
    It looks like anything you find is likely to be DSP chips running the encoder. For example, here's a little announcement [ti.com] that at first looks like a custom encoder chip, but actually turns out to be a Texas Instruments TMS320C1500 DSP chip. Even the dedicated decoder chips (MAS3507D and STA013) are just DSPs packaged up with built-in firmware. I recall seeing a board with a few DSP chips on it a couple years ago, that could do real-time (back when no PC could). Obviously that's obsolete... maybe that's why I couldn't find the page again.

    It looks like PCs are hard to beat. Here's a claim of a Fraunhofer encoder at 10X real time on a 500 MHz Pentium3 [216.205.158.3]. Maybe it's vapor, but if it's real it's certainly a lot faster than LAME 3.87 (beta) [sulaco.org] when I tried it on a 800 MHz machine (Lame ran at approx 2X using "-V 3"). I recall hearing claims of LAME doing real time on a 266 MHz PC... maybe it does with different settings.

  • Constraints (Score:4)

    by maggard ( 5579 ) <michael@michaelmaggard.com> on Monday November 27, 2000 @05:36PM (#597892) Homepage Journal
    You're going to have several constraints ripping 1000's of CD's to MP3s.
    1. Physically loading the CD's into the readers. The cycle for retrieve CD, open CD, remove disk, insert disk into drive, retrieve CD, replace in box, refile box is slow & complex. I don't know of any system that automates this for jewel-box CD's; all autoloaders *I* know require the CD's be first loaded into special magazines which of course makes them pointless.
    2. Reading the CD into the system. Even with fast drives we're talking several minutes of time to read in each CD. Bus bandwidth would also come into play - I'm betting that SCSI would be the way to go here.
    3. Converting the CD audio to MP3s. Here you finally have a straight-forward read-compute-write process.
    All three processes would be best tuned to each other. Given that the only means I'm aware of actually getting the CD's into readers is manually then one can assume a hard limit of loading & reading any CD is going to be about 5 minutes under best-case hardware conditions (fast SCSI reader etc.)

    Since a single load-person could likely service two readers alternately this would make a work station. This work station would then produce 2 CD-datasets every 5 minutes. Assuming about six hours of loading per day one would achieve around 144 CD's of data loaded per station per day.

    Assuming two CPU's per station (1 CD-ROM/1 CPU) one would have 20 minutes to process each CD over 24 hours in order to keep pace. As I believe this rate is achieviable with a current fast system then you have a well tuned solution.

    Workstations consisting of 1 load person with two fast PC's = ~140 CD's per day processed

    No need for custom hardware, no need for clustering, all off-the-shelf stuff.

  • This review http://www.tech-report.com/reviews/2000q3/kenwood7 2x/ [tech-report.com] found significant problems with the quality of the Kenwood's Direct Audio Extraction. They were particularly a problem for a less than new disk which may not be an issue for the person asking the question.
  • May not be very helpful, but you can save alot of money and time (no need for clustering) by just using DistMP3 [freshmeat.net]. Its a distributed client server system that offload encoding to many computers. Like distributed.net.

  • Unless your use is tied to current MP3 technology ONLY, you're being very short-sighted by making MP3 compression your primary concern.

    Your concern should be about how you can read and store as many CDs in raw format so you're always ready to convert the library to the current audio format, be it MP3, Ogg Vorbis, WMF or whatever else tomorrow's standard may become.

  • couple the cddafs of BeOS with the gogo encoder and an SMP box, and you've got a great encoder.
  • Many manufacturers make 100, 300, 500 CD jukeboxes for use with audio CDs. Is anyone making one with a data-out option or an ethernet port for remote control, etc? How about using the TOS-link fiber out to send data to the PC for reinterpretation?
  • Well, it really depends on what you use, although I agree what the Fraunhofer at 10x is kinda incredible. Either way, it's a moot point, since Fraunhofer is good for lower bitrates. LAME seems okay for higher end -- certainly far better than most, but I've found Blade to be the best.

    Get a good Blade front-end, and even a decently fast machine, and you can do pretty fast encodes. Doing 320kbps with Blade on my Tbird 900 under Win2k Adv Server goes up to 4.6x, if I'm not running much else, and dips to around 3.95x if I'm using the computer heavily.

    I'm not sure if things like the P4's SSE2 would help, but seeing as how an optimized version rips the Athlon to shreds in Flask MPEG4, (see Tom's [tomshardware.com] perhaps MP3 audio is similar enough that it could also take advantage of SSE2 to great benefit...

  • I had limited space to elaborate on my questions. You are correct that storing the audio in a raw uncompressed format is the way to go. Initially we won't be doing this soley for the added complexity however we do expect to add this as an enhancement to our platform later so that we can convert the uncompressed audio to any compression type at a later date.

    Paul

  • Yes, we have looked a that type of solution. I have, through my limited tests, come to the realization that RIPing the audio is a slower part of the work than the actual compression. I've tried RIPing on both a DVD and regular IDE CDROM drive and they both took way more time to RIP the audio than the computer took to compress it. So perhaps my concerns in compression are unfounded and I should be looking for a very fast CDROM drive that can RIP audio the best.

Slashdot Top Deals

Beware of Programmers who carry screwdrivers. -- Leonard Brandwein

Close