Mixed MP3/Ogg Streaming

haplo21112 writes "I am working on coding a web front end to choose music from the collections stored on my web servers hard drive, and then stream this music to machine connected to my stereo. Like many people I would think, my music collection is mixed between .ogg/.mp3 formats since I started ripping before .ogg(which I now use) was available. The problem I have run up against is I can find no utility for streaming, that supports both formats, specifically both in a mixed playlist. Mod_MP3 came close, but it turns out that it can handle only one at a time either all .mp3's or all .oggs, the playlist can't be mixed. I looked into the Icecast streaming utilities, but they seem to be mp3 bound at least till the icecast 2.0 release. Has anyone run into this same problem of mixed media, did you come up with a solution that didn't require redoing half your collection in the other format." Ah, if only the problem was that simple...

This is the problem, when a client, like XMMS, connects it negotiates the stream type. After this it just assumes all data after this point is of the same type. At no point can it switch content types. You can use something like a play list which lists multiple connections to simulate playing different formats in a row, the thing is this requires a reconnection to the server each time.

You can blame the two dominant protocols, SHOUT and ICY. One was created for the Shoutcast server and the other was created by the Icecast folks. Neither of them really considered the issue of carrying any other payload then MP3, or more to the point, changing content type in midstream.

At some point it would be great if Vorbis/Ogg became dominant because of the silly copyright restriction with MP3 that make the use of the lame encoder pretty questionable. It still has a way to go though since the code for bit peeling has yet to be finished and until that is completed, to down bitgrade an Ogg stream you have to decode it to some format like PCM and then reencode to Ogg (which is certainly not optimial for real time demands).

mixed payload

[JDizzy]

What we need is to create a new protocol that can handle different payloads. There are many problems with both methods in use today.

For instance, id3v2 has lyric tags that can be synchronized to the music. However, the reason you cannot have a player with the feature to sing-along to the bouncing ball is because the shoutcast, and icecast protocols don't support title streaming. They are also designed for one media type at a time... mp3 or ogg. Both were originally designed for just mp3 streaming.

The two methods both seem like hacks...
I'd like to see a new system that is based on xml for the meta data.. like the title-streaming. Maybe a multi channel system one being for the data stream, another for the Meta data.

Honestly the hard part isn't getting a new standard for streaming... its getting the people who make the decoders that have the issues. They would have to understand the protocals, and since these are the same people that came up with the two compeiting formats we have now... its pointless batle.

Re:mixed payload - and a shameless plug

[Siax]

I am currently working on a small perl program, which streams music, called trout [sourceforge.net] which is sort of part of a bigger kde program, but will soon get its own release. Its still fairly unstable to say the least, but it sort of works.

There is a reason for this plug, this post has got a point :-). I have been fiddling around with various headers being sent (and I still am) and at the moment trout pretends to be an ICEcast server, but I am trying to use x-audiocast headers, with content-length and connection: keep-alive. I'm still working on this (so it isn't even on CVS) but I had never realised I could use it for streaming oggs and mp3s, I was more concerned about sending information about songs, but having read this article I might have to see how it works.

We'll see.

Re:mixed payload - and a shameless plug

[haplo21112]

let me know how it comes out...as it may be useful for what I am doing in my project.

Edna can do this.

[stevey]

Edna [sourceforge.net] sounds like it could do what you want.

Basically its a smallish python program that acts as a, tiny, webserver - generating lists of all your songs, and streaming them to your clients browser - where you have XMMS, or Freeamp [freeamp.org] setup to play the stream.

I suppose if the music is already on the same machine as your webserver then it may be a bit redundant to have a second server - but Edna is neat, (this is the one program that made me start hacking Python...).

Re:Edna can do this.

[haplo21112]

ogg support? I don't see that listed on the Web page?

Re:Edna can do this.

[stevey]

Enda creates playlists, and a nice interface to them, so it doesn't directly support Ogg files.

.

Still, if you edit the source code for the program, you can add a line to include them in your playlists - afterall its XMMS which plays them.

Patch below:

+ '.ogg' : 'audio/ogg',

If you need any help with this then mail me..

Perhaps...

[Beowulf_Boy]

Decode all of your .mp3s into .wav
then recode them as .ogg?
It would work, its not doing what you want,
but it would work.

Re:Perhaps...

[KjetilK]

There was a bit of discussion on this topic on Vorbis-dev [xiph.org] not long ago. Worth a read if you're considering this.

Re:

[account_deleted]

Comment removed based on user account deletion

An Update

[haplo21112]

Last night while toying with ideas on how to proceed, while waiting to see if Question would make Ask Slashdot, I discovered something that does work after a fashion anyway. I setup shoutcast on the server(Linux). I am feeding shoutcast with Winamp's(NT) streaming component, from a playlist of all the MP3's and OGG's on the server. Winamp seems to be able to make the change back an forth between the two formats as it sends the data to Shoutcast(and presumably Icecast.) I can then pick up the stream on another winamp session(or freeamp or XMMS) on another machine. So it is possible in a way to do what I want if there existed a program that could do what winamp does, and have its playlist populated by a web interface while it is running.

Re:An Update

[krow]

Now mod_mp3 can produce output like that... aka dump its contents out in a couple of different playlit formats. The only one issue at the moment is that I am not sure what would happen with the different content types as they were recieved.

Re:An Update

[haplo21112]

The thing is its the exact opposite of what I am taking about. To really make this work the Streamer(in this case MOD_MP3) needs to beable to take a list of mixed format files. Then read that list and stream them out the client. I strugled with this for a while, but finally came to understand, where the confusion was coming from.

Please correct me if I am wrong, but Mod_mp3 is actually sending the files to client. So if I create a playlist.m3u and specify MP3playlist playlist.m3u in the mp3.conf file when Mod_MP3 starts up it reads the file into memory, and then waits for a client connection. When the client connects Mod_MP3 says ok here comes the first file and starts sending it. It assumes the client as asking for files in .mp3 format unless the ?type=.ogg is passed as part of the connect string. If this is passed it then assumes the client only wants .ogg files. Therefore the reason why it trips and can't play mixed format playlists(to clarify I am not talking about a playlist of files requested by a client, I am talking about the internal playlist in playlist.m3u) I am not interested for this project in the clients issuing ?op= commands to view and choose from the available files I just want them to connect and take what the server gives to them. I have also noticed that Mod_mp3 can't start a feed mid stream it starts over in the playlist for each new client that connects.

Winamp or similar is doing basically the opposite it reads the playlist of files its told to play and then can stream them back out to a recieveing source such as shoutcast which can then send them back out to the world. Winamp is reading the files in and playing them back, but at the same time through the streaming component sends the output as input in only one format to shoutcast to be sent back out agian. When I started I had hoped that I could eliminate the middle man by providing the playlist(which could be updated the same way one can add to the winamp playlist while its running) as a file to Mod_mp3 it would then "shout" them back out much the same as shoutcast/icecast do. Infact this is what it even appears to do until more than one format is put in the playlist. Its missing the internal conversion that is accomplished by what I did last night passing the files through winamp and then to shoutcast. This is still not ideal however, since it does support multipule file formats, but I can't easily code a web front end to add files to that winamp playlist.

Honestly I am only a web coder and know very little of C/C++ on the level that it takes to code such programs as winamp/mod_mp3 my job path took me away from that area over years so my skill in that area is very erroded, especially since I have not kept up over the years. So I really have been looking for a widget that fit into the back end side of what I am doing, and hoping to find the one size fits all soution which might not exist or for that matter even be possible.

For the Record if anyone is interested I can state the design goal(and I am thinking it might even be a good shake off the rust project were I to try to code it).

A program that can read a playlist which can be added to while the program is running. Ideally can be run from a command line, doesn't really need a GUI because its data is all those mp3s/oggs/or other format that might come along, and the playlist file which is intended to be added to/deleted from via another process such as a PHP/PERL/ASP/CGI script from a web site. It should do at least one of three things.
1. "shout" the files out onto a network port that a client connects to like a shoutcast/icecast server does.
2. Decode and then send the files to shoutcast/idecast like Winamp does, so the "ICY/Shout" protocol takes care of the rest.
3. Offer the files up like Mod_Mp3 does, but send them in such a way that the end client isn't concerned with the file type.

A note that there are programs that are avaiible, Idecast,IceS and Mod_mp3, that are almost capable of doing these things but each is missing one little piece of the equation.

Icecast/Shoutcast, currently has no support for Ogg, and in the current versions I can't see anyway of feeding it a changing playlist while it is running.

IceS, Currently no support for Oggs, but I am sure thats coming. Also I still see no way of sending it playlist changes while its running.

Mod_Mp3 really seems the best of the Crop for bing close to what is needed, in that it does support both formats, but can't handle a mixed playlist. It also seems to lack the facility for changing the playlist while its running.

Anyway a long brain dump for anyone interested.

Re:An Update

[krow]

Ok, I think I can sum up what you wrote into the following. mod_mp3 assumes that a file is an MP3 unless you tell it that the file if ogg. If you tell it that its reading ogg, it assumes all files are ogg.
Solution is to add a directive saying "mixed is ok", setting the default op to "send playlist" and then add code to load.c to tell the difference between ogg and mp3.
Once the "this is this type of file is written" the rest is trivial.

Re:An Update

[JDizzy]

Actually, like I posted above..... I think it would almost be best to take a stance on this... and creatre a new multi-media protocal for the streaming... Like... simply encapsulate the OGG or MP3 frames in another protocall. Yuck... I know... but it will eventually have to happen anyways.

Besides, I think mod_mp3 can do what they want.... mod_mp3 can act as a mp3/ogg repository.... and this is one way to use it. For example... store all our music on a mod_mp3 server, then load the mod_mp3 playlist (op=m3u2) into winamp, or xmms. Winamp can then simply act as a relay, and mod_mp3 can act as the source. THe question is if shoutcast, or icecast can then take the winamp input, and manage the change in formates from ogg to mp3, and back-and-forth. I don't think shoutcast can handle that situation... at least it couldn't a few versions ago (prior to the advent of mod_mp3)

Basically put, shoutcast make a few assumptions about the input stream based on what the input type was of the first item it streams... if that was an mp3... the rest of the stream will be handled like a mp3....

also... winamp is weird... if you notice... it has to first sample the input stream to check if it is ogg first, and then it check to see if the input is mp3. I found this info out after some serrious debug time with winamp... so.. if you want to have ogg stream... it had better be the first item that winamp gets.

Re:An Update

[krow]

New protocol requires support in players. Sure, we could get a plugin done for XMMS, but that still leaves the majority :(

Re:An Update

[haplo21112]

Exactly...then in the end it shouldn't matter at all...what kind's of files are in the playlist.m3u on the server.

Re:An Update

[krow]

Its now in the TODO. I have thought about it a bit and have a couple of ideas about it. Use default content type to determine what is played and allow file directives to get you to the other stream.

I have to say conversion sounds like more sense.

[biglig2]

Hmmm. It sounds like it might be as much trouble to set up your system to batch convert all your MP3 to OGG as it is to set-up mixed format streaming, and since that is the way you want to go, bite the bullet. Of course, ?I can imagine that if you have a huge collection it could take a lot of time. However, if you set your machine up to just chug through the MP3s when idle you'll mkae real inroads. Set it up so your streaming server just streams the Oggs, then you'l have the delight of more music coming on-line every day ;-)