Subversion as Automatic Software Upgrade Service? 41
angel'o'sphere asks: "I'm working on a contract where the customer wants a automated, Internet-based check-for-updates, update and install system. So far we've considered a Subversion based solution. The numbers are: a typical upgrade is about 10MB in size. Usually it's about 30 to 50 new files (which have an average size of about 200kB) and 2 database files (which can be anywhere from 500MB to 2GB) that change regularly. Upgrades are released about every 3 months, and this will probably become more frequent as the system matures. The big files are the problem as we estimate about 100-300 changes in every file.
The total user base is currently 2000 users, creeping up to probably 5000 over the next year, and might be finally end up at some 30,000 users.
Any suggestions from the crowd about setting up a meaningful test environment? How about calculating the estimated throughput of our server farm? Does anyone know of projects that have tried something similar using an RCS or a configuration management system?"
"We want to support as many concurrent users as possible (bandwith is not an issue). We use an Apache front end as a load balancer and as many Subversion servers as necessary on the backend.
My largest worry, from my calculations, is disk access on the Subversion server. We could not run meaningful tests, because a typical PC kills itself if you try to run more than 4 or 5 parallel Subversion clients doing an upgrade (due to insanely high disk IO, and high seek times)."
rsync (Score:4, Insightful)
If this was in java... (Score:3, Insightful)
Not Subversion (Score:3, Insightful)
CVS (Score:2, Insightful)
I have been using CVS to manage many different websites and/or projects on various servers. It doesn't store more then it needs (just the CVS folders) and it add, updates, patches and removes the files according to your repository.
Additionally you can use branches and sticky tags to keep track of files that don't need to be updated, or files that vary from client to client.
It is also easy to trigger and update over ssh or cron.
One downside compared to SVN is the lack of a binary diff mechanism, but I have been able to get by fine without it managing projects up to a GB in size.
Alex
Disk I/O (Score:2, Insightful)
Your problem is either that you don't have enough RAM in the system, or you have an OS that doesn't do a rational job of caching disk.
Or both.
-Peter
Re:times two (Score:3, Insightful)
Re:Some clarifications, especially about rsync (Score:4, Insightful)
Subversion doesn't need to cache requests -- the OS* does this itself. With plenty of RAM, whatever isn't being used by processes is used for cache. If you don't trust the disk caching algorithm, just make a 2.5G ramdisk and copy your files over to that when you want to release them. Then the disk won't be a problem.
* Assuming you're using a Real OS, and not Windows. Don't use Windows for anything that requires speed or reliability.