Ask Slashdot: Do Any Development Shops Build-Test-Deploy On A Cloud Service? 119
bellwould (11363) writes "Our CTO has asked us to move our entire dev/test platform off of shared, off-site, hardware onto Amazon, Savvis or the like. Because we don't know enough about this, we're nervous about the costs like CPU: Jenkins tasks checks-out 1M lines of source, then builds, tests and test-deploys 23 product modules 24/7; as well, several Glassfish and Tomcat instances run integration and UI tests 24/7. Disk: large databases instances packed with test and simulation data. Of course, it's all backed up too. So before we start an in-depth review of what's available, what experiences are dev shops having doing stuff like this in the cloud?"
Bamboo OnDemand (Score:5, Interesting)
Atlassian is already trying to push their customers in this direction. Their Bamboo OnDemand offering spins up AWS instance as needed for builds. In this case, you could still host a local Bamboo instance and use elastic remote agents.
One thing I do like about this sort of setup is that it keeps you honest about deployment. Your build environment stands up a new instance every time your remote agent goes stale and is recovered to reduce costs.
24/7 operation + Amazon (Score:0, Interesting)
You'd better get some serious analytics done on your platform before moving it to a service that has open-ended billing.
We do (Score:3, Interesting)
We're not at your scale, but we do everything with AWS and have found that it works well.
One thing you might want to do is reexamine your mentality around 24/7...you need to evaluate what really does need to run 24/7 and what needs to be available 24/7 (i.e. something that can tolerate the time it takes to spin up from an AMI).
For example, your Jenkins server could be configured with a master/slave arrangement that allows the main Jenkins server to be a small or medium instance that runs 24/7 and then when a build needs to happen, spin up a beefier slave to rip through it as fast as possible and then shutdown when done. Each build then has a fixed cost, regardless of whether it runs serially or in parallel.
Our main reasons for choosing to use the cloud were:
- We have remote workers, both permanent and a WFH policy...cloud makes it not matter where you're working from.
- Less maintenance...stuff mostly just works and most things are scripted rather than configured.
- We like the mentality of thinking of computing as a resource, not a collection of discreet machines. Running 5 builds in parallel is expensive when you think machines but costs the same as 5 serial builds when you follow the spin up, build, spin down philosophy.
Your CTO is an idiot (Score:3, Interesting)
He doesn't want to manage stuff in house because it's hard. But wait, that's his job, and why he draws C-level pay. If you are not just occasionally using it, the whole advantage of "cloud" goes away, unless you replace it with the concept of "outsource". Which might be his goal all along, either way, I would look for a new job. Cloud would be great if you needed to load test from 1000 machines or something, but even for that there are simulators.
Just do it (Score:4, Interesting)
Amazon has a detailed AWS cost estimator:
http://calculator.s3.amazonaws... [amazonaws.com]
When we migrated to the cloud, our actual costs were within 15% of the estimated costs.
But really, the easiest thing to do is just build a test environment and try it -- you only pay for the time you use.
When we migrated to AWS we knocked 70% off our colocation bill (we had more space at the coloc than we needed, but it's hard to move production hardware to a smaller space without downtime, plus we had significant savings in equipment leases and maintenance contract costs).
Our dev/test hardware was aging and becoming unreliable (and no longer matched production since we moved to AWS), so we moved that up to AWS as well, but even after that migration our total AWS bill less than half what we paid at the colocation center. We only run the dev/test hardware during business hours, or on-demand as needed -- we set up a simple web interface that lets developers spin up test instances as needed. AWS keeps dropping prices, so we're even as we've grown, our costs have remained relatively constant.
I'm in the process of migrating my business (Score:3, Interesting)
I've spent over ten years on dedicated servers, and have been very happy. Over the next year, I'll be moving into a private cloud scenario -- (not amazon or google, yuck. A local datacentre rolling their own.) I'll have some dedicated hardware (physical servers: CPU, RAM), and be sharing the rest of the cloud (storage, power, network, et cetera.).
It's interesting because there are no actual benefits to me in terms of performance, capacity, stability, or price by moving -- even backups aren't any more fluid. Of course, my platform and business model have been well-tuned over the years, and my sub-industry doesn't have the fluctuations that are typically heralded by cloud services.
So why am I moving? Abstracted hardware. I've reached that point where migrating from one dedicated server to another is a major undertaking. It's days of work, weeks of testing, and a huge risk to my business if I were to move any significant number of clients at one time; that means spreading it out over a year which means paying for the old and the new at the same time with zero additional revenue.
I've got no problem with resource management and capacity planning. I just have trouble actually growing through the transition points. Moving to a private cloud is likely to give me the convenience of being able to upgrade physical servers instantly without any worries -- it's the virtualization layers and load balancing mostly.
Wish me luck.
Says who? (Score:4, Interesting)
AWS has some of the lowest cloud prices I've found anywhere. You can get AWS instances for under $3/month reserved according to what you need. 'Small' Linux instances cost about $15/month reserved last I checked. In fact they'll even give you a Micro instance free for a year without spending anything as part of their 'free tier'.
How did you come to the conclusion AWS was expensive?