Should a Web Startup Go Straight To the Cloud? 442
Javaman59 writes "I am a one person company developing a web site from home. The site is hoped to attract millions of accounts and daily hits (just to give an idea of the scale of things, as its important to the question). My infrastructure is currently Visual Studio 2010 on a PC. To progress the site I need to set up version control, continuous integration, and staging. I have a Win2008 server VM, with all the Windows software (free and legal) to do this. However, I am only just competent as a Win admin, and I foresee each step of the way (setting up a domain; SQL-Server, etc) as a slow, risky process, and a big disruption to development. Should I forget my VM server (it will make a nice games machine!) and just go straight to the cloud for all my infrastructure?"
I call shenanigans. His username is "javaman" (Score:5, Interesting)
I call shenanigans. The username of this "Windows developer" is "javaman". He names the major Windows dev and SQL Server brands but doesn't mention Azure. He refers to "continuous integration" but pretends that setting up SQL Server is hard. Methinks this is a plant...though I'm not sure for what.
Re:Haha (Score:3, Interesting)
"Uh, he specifically wants to spend more time on developing and not worrying about the hosting part."
Considering the problems cloud services have had this past year, he's sorely mistaken if he thinks he'll be spending more time developing.
The cloud was not even viable for our small business - not reliable enough, and the people running the cloud are just about as clueless.
Watching Reddit go down hardcore because of cloud failures was even more of an eye-opener.
If you can't be responsible for every part of your business, you don't need to be in business, PERIOD.
It's too bad you're getting flamed so hard (Score:5, Interesting)
Because it's not a bad question. You're getting answers from a lot of people who are either so buried in the deeply technical side of things or locked into the past that they don't really understand it. Having shepherded a couple of dreamy startups through this phase, though, I don't think you are either crazy or necessarily under-qualified to make a successful site. To be honest, it no longer takes the hardcore technical skills that it once did, if in fact it ever did. Technical competency is over-rated by technical people; there are super-successful web businesses out there that started out (and sometimes even continued) with really shoddy coding and infrastructure setup. Craigslist and PlentyOfFish come to mind as examples. I know several others without such name recognition but which nonetheless did quite well for their owners, who slipped along with very basic programming skills and almost no hardware competency whatsoever.
Having the ability to outsource your infrastructure makes it even easier to do this today. I'm going to stay away from the "C" word because it's so shot through with marketing dross and misunderstanding now. But it's entirely possible to effectively do away with almost all the Windows admin if that's not your strength by going with hosted services. You are probably a long, long way from having to worry about Windows/LAMP stack comparisons even at your stated traffic goals, and using a hosted service will abstract that to the point where you aren't going to need to worry about it anyway. I wish you the best, but the reality is also that few sites make it big anyway, so while scalability is certainly something to consider at this stage, you shouldn't allow it to hobble your choices excessively. If you actually get there, it's almost certain you are going to have both the resources and the need to rip everything up and re-do the entire site from scratch once or twice along the way anyhow. You can re-tool then if you must.
Abstracting hardware doesn't absolve you from making other design choices that will afford scalability, and you should have some understanding of what's going on under the hood so you can make those appropriately. But you don't need all that to get started. I don't know anyone, at any skill level, who actually correctly made all the right choices on their first pass. You'll be learning along the way. That's actually an advantage; tech is filled with people who found their comfort levels and can't adjust to newer models.
It sounds like you are asking as much about your dev environment as production. I would say "yes," move it all to a hosted environment. At this stage, you don't need to be worrying about the underlying nuts and bolts. Get up and running quickly and easily. Be flexible and make adjustments along the way. You probably don't even need to go with a full-on PaaS provider right now, either; get a cheap hosting plan with a company that will help you scale when you need to. Depending on your service requirements, you can go with best of breed hosting to find the most efficient solutions for your various problems... use a SaaS vendor for version control, use a CDN for content hosting, and so on. It's cheap, it's fast, and it reduces the time and cost of failure. Failure is undoubtedly something you will run into a few times along the way. That's going to happen whether you are a technical genius or just some schmoe with a good idea. Build it in to your plans; don't over-invest (whether in time or money) in things until you can see better how things are working out.
If you have a good idea, don't be too afraid if you don't know what you are doing. A lot of the best people don't. One of the most valuable lessons you can learn is what not to spend time on, and a lot of things that certain folks here on Slashdot hold dear are things that you don't necessarily need to spend a lot of time on right now. Prove your concepts first. If you turn into the next Facebook, you can worry about infrastructure then. Until then, don't let the idea that you need a Facebook-worthy infrastructure before getting started to hold you back. Re-format that VM server into a games machine and go rent time elsewhere.
Re:Google App Engine (Score:5, Interesting)
I'd recommend you drop Visual Studio and Windows and "go to the cloud" on an environment that is already scaled out. There are a few options, but I think one of the easiest to set up and get going on is Google's App Engine system.
Yeah, drop everything you've created and move it to a completely different platform. One that is is proprietary so you're locked into that one vendor forever unless you want to rewrite your app.
Or, since you are already familiar with and comfortable with Windows development tools you can pick and choose from dozens of hosting providers that will provider a Windows VM or physical machine and if they change their price or terms of use to something you don't like, you can easily move everything to another hosting provider.
None of which will be able to scale your application to the levels that the guy mentioned. Scalability like that comes from architecture, not hardware, and Google has an extremely scalable distributed storage and processing architecture.
Of course, the odds that the OP will actually need that kind of scale are almost negligible, but it's what he asked for. As for the "Google lock-in"... that's manageable with a little up-front thought, as I suggested. Keep your business logic separate and you should be able to port easily to other environments. And, actually, most of the App Engine infrastructure is based on fairly standard stuff -- in Java you're writing Servlets and JSPs and in Python you can actually write most of your app as Django plugins. The main area you end up "tied" to Googel's APIs is around storage and task management, and around user identity/account management, should you choose to use Google's APIs rather than something else (there are lots of options) and it's not difficult to introduce a small abstraction layer to allow you to break those connections when required. If you grow to the point that it really makes sense to get off of Google, making the move will cost you a lot of effort and a lot of money, but it's just the effort and money required to build the scalable infrastructure, and you have to pay for that at some point, regardless.
The bottom-line reason for my suggestion was that it allows you to start small, almost ignoring issues of scalability, but to be able to scale up to massive volumes almost trivially. The Visual Studio and VMs approach doesn't provide that scalable, highly-available infrastructure out of the box, you have to build it, which takes time and effort which could be better spent on building the meat of the app.
Re:Hmm... (Score:5, Interesting)
For my websites, I have a PC in my garage, running on my DSL box. It works perfectly. I have some downtimes, sure. Just this morning, my kid found my router's plug and thought it was fine to unplug it because his DSi was out of battery... It lasted 5 minutes.
When I'll have bandwidth issues (which will come before HW issues - 1.5Mbps up) we'll see. For now, I have about 1000 hits/day and my DSL box is doing fine. It is most likely that my outage of 5 minutes went completely unnoticed.
That said, to do this, you'll have to be comfortable running a PC, administering a database, managing backups, etc. But you get the liberty of your own house.
Yes, Start at the Cloud (Score:5, Interesting)
(My background: I've been doing IT for the last 18 years or so; for the last two years, I've been working at Netflix, one of the highest-visibility cloud consumers out there. Until two weeks ago, I was on the IT side, focusing on the datacenter; about two weeks ago I moved over to Cloud Operations, focusing on the cloud (duh) and monitoring, specifically. The following is my opinion only, and does not necessarily represent the opinions of my employer).
In my opinion, the cloud is the easiest way to launch a new service with reasonable redundancy and growth potential. It's how I would start off any new business. Have there been failures? Sure. But largely, cloud failures have only impacted cloud consumers who engineered their environments in a non-fault-tolerant way, in the mistaken belief that "the cloud never fails." The cloud fails. It fails all the time. But following good design principles (ideally, be in multiple regions; at minimum, be in multiple availability zones; test what happens when an AZ dies an ugly death) will give you better uptime, with better cost, than you would achieve for a reasonable amount of money running your own datacenter systems.
And then, once you've got a significant enough size with a big enough ongoing consumption of cloud resources, you can look at creating your own DC environment.
Author's response (Score:5, Interesting)
Thanks for all the comments.
I put on a very thick skin before posting, so the "you can't be serious" and "you are obviously not qualified" comments have not bothered me.
Having been on Slashdot for some 10 years, I expected that (if my question was posted) that I would get some useful ideas from left field, and also some useful direct comments, and that is what has happened, and I am especially grateful to all who took my question seriously and shared of their highly valuable expertise.
One bit of info was unfortunately omitted from my question, because of finger trouble while I was editing it. Obviously I am using Microsoft products, but it's not just because it's all I know (it isn't) but because I have used .Net for 8 years, and find myself always enjoying the latest stuff (now Linq, ASP.NET MVC, and F#) and, as a developer, this enjoyment is central to my productivity. I would never tell a RoR developer to stop enjoying his stuff, and I give myself the same license.
A brief comment on the useful Win vs Linux comments. There has been some advice that, were I to switch to Linux, that I would find a Linux network easier to manage, as well as obviously saving money. I am also "barely competent" as a Linux admin, but I can see the wisdom in that, and I have found Linux administration much more enjoyable than Windows. If I had time to spare, then I would probably go with Mono, for the "best of both worlds", but that's not realistic.
A lot of the advice relates to the cost of scaling if the application takes off, with warnings about the costs then of Windows. That is a factor I had not considered. I am, because of my free Windows infrastructure, immune to the costs for development, but I will have to look at the costs for scaling. I had thought that, if I'm getting many hits, then I would also being generating sufficient revenue to pay for servers, or that I would have sufficiently proven my concept to get investor funding, and possibly switch to Mono then. IOW, that would be a problem I would like to have. The comments here have suggested that I reconsider this optimism
I haven't yet processed all the most helpful comments, and I will certainly be taking my time to do so, but there does seem to be some repeated advice that Azure is the way to go. This will, hopefully, provide a uniform environment for development, staging and scalable hosting.
Some advice to young people out there. Don't be afraid to ask stupid questions. The chances are that your question and the answer will save you a lot of time.
Thanks again, Slashdot
And, a quick comment on my username. I adopted "Javaman59" when I moved to modern OO development in 2001. I have kept it, with some fondness for those roots in Java, Linux, and the GOF patterns.
Javaman59