Ask Slashdot: Building a Web App Scalable To Hundreds of Thousand of Users? 274
AleX122 writes "I have an idea for a web app. Things I know: I am not the first person with a brilliant idea. Many others 'inventors' failed and it may happen to me, but without trying the outcome will always be failure. That said, the project will be huge if successful. However, I currently do not have money needed to hire developers. I have pretty solid experience in Java, GWT, HTML, Hibernate/Eclipselink, SQL/PLSQL/Oracle. The downside is project nature. All applications I've developed to date were hosted on single server or in small cluster (2 tomcats with fail-over). The application, if I succeed, will have to serve thousands of users simultaneously. The userbase will come from all over the world. (Consider infrastructure requirements similar to a social network.) My questions: What technologies should I use now to ensure easy scaling for a future traffic increase? I need distributed processing and data storage. I would like to stick to open standards, so Google App Engine or a similar proprietary cloud solution isn't acceptable. Since I do not have the resources to hire a team of developers and I will be the first coder, it would be nice if technology used is Java related. However, when you have a hammer, everything looks like a nail, so I am open to technologies unrelated to Java."
Silly priorities (Score:5, Interesting)
Youtube was a lame app with basic mysql setup. Same with Facebook. When it took off, they hired gold people and fixed the scalability issues. Twitter didn't exactly put scalability first either.
So get real. Don't worry about "hundred of thousands of users", but about getting something decent out there for users to try. If users come, you'll get scalablity sorted out.
Azure (Score:2, Interesting)
Sounds like you want a PaaS provider that doesn't lock you in to a platform. I have a similar problem to you (PHP not Java) and I rejected AppEngine for the same reason as you. To my surprise I am leaning towards Azure, Microsoft's cloud offering. Their website service allows you to write your web app in a few different frameworks without having to customize it for their platform and then only pay for what resources you use. Management is as simple as manipulating sliders to how many resources you are willing to devote to your app and are willing to pay for.
I have no interest in configuring VMs, configuring memcached, handling load balancing etc. My needs are simple, very basic PHP and Mysql. Traffic will probably start small but hopefully will spike big, but maybe it won't. Azure lets me handle this situation with a minimum of effort and expense. If they raise their prices or start to suck I can easily move my app since its simple PHP.
Re:Premature optimization (Score:5, Interesting)
Agreed. This guy doesn't really understand scalability.
The OP needs to read how Plenty of Fish started off:
http://highscalability.com/plentyoffish-architecture [highscalability.com]
* PlentyOfFish (POF) gets 1.2 billion page views/month, and 500,000 average unique logins per day. The peak season is January, when it will grow 30 percent.
POF has one single employee: the founder and CEO Markus Frind.
* 30+ Million Hits a Day (500 - 600 pages per second).
* 1.1 billion page views and 45 million visitors a month.
* Has 5-10 times the click through rate of Facebook.
* 2 load balanced web servers with 2 Quad Core Intel Xeon X5355 @ 2.66Ghz), 8 Gigs of RAM (using about 800 MBs), 2 hard drives, runs Windows x64 Server 2003.
And also about NginX:
http://www.aosabook.org/en/nginx.html [aosabook.org]
If you "need" multiple servers when you are first _starting_ out you're probably focusing on solving the wrong problems.
Re:Show me the users! (Score:5, Interesting)
This. The submitter has made an assumption that there will be hundreds of thousands of users. There might not. The only sure thing is that if he spends all his time trying to build a platform capable of serving hundreds of thousands of users right out of the gate, the project will probably fail before a single user sees it.
Remember: not even Facebook, Twitter, or eBay started off with platforms capable of handling their current load. They all started with something quick and built things out as their respective user bases grew.
Re:Show me the users! (Score:5, Interesting)
Pretty much the same thought I had.
Step 1 is to get a version that works for one user.
Step 2 is to get more than one user.
You're jumping a few steps ahead of the game.