Balancing Performance and Convention 171
markmcb writes "My development team was recently brainstorming over finding a practical solution to the problem that's haunted anyone who's ever used a framework: convention vs. customization. We specifically use Rails, and like most frameworks, it's great for 95% of our situations, but it's creating big bottlenecks for the other 5%. Our biggest worry isn't necessarily that we don't know how to customize, but rather that we won't have the resources to maintain customized code going forward; it's quite simple to update Rails as it matures versus the alternative. What have your experiences been with this problem? Have you found any best practices to avoid digging custom holes you can't climb out of?"
Concentrate on the 5% (Score:2, Insightful)
The customization is what is going to keep you employed. That's the specialization that keeps the customers from going overseas - even if you're Indian yourself. There's always someone willing to do it for less. No exceptions. So, develop those resources if you want to keep doing what you're doing.
Expand the toolkit (Score:5, Insightful)
Would it be out of the question to assess the needs of the troublesome 5% (and perhaps the other 10% that were shoehorned into Rails) and add a framework that's more in line with those needs?
Data exchange is sufficiently mature that interaction between applications in different environments is not an issue, so all you are left with is the added overhead of supporting two frameworks; not a bad thing if you consider the added flexibility of using the framework best suited to each application. Having options is usually a good thing.
Re:Expand the toolkit (Score:5, Insightful)
Mod up! If the only tool you have is a hammer, you end up treating everything like a nail.
As the OP pointed point, all development environments sooner or later hit this wall. I've never found one, (and am pretty sure that one will never exist), that ticks all the boxes for a typical mixed environment (the GUIs, OLTP, distributed database management, web access, making the coffee...).
In the 80s, I worked for a company that - as was fashionable at the time - developed its own '4GL' and decreed that everyone use it. In a satellite office, we tried, but found that in certain circumstances it just plain would not do what we needed. Guess what? We went back to 'hard coding' those key functions...
Don't worry about it, (after all, same bits of windows, to name just one OS, are written in assembler to speed performance).
So as the man says, expand your toolbox and your competencies - it'll make you a better project manager (to handle the interfaces) and programmer.
Re:Switch to django and python for starters (Score:5, Insightful)
Comments like this really don't address the core issues. Frameworks, and especially heavily-developed ones like Django/Rails, are rarely cause for insurmountable performance problems. Of course, caching and setting up distributed workload processing queues will be necessary in any framework, but that's up to the development team working on a specific app rather than the framework itself.
I've been working on a large Rails application for over a year, and we haven't found any scaling issues related to Rails. Instead, there are a number of things that we've done in our application in order to help it scale for our given workload.
That said, there are a number of things that Rails will do to help you scale your application. You may want to look at how Rails 3.0 (the next release of the Rails framework) is incorporating several ideas from Merb to make the framework lighter-weight and more extensible through Rack and the concept of Rails "metals." For offline processing in a Rails app, look at plugins like workling/starling and other message queues. I think you'll find that 1) Rails doesn't have scaling issues, programmers on Rails have several challenges that they have to face to make their applications scalable (just as in any other framework) and 2) As far as providing tools to facilitate developers in scaling their apps, Rails is quite a powerful framework and one that should be given a lot of consideration by development teams.
In conclusion, though, vague comments like "avoid Rails, use Django" are just unproductive spreading of FUD and don't really contribute to any significant discussion or understanding of how to get any of the viable web application development frameworks to scale in any specific case.
welcome to the real world (Score:3, Insightful)
Lots of tools have the property that they make the first 95% or so of the solution really easy and the remaining 5% nearly impossible. On balance, you're worse off than if you had done the whole project in another tool.
Re:Switch to django and python for starters (Score:5, Insightful)
Write your extension/hack as a modification of Rails and/or Ruby. Send them back to the maintainers of said products and see if you can get them inserted as a standard part.
If they do not exact a significant penalty elsewhere for the improved performance in your pretty mundane scenario. It is highly probeble, your changes will be accepted.
This isn't just true for Rails. It's true for any software (except stuff maintained by immature, anal retentive morons.
If it's L/GPL, BSD or other common free license, then this adoption of customer written code is a routine matter. If it's closed source and proprietary they may feel the need to get a lawyer involved and have you sign over all ownership rights to the code you wrote to improve their product (ask for money or stock when that happens).
Either way, in most scenarios good clean extensions tend to get adopted as part of the main program.
PS: Yes. Even "The Evil Empire" (Microsoft, Sun, Apple, IBM etc...) have been known to do this.
Re:I agree with the above poster. (Score:1, Insightful)
Having created large, complex websites, and maintained them for long periods of time, we have yet to run into intractible problems as a result of the language or framework themselves.
I guess that means that you by "large and complex" mean that the codebase is large and the functionality is complex. Sites that are large in terms of actual traffic often run in to limitations caused by the framework(s).
Don't trust a framework until you've tested it under real load. The fact that some company uses a framework is not a guarantee that that it will work for you, or even that it works all that well for them.
Re:Switch to django and python for starters (Score:5, Insightful)
And when the guy knowing Django moves on, what's the chance of finding a replacements who knows it? How much will that reduce the number of eligible applicants for a position -- will you disqualify fifty really smart guys for five mediocre ones who happen to know Django?
In my experience, being a system administrator for over a generation, what is the most maintainable is to go for standards and well-defined APIs, and document it, document it, document it. Especially document what's obvious and "standard practice", because it may be "standard practice" to do a certain thing in a certain language or certain package, those who come around ten years down the road may not know that this was standard, or why. So document, not so much what you're doing as why. Any coder worth his pizza will figure out what but be clueless as to why.
I feel compelled to add (Score:4, Insightful)
Most people who speak of "limitations" to their framework are people who expect the framework to do all their work for them: this is simply unrealistic. If that were how it worked, it would not be called a "framework", but rather a "product".
Re:Switch to django and python for starters (Score:4, Insightful)
>
No problem is insurmountable. But they are commonly cause for surmountable performance problems. And surmounting costs time and money, while not impressing those who noticed the problems or pay for fixing them.
If in doubt, KISS. Adding extra abstraction layers is generally not simplification. It's like adding another layer of management in a company, in that it is sometimes necessary, but it will rarely increase productivity or responsiveness, at least not in the long term. Especially not when a middle level manager gets replaced with someone who doesn't do things the same way.
Re:Switch to django and python for starters (Score:4, Insightful)
I worked in a small shop that used rails, we found that rails is... constricting, just for the reasons you posted to slashdot for. We looked at our options and switched to django+python. Maintenance wasn't a problem after that. I'd suggest investigating a switch now while you have an opportunity.
Out of interest, what did you find constricting about Rails? Your comment here (and the article, such as it was) are not very illuminating on this score.
I've found it fairly easy to modify, very easy to add plug-ins to, and very easy to add custom SQL queries to if necessary for performance reasons (though most of the time that's not necessary). The only area I can see it being constricting would be when trying to interface with a legacy db structure which can't be changed, in which case Django would have similar problems.
Because the article is not specific enough on the problems they'd like to work around, it asks an unanswerable question - there is no magic formula for trading off convention and performance, no magic bullet; at a certain point you have to stop relying on a framework and add something yourself if it doesn't do what you want or perform to your satisfaction. The same holds true for any framework, and though the received wisdom on Slashdot is that Rails is difficult to work with, or doesn't scale, many companies don't find that to be the case (yellow pages, etc etc).
Ask Slashdot (Score:5, Insightful)
Hardware is Cheap, Programmers are Expensive [slashdot.org]:
Re:Expand the toolkit (Score:4, Insightful)
so all you are left with is the added overhead of supporting two frameworks
And that's often much better than supporting one framework plus your own customizations. Once you customize it's often extremely difficult to upgrade the framework and have your patches still work. So there can be a lot of extra development time involved. Plus just the learning curve to get intimately familiar with the internals of the framework when that's often not necessary.
Version Control (Score:4, Insightful)
Use a good VersionControl system, git, murcurial or Subversion if you must, __but__ keep the meta-data out of the code. If you use sensible tools you get all that, and its much easier to work with the upstream,
Re:I feel compelled to add (Score:2, Insightful)
I like that.
Equally, most people who dismiss Linux because of its perceived lack of something or other, are people who expect the computer to do their thinking for them (or at least restrict them to safe choices). If that's what operating systems were about, then we would all be using appliances.
Big Surprise (Score:3, Insightful)
Those you know me, know me a lot. I had the same problem five years ago. And it started to turn working for clients and revenue-generating work into working for the framework and framework-customizing work.
So I built my own framework, from the ground up.
Now it's all my conventions, and no one else's. I customize in ways that I know will be compatible for future upgrades. Of course, I get to make those upgrades myself too.
But it means that I do everything "for the last time". If it's something that seems valuable, it gets thrown into a framework-compatible routine, and I never need worry about it ever again.
A lot more work early on. A lot more work for quite a while. And then, blissful no work for anything. Seven of the nine things in the "essentials" section of my invoices -- that every client project needs just to start -- amount to 90% of the essentials dollar amount, and amount to, no kidding, the twenty seconds to install hte framework for teh new client -- and by that I mean run the script that copies files, migrates databases, and changes passcodes. From then on, approximately 75% of the work is remembering the name of the existing function -- or looking at another client project to find it; writing documentation simply hasn't made it to the top of the list in five years.
I spend about 85% of my effort dealing with client content/sales/training/trust, and only 15% on the actual programming.