Forgot your password?
typodupeerror
Programming

Ask Slashdot: Changing Career From OLTP To OLAP Dev 129

Posted by samzenpus
from the switching-teams dept.
First time accepted submitter xby2_arch writes "After spending over 12 years writing OLTP applications (Java EE/JDBC/ORMs), I decided to dabble in the OLAP world. I had decent DB skills, considering most of my previous projects had involved data modeling and coding using Stored Procs, etc. Yet I hadn't designed or implemented any dimensional databases. Luckily for me, I had enough relevant domain knowledge to land a developer job in a data warehousing project. The work was enjoyable enough that it motivated me to spend that extra time and effort I needed to cope with the different dynamics of coding in the OLAP realm. In my past life, data volumes weren't the primary concern (instead, transaction volumes were), here, everything was about data. ETL/Integrations present another set of problems you generally skirt in a typical web/app-tier developer role. All in all, it turned out to be a non-trivial, yet worthwhile transition. I am certain that there are plenty of seasoned developers out there who plan to make a similar move (or have made already), who see data as the next chapter in their careers evolving toward becoming Enterprise Architects. I want to hear what's holding them back, or what helped them move forward. What should be considered a prerequisite to make this switch, and what are the risks, etc.?"
This discussion has been archived. No new comments can be posted.

Ask Slashdot: Changing Career From OLTP To OLAP Dev

Comments Filter:
  • by Anonymous Coward on Sunday January 15, 2012 @03:04PM (#38707456)

    Please stand by while the trolls try to figure out how to respond to your question.

    In the meantime, you should expect about 50 posts by PHP developers asking what a stored proc is, and suggesting you move to RoR if you want an ORM.

    • by hedwards (940851) on Sunday January 15, 2012 @03:06PM (#38707476)

      The OP should quit his job.

      • Anyone who works at a business that hires employees with the title 'Enterprise Architect' should probably quit their job.
        • by cayenne8 (626475)

          Anyone who works at a business that hires employees with the title 'Enterprise Architect' should probably quit their job.

          Why?

          Those positions usually pay quite well...

    • by alphatel (1450715) *
      On behalf of the trolls I would like to ask, why are you anonymous?
      • by Anonymous Coward

        So we can post stupid answers and not lose karma...duh.

    • 50 posts by PHP developers asking what a stored proc is

      we feel offended. as if a thousand developers cried in agony because someone who is in a different field thinks that his field is more elite than ours. oh the humanity !

      hear ye ! hear ye ! person in random field thinks those in another field are less important and knowledgeable - and their work too.

      • by Anonymous Coward

        The irony is that in 15 years, if you keep learning and advancing in the art of software development, you will be making the exact same post as the GP.

        • nope. (Score:4, Insightful)

          by unity100 (970058) on Sunday January 15, 2012 @05:02PM (#38708342) Homepage Journal
          Actually i matured 5 years or so ago, after spending some 10 years in the delusion of thinking that the fields i was in were more 'elite' than the others. they were different fields than programming. but, the core concept of 'our work is tougher and more elite - other people are not as elite as us' nonsense was the same.

          i grew over it.
      • by Surt (22457)

        And yet, PHPs raison d'etre is that real frameworks were too hard to figure out.
        And ever after, as people struggled to deploy real applications on it, it has evolved towards being a real platform. Now that they're half-way there, it's now half as complex to learn as a real framework. Quelle surprise.

        • by unity100 (970058)

          And yet, PHPs raison d'etre is that real frameworks were too hard to figure out

          wrong. the main reason php came into being was the need for a framework that was fast to develop on, and non proprietary. asp was there otherwise. cgi was also there, but it was not flexible.

          'people struggled to deploy real applications on it' ? you probably are joking. or you havent at all delved into what php has become. flickr, facebook, yahoo ... if you are not counting what runs on these sites as 'real applications', then you are probably referring to non web related or niche applications that do ot

  • by Ethanol-fueled (1125189) on Sunday January 15, 2012 @03:08PM (#38707494) Homepage Journal
    The first thing we did to strategize mission-critical web-readiness to expedite wireless users was leverage our dot-com ROI and aggregate robust e-markets. Being able to synthesize cutting-edge channels enabled us to unleash extensible users, which in-turn enabled us to orchestrate turn-key mindshare.

    Utilizing our synergistic functionalities, we were then focused towards mesh visionary markets and envisioneering collaborative initiatives with our partners. The net result was the incubation of plug-and-play experiences to transform vertical vortals and utilize cutting-edge deliverables.

    Plus, I have a large cock. That helped a lot.
  • Eh? (Score:5, Interesting)

    by PCM2 (4486) on Sunday January 15, 2012 @03:13PM (#38707538) Homepage

    So... you're saying you've already made the switch from OLTP to OLAP and you'd like to take this opportunity to gloat about it, but you'd still like to hear from other developers what they think the prerequisites are for making such a move and what has held them back from doing all the cool stuff you're doing? Or am I missing the question?

    • So... you're saying you've already made the switch from OLTP to OLAP and you'd like to take this opportunity to gloat about it, but you'd still like to hear from other developers what they think the prerequisites are for making such a move and what has held them back from doing all the cool stuff you're doing? Or am I missing the question?

      You forgot to mention that he thinks that moving from being a code monkey to a data monkey is suppose to land him an architecture role. I would have thought his original job would see him better qualified. At best this is a step sideways but in reality it is probably a step backwards....and if he doesn't realise this it's probably just as well for all involved.

      But then even slashdot's heyday ask slashdot was about clueless time wasters asking how to do their job or apply for one they weren't qualified for a

      • by hedwards (940851)

        Since when does programming qualify one to design a house?

    • I mean the switch isn't so binary. He recently switched, he wants to hear other experiences to help him, and he generally thinks it would be a good Slashdot discussion.

  • No barrier at all (Score:5, Insightful)

    by Intropy (2009018) on Sunday January 15, 2012 @03:24PM (#38707630)

    Moving from one specialized type of programming to a closely related type of specialized programming is pretty straightforward. Apply for such a position and you wont suffer compared with other candidates. Or, if your current employer needs something new done, do that new thing. You're not talking about a major career change here. Programming is programming. Even moving from something like standalone application GUI programming for windows in C# to back-end web service programming in C++ on Unix isn't that big a deal. If you can program, you'll pick up the new tech/language/idioms as needed and notice the striking similarity in the work you actually end up doing.

  • Need advice (Score:5, Funny)

    by michaelmalak (91262) <michael@michaelmalak.com> on Sunday January 15, 2012 @03:25PM (#38707636) Homepage
    I'm thinking of making the leap from C# 3.0 to C# 4.0. Does anyone have any advice?
    • by Intropy (2009018)

      Go.

      For it.

      • by PCM2 (4486)

        Go.

        For it.

        Will that still work with WinRT? I'd hate to end up having to duplicate a lot of work.

    • by Surt (22457)

      Well, considering the rather dire straights Microsoft is in (most of their earnings tied to dieing platform), you might want to consider Java. Databases don't seem likely to go away any time soon.

      • by Cederic (9623)

        Given the way Oracle are buggering up Java, I'd be sorely tempted to find out what IBM consider their strategic language of choice.

        It may indeed be Java, but I know at least one dev house is switching to Ruby, HTML5/Javascript is guaranteed income these days and C or C++ will continue to get you jobs for some time to come.

        Or do what I did and get promoted. I get to draw pictures on whiteboards and let other people do the hard work :)

    • This leap is the same as going off a cliff. Use Java please.
    • by julesh (229690)

      I'm thinking of making the leap from C# 3.0 to C# 4.0. Does anyone have any advice?

      Have you considered Java 7 as an appropriate alternative? You're much more likely to get a job as an Enterprise Architect if you make the switch.

    • I love the fact that you made that joke, and people still needed to blast you for mentioning MS. And they actually claim Oracle-controlled, lawsuit hell Java is in a better position.

  • Changed Job (Score:2, Insightful)

    by 1s44c (552956)

    So you changed job from one thing to a highly related thing. Great, but why tell us about it?

    You totally lost it on '...evolving toward becoming Enterprise Architects', Seems you have been hanging around the buzzword management types for far too long.

    • by Gothmolly (148874)

      To paraphrase Syndrome:

      "When everyone is an Enterprise Architect, no one will be."

      I'm sick of this grade-inflation/feel-good mentality where everyone is somehow an "Architect".

    • by sribe (304414)

      You totally lost it on '...evolving toward becoming Enterprise Architects', Seems you have been hanging around the buzzword management types for far too long.

      Duh. That's a necessary part of the evolution he mentioned...

  • by Kittenman (971447) on Sunday January 15, 2012 @03:38PM (#38707726)
    Reminds me of the cartoon of a dull-looking man talking to a woman at a party - text below was "You may not think it to look at me, but in my time I've been a bank clerk, bank teller and bank customer liaison officer".
    • by sribe (304414)

      Reminds me of the cartoon of a dull-looking man talking to a woman at a party - text below was "You may not think it to look at me, but in my time I've been a bank clerk, bank teller and bank customer liaison officer".

      Great thing about that, you didn't even have to tell us that it was a New Yorker cartoon ;-)

  • Is this a conversation from a Dilbert strip, but without the punchline?
  • Whats a "career"? (Score:5, Interesting)

    by vlm (69642) on Sunday January 15, 2012 @03:45PM (#38707774)

    ... the next chapter in their careers evolving toward ...

    Whats a "career"? We don't have those around here in "IT related fields". I suppose if you live in silicon valley there is a chance of upward mobility, or maybe TLA .gov jobs, but for everyone else, its just luck that got us in a good spot in a downsizing economy in a downsizing company and downsizing department where they haven't axed us yet.

    "Career" in general would be a more entertaining "ask /." topic. Work in the above plus the "ha ha noobs don't realize than even the concept of my job didn't exist when I was their age" and plenty of ageism whining and funny stories about nepotism and stuff like that.

  • by Dishwasha (125561) on Sunday January 15, 2012 @03:52PM (#38707828)

    The prerequisites to making the switch is first and most importantly having an appropriate business case for OLAP. The second prerequisite is that you've tried doing analytics in a traditional RDMS, perhaps jumped on to the NoSQL bandwagon, and you've failed at it (i.e. success for a little while but then your data eventually brings your queries down to its knees). Don't worry, failure isn't necessarily wrong, it's just you and your team needed the experience before you could make the next leap.

    The risks are a knowledge jump in to an OLAP mindset from a traditional SQL mindset. Invest in you and your fellow developer's knowledge. Push back on management and sales when they want more immediate results and let them know that it will take 3-5 months to replace your current system. Do your proper technology evaluations. Learn FoodMart [microsoft.com] and Adventureworks [codeplex.com] and let them guide you down the path of good fact and dimension design. Don't snub your nose at Microsoft as they absorbed the company in the 80's that basically pioneered this stuff and made billions, but also don't take their stuff too literally as there are several products out there and some that do things better.

    Read The Data Warehouse Toolkit [amazon.com] thoroughly and practice using Mondrian [pentaho.com] which is an open source Java OLAP engine that can sit on top of PostgreSQL, MySQL, and others. Find a good ETL tool rather than trying to write your own at first and don't be afraid to force your internal users to use this tool to create their facts. Don't worry if you don't get it the first time, but keep trying and keep discussing with your fellow developers as it takes a team to work out all the kinks. Later on you'll probably end up seeing how you did things wrong, but hopefully you can get most things right in the beginning.

    • by jchevali (171711)

      Dishwasha, I agree, that is the way forward (and I'm glad you've actually engaged with the question instead of deriding it as others have done).

  • OLAP (Score:3, Insightful)

    by klahnako (209184) on Sunday January 15, 2012 @04:15PM (#38707948) Homepage

    From my limited experience, the OLAP community is small and/or behind walled gardens, the tools are poor and closed source, and potential employers are only interested if you have experience in *their* BI tools (Pentaho, Microstrategy, Cognos, etc). Microsoft appears to be the only one trying to establish a theoretical basis for BI, but their efforts are starting to show age despite their being so much more that can be done in the field. Finally, you will be misunderstood by the majority of Rails/PHP/Web developers: The same one who think Key-Value stores and NoSQL are the height of modern technology.

    That said, BI can be technically satisfying. If you get down to the SQL/MDX you will appreciate what a database can do; which allows questions to be phrased succinctly. I have seen too much code written in procedural languages (Javascript being the worst of them) that are many lines long and run atrociously slow, that can be restated in SQL (or MDX) simply, and run a 1000x faster. I love that fact there are no loops!

    From a business perspective, you have much more exposure to management and other departments: You will have improved visibility in the company, and your worth will be inflated - as you will be the one that satisfies management's appetite for more information to help make decisions.

  • by afabbro (33948) on Sunday January 15, 2012 @04:17PM (#38707958) Homepage

    Your previous experience as a developer/DBA is largely irrelevant. Data analysis is a completely different discipline (and depending on tools, may even be a different language than SQL).

    Basic building cubes is something you could learn in 5 minutes. Advanced stats, how to look for patterns, what to look for, etc. is much more involved. Most of the people I know who are good data analysts have advanced degrees in mathematics.

    BTW, why do you think the career path is dev -> data analyst -> enterprise architect? Completely irrelevant. Plenty of data analysts who couldn't write code. Plenty of EAs who couldn't construct an OLAP cube - they're focused on infrastructure, apps, etc. EA really has nothing to do with data analysis, other than designing systems to support it. In many companies, EA is not some priesthood at the top of the food chain - often they're a virtual team made of different disciplines.

  • Buddy, listen. Moving from OLTP to OLAP should be a minor, if not a micro move for anyone acquainted with database design. Don't waste our time and bytes with it.

    As for becoming an "architect", this is how I became one: I took all the ( little ) experience I had, I designed some stuff, made sure the code monkeys could and would actually code it ( me knowing their life as I had always been one ), got some $$$ incentives for them so they would build what I had thought up. I defended it in hard fighting with

  • I made a similar transition and my furious pace and insistence on sub 300 millisecond response times for any query, or web page load was foreign to those in the OLAP world. I would submit a query to our OLAP (I was doing pricing optimization—cool enough) which took over 60 minutes to get a useable dataset. Most of the people I worked with were very nice and extremely bright though they existed on a very relaxed pace; one I just couldn’t adjust to. I had such a tough time of it I moved back to
  • This is one of the things described as "If you can see a difference, you are not qualified to work with either".

    Face it. You have spent 12 years mucking around with one single application that happens to be easy to shoehorn into multiple systems. You do not understand underlying theory. You can not solve a simple problem -- how to organize data to perform complex queries that can yield some analytical information. This is not something you can "learn" from a vendor manual to some expensive chunk of unreadab

  • by Anonymous Coward

    Development is development. There are differences, but its still development. I am a DBA and a database developer. I have worked on very large OLAP and OLTP systems. Here are the main differences. My experience is primarily with oracle.

    1. You need to really learn SQL well to handle OLAP well. You often get very complex requirements. Processing large amounts of records always performs exponentially better using straight sql if possible. You want to use as little procedure code as possible. Don't write loops,

"If that makes any sense to you, you have a big problem." -- C. Durance, Computer Science 234

Working...