Become a fan of Slashdot on Facebook

 



Forgot your password?
typodupeerror
Programming IT Technology

4GL to J2EE Conversion Tools? 38

FunkyGibbin' asks: "I've just started with a new company that has outsourced a project to translate a 4GL language - some 1.5 million lines of it - into J2EE. It's way behind schedule, and keeps on tripping over itself at acceptance test points (one item is fixed, others are broken). The outsourcing company chose as their conversion tool IBM's Rexx. They don't parse the 4GL to a syntax tree and then generate from that. They 'match patterns' which then generates the output. In addition, they're not even using the object oriented features of Rexx; it's in procedural Rexx! They have no regression test suite that runs through the original system and the newly generated system to ensure that they produce the same result [this apparently was 'our job' to do it manually]. Frankly, I consider the outsourcing company to have adopted completely the wrong approach, but I'm happy to be proven wrong if compiler construction or language translation experts can tell me that Rexx was their best choice, or even a reasonable choice! Any and all information gratefully accepted - this project has so far cost many dollars, and the board is unsure what to do with it."
This discussion has been archived. No new comments can be posted.

4GL to J2EE Conversion Tools?

Comments Filter:
  • Easy (Score:3, Funny)

    by Clue4All ( 580842 ) on Wednesday June 05, 2002 @04:12PM (#3647830) Homepage
    Outsource the conversion to India and go play golf. You can probably still get in a good 18 before the sun goes down if you're on the East Coast.

  • Consider the fundamentals of the problem: You're attempting to translate 4GL code into 3GL code. Think about why 4GL exists, and its relationship to 3GL. It's not hard to see that you face a fairly insurmountable problem. You might find it easier to just write a new 4GL system capable of running your old 4GL system's code. The process of churning 4GL into a generic 3GL is somewhat harder (but very similar to) the job of a 4GL compiler or interpreter.
    • Consider the fundamentals of the problem: You're attempting to translate 4GL code into 3GL code. Think about why 4GL exists, and its relationship to 3GL. It's not hard to see that you face a fairly insurmountable problem. You might find it easier to just write a new 4GL system capable of running your old 4GL system's code. The process of churning 4GL into a generic 3GL is somewhat harder (but very similar to) the job of a 4GL compiler or interpreter.

      The only thing I have seen that comes close is certain RAD/CASE tools like Oracle Designer. Designer can absorb applications written with itself, Oracle Forms, etc, and output them as Java, and to a certain extent can absorb other applications (say, VB) and convert them to Designer. It's far from perfect, but it might get you 80% of the way there. I'd wager that IBM, Sybase et al have similar tools, but he didn't say which 4GL he was using.

      I'd say that the business case for doing so needs to be very carefully examined. If the 4GL is due to be discontinued and there's no-one other than the vendor who can support it (i.e. no contractors available who know it) then maybe there is a case for conversion, maybe there is a case for rewriting, particulary if it needs substantial enhancement. If it's to be buzzword-compliant, forget it, it's a waste of time and money.
  • by WasterDave ( 20047 ) <<moc.pekdez> <ta> <pevad>> on Wednesday June 05, 2002 @04:30PM (#3647983)
    It's a gigantic project being run by people who don't know what they are doing. You're all screwed, it's only going to get worse. If you push to meet the deadlines it will get a lot worse. Read "Death March" my Ed Yourdon and spend your time contemplating what a shame it is the IT industry is so slow at the moment.

    Dave
  • It sounds to me like you ought to do one of the following:
    1) Hire a team to write you the converter. If it comes out good, maybe you could sell it to other companies in the same boat.
    2) Give up and hire a team to start over, from scratch, and do it right this time. Will take longer, but will be easier, smaller in size when done, and cause fewer potential headaches along the way.
    Hope this helps. :)
  • So if I understand what you are saying correctly, is that you have an application that was written in a 4GL language and you want it converted to another compiler/system.

    Unfortunatly, there generally is no quick fix to doing this that I know of... Your outside contractors are going to basically have to write a language parser and then output syntax to the new language you are moving to. It's very likely to take a lot of time.

    Personally, I think trying to write a system to do this is going to take longer than going in and recoding for the new language, ie go through the system module by module and figure out what each one does, rewrite it as an object and see if things work correctly, move on to next module, repeat ad nausium.

    As far as using rexx, other than being slow if they are trying to write an automatic conversion utility, it's probably not a terrible choice however I would probably use perl for syntax structuring since string manipulation is perl's strong point.

    As far as object oriented design goes, REXX OOP is rather clucky and is not really OOP, been a while since I used it, but I think it basically provides OOP like interface by using linked lists and such... Nor does it provide for inheritance, better to go with a more OOP oriented language than to try to do so in REXX.

    What you probably need to do is to go back to the company, spend a week or two finding out what progress they have made, and determine when the project is likely to be done based on what they have done.

    Then decide if that is acceptable.

    Also try to decide whether the project is more than they can handle... Do they understand what is going on? From your description, it appears that they are not checking whether the project even works as the are doing it, therefore they probably are in over their heads... If so, yank the project and get someone else to look at doing it... If they are making progress and can show you that they do understand what is happening, it will probably be better to work with them to complete the project since a new development team is going to have to relearn the application before starting.

    It appears that your company wanted a silver bullet solution, ie an easy fix to convert the code from one system to another. They don't really exist. The other company promised to do so, but they weren't able to make it work either. In conversion was a simple affair then all windows programs would work on linux and vice versa... The conventional path would be to hire programers to do conversion line by line as suggested above...

  • The fact that REXX is being used isn't the problem. REXX is a fine language for text procesing and such. Converting those pesky "english-like" 4GLs are a pain to convert or even read- this project sounds like a pain in the ass. No other language is going to be a magic bullet for it, unless you find some special one dedicated to conversion between languages. :P

    Why are you so keen on their using the OO aspects of REXX? While REXX can (it depends on the implementation) full OO facilities (contrary to what one other poster said), that's not going to make it go any faster either.
  • The description lacks two pieces of important information:
    • Why? Does the 4GL program work? Why change it?
    • Which kind of 4GL? Some 4GLs are event driven, some are 3½GL, some are database-centric, some are GUI-centric, etc.
    To me it sounds like a project either heavily underestimated, or a project doomed to fail.
  • There is only one way to go about this that works and everyone will like:

    Gather requirements. Get input from everyone. Give a small drinking budget to programers who know someone who left (someone who left the company won't officially do work for you, but for a friend and a drink might give some insiteful rants on what is wrong, and this is invaluable in a re-design). Figgure out what was right, and what was wrong. Document how it should be, based on expirence.

    Get some sort of translation between the 4gl, and the J2EE. (ip sockets work), and re-impliment whatever needs to be done. Look at exteem programing, re-factor all the code, but make it work with the old stuff so you always have something working. Re-do the stuff that is buggy, slow, or has new requirements first. (Don't be surprized if some unchanging piece never gets re-factored)

    I've never done 4gl or J2EE (much less coversions one to the other), but my expirence in language conversions is everyone wants to do them not because of the limitations of the current language (even though there often are serious limitations, you can code around them), but because there is a lot of baddly designed code in the old system, and the conversion to something else is an excuse to re-write the bad parts. So get your requirements right this time, and get the design right.

  • Disclaimer: I am a software contractor.

    I'm confused. Are they writing a 4GL-to-J2EE translator, or are they porting the 4GL compiler to J2EE?

    The big decision for you and your board is whether you think the contractor is going to get you where you want to go, faster and cheaper than if you fired them and hired somebody else (like me! ;-)

    It sounds like you are unhappy with their design, implementation, project management, and QA -- and that's just about everything a contractor offers.

  • by zulux ( 112259 ) on Wednesday June 05, 2002 @06:04PM (#3648750) Homepage Journal
    ... all you buzzwords are a least 5 years old. Thats why it won't work.

    See what you really want is a Peer-To-peer B2B Distributed Cluster of High Availablity C# Tablet-PS Servers running a Micro-Kerenel Database. Preferably using Wi-Fi over Firewire. Oh yeah, don't forget that you need some of that Linux in there too. I've heard that that's good for cross-platform open-source binary-only drivers.

    OOP is sooooo early 90's, that I'm supriesed your project is still being funded.

  • I'm working a doomed-to-tail outsourcing project myself right now, and some things you've said strike a chord with me:

    You say:

    It's way behind schedule, and keeps on tripping over itself at acceptance test points (one item is fixed, others are broken).

    Then get on their ass. You're undoubtedly paying these people a healthy sum of money to do this job. Tell them to get it done or else. This is the singular advantage of outsourcing work: you spend money to rid yourself of some pain. Quit worrying about HOW they do it and demand the results agreed upon in your contract.

    However later you say:

    They have no regression test suite that runs through the original system and the newly generated system to ensure that they produce the same result [this apparently was 'our job' to do it manually].

    So, you didn't read your own outsourcing agreement? You can't blame this vendor for being this way. Stop letting the tail wag the dog and show them who's got the money. It's a buyer's market, use it to your advantage. If they keep this up, kick these fools out and cut your losses. I know some excellent people available right now who can get this done.

  • this project has so far cost many dollars, and the board is unsure what to do with it.

    so the slashdot crowd must know! *sigh*
  • by plsuh ( 129598 ) <plsuh@noSPam.goodeast.com> on Wednesday June 05, 2002 @06:37PM (#3648973) Homepage
    Take it from some folks who've been there and done that -- in a much smaller way and have nevertheless gone through a lot of pain.

    For the transition from WebObjects 4.5 to WebObjects 5.0, the developers at Apple created a converter that transforms Objective-C source code to Java source code. This was done by parsing the Objective-C source, creating a parse tree, and then running scripts against the parse tree. Simple pattern matching was found to be inadequate.

    This was a much smaller project than the one that you seem to be involved in. Why? First off, as computer languages go Objective-C and Java are very, very close in terms of structure and design. Second, even with the talent involved in this project -- whom I am completely in awe of -- they only created an 80% solution. The goal was to produce something that could get a developer well down the road on the way to a conversion, not do the entire conversion.

    Then, the Apple engineers tested this tool in the harshest environment possible -- their own code. It was used to convert the entire WebObjects source code from Objective-C to Java; they made it work before releasing it to customers.

    Management needs to take a step back and ask, "what the heck are we trying to do here, anyway?" Unfortunately, it sounds a lot like your management has their heads 'way up in the wrong place. A useful piece that I read recently is the Happy Valley Tax Authority case study [linuxworld.com]. <HINT>Consider yourself lucky that your resume is relatively current.</HINT>

    --Paul
  • What is wrong with the existing 1.5 Mloc? Translating/rewriting is going to cost $250k-$2M, so the switchover must improve profits at least that much over its first couple of years of operation.
    • $250k? Working? For 1500kloc? So you're assuming 6 working lines of code/dollar... at the $150/man hour they're paying the outsourcing company that works out as 900loc/man hour. Working. Debugged, fixed, documented, with regression tests, everything. No fucking way. Start at two mill and work upwards.

      Dave
  • These types of project are typically doomed to fail from the outset, and when it goes boom, the external guys will get off relatively scott-free.

    I would suggest checking the details of the original contract to see exactly what was included in it. My guess, however, is that the contract will be so vague that the contractors can get away with blue-murder. Ideally there should be performance clauses, milestones and details of the outputs required - if they're not there, try to see what out clauses you have and get out of it as soon as possible.

    Unfortunately management can be very reluctant to pull the plug on a project. My previous company would not accept that they ever failed in any software development projects, despite running 200% over schedule and 500% over budget in one disastrous case! (The reason I left BTW.) It would definitely be worth while to at least put a freeze on the project and perform an audit of the progress to date - get an external party to perform this audit, and make it clear to all involved that the company doing the audit will not be eligible to picking up the the contract if you decide to ditch the original contractors. Be prepared to pay for GOOD quality auditors too!

    It would also be worthwhile reassessing the need to convert the code. Is it really necessary or could you just modify/fix any broken bits? If you really DO need to move to a different language/platform, are you sure you have made the right choice with J2EE, and should you be converting or re-writing? These can be very difficult issues to answer in many cases - it is sometimes hard to determine where the trade-off point is between maintaining the existing code-base, converting to a new platform and re-writing from scratch...

    I would suggest that you look at books such as Death March, and even The Mythical Man Month, as they can give an insight on how to play the politics required in such situations...

    Whatever you decide to do, keep an eye out for a better job, but be prepared to come across similar situations... 8^)

  • There's not a snowball's chance in hell that pattern-matching is going to do the conversion. REXX scripts are only going to futz the syntax and that's not nearly enough. As a bare minimum the conversion tool will need to understand the semantics, and as you correctly surmise this is going to require at least a 4GL parser. If your company is lucky they'll get 10% of the way with the REXX scripts - the easy 10%.

    Afterall, porting is more than making it compile! You have to make it work too. That the contractors don't even have regression tests proves that they are inexperienced, ignorant, or con artists.

    My advice is to bail ship now: the company is doomed because the managers don't know what they're doing. Even the choice of J2EE makes it look like they're playing buzzword of the month. They've sunk lots of money into a half-assed attempt at a port instead of just biting the bullet and doing it properly. And the outsourcing company is taking them for a ride: whether this is fradulence or incompetence doesn't matter. Either way you don't want to be anywhere near this stinky shit.

  • As someone who has worked with a lot of programming languages, I can tell you that source code conversion is no easy job. And pattern matching is certainly not going to make it (believe me). And just making a parse tree from the 4GL code, and unparse it in java syntax is also a doomed approach. You also need to simulate the underlying execution model of your 4GL language. (PS: Whether they use Rexx or some other language, and whether they use OO, procedural, functional or rule-based programming is beside the point, they could all be made to work well).

    And of course, the politics in the situation is a real problem. You obviously have the wrong contract, if they can keep going on without even rudimentary simple regression tests. And if you hired them to make a converter instead of converting your code, you are also doomed (unless you can really afford to pay up until their converter is finished, which it will never be).

    First of all, you should stop paying them. You can also consider suing them for incompetence (although that is unfortunately not possible in most countries, so you should consult your lawiers and try to find a contractual break instead).

    Then you should consider your options.

    1. Convert manually. This is fail-safe, but likely to be expensive, boring, and take a lot of time.
    2. Fully automated conversion. This is likely to be very expensive, but it should be possible to sell the service later on.
    3. Half-automated conversion (what you are doing now. It's likely to be a mess, but with proper care it is possible to be finished sometime, although a schedule is hard to come up with, and as you know by now, a contract is hard to write if you decide to outsource it (remember: proper care!))
    4. Don't convert at all, keep using old code (that would be best)
    5. Use binary linking (JNI) to access stuff in old code, so you can use it from java (depending on source language, this can be pretty easily done in a half-automated manner. It is not very safe, but if you can limit yourself to a relatively small interface, it could work well).
    6. Use CORBA or some other (COM,DCOM,RMI) technology instead of binary linking to do the same thing. This would probably be better, but it depends on a lot of different factors.
    7. Make a partial or complete implementation of the entire 4GL programming environment in J2EE and use the old source code unchanged (or with minor manual changes). This could be simpler than code-conversion, and like option 2, it can be sold later on.
    8. Don't reuse, write a completely new system in J2EE, but while that will take some time, make sure you keep maintaining the old system all the way through.

    Personally, I would recommend option 4. If you use option 8, it can be combined with option 5 and/or 6 in the initial stages, you might be happy with the result. I would believe option 7 would be easier than option 2 or 3, and a partial solution would also be easier to maintain. But I would prefer both 3 and 1 to option 2.

    But anyway, this is a high-risk project, and with high-risk projects, you don't put all your eggs in one basket. Whatever you do, make sure that you maintain and keep using the original implementation, untill you have something you are happy with (or at least think you can see the light at the end of the tunnel). Try different options to see what is likely to work before you put a lot of money into it. And if you already spend a lot in something that is likely to fail, stop doing that untill you are sure you want to pursue this direction at the cost of abandoning the others.

    • You made a lot of good points but I think the asker has looked at them allready. I mean: not reusing the old code and wrapping it into a modern API(CORBA or whatever) surely has a good reason.

      Several posters question the point about regression testing.

      Unfortunatly the contractor CAN'T make a regression test suit! So its perfectly logical that he deffers that to their customer.
      The original code alone usualy provids not enough informations to craft a test suit.
      BUT: if the customer allready had one for the old code, the converter could use that to craft a minimum set of regression tests of course!
      The question is here: who pays it and on what was agreed on?
      (( I did some 600k LOC Y2K reengineering, we guranteed to find all Y2K problems, and we did, but testing after the conversion was the clients job. The reason is: the environment in which the converted code is running is the clients environment, not our test environment. We had about 20 sample cases in our whitepapers where we showed under which conditions our regression tests would be successfull but a client could have a case which would break them))

      In general the best approach is to have a true compiler and to generate a "data base" with the model of the old code. OF EVERYTHING.

      Creating such a data base could be done with simple REXX scripts, of course also with PERL and similar approaches. As you only need to find all tokens of the language and you only record their position in the source file. The compiling step or semantical analysis before the transformation is then done in the database.

      *IF* simple REXX scripts are enough, that depends on their approach: do they generate a reporitory of all source codes to be converted? Or do they work file by file (or class by class, or what ever the smalest 4GL unit is) and directly try to put out Java in a single pass?

      Working file by file sounds not good!!

      How does the output look like? Do they try to craft good looking Java code? I mean: would the Java output look as if a programmer had done it from the same spec from wich the 4GL code was crafted? That would be the case if a simple 4GL/switch or 4GL/if is translated to a Java/switch or Java/if.

      That is HARD, if not impossible to automate. Something like that can only be done half automated. A typical way is to have a work bench collecting all transformations in a data base and asking for each transformation step how to do it. (Probably with a set of proposals for the engineer to pick from).

      A logical but not necessaryly the best way to transform such a program is to have for each language construct in the 4GL language a class in Java.

      If that is the case a easy pattern matching approach can do 95%. BUT: you need for all language construcs a Java class!! If you have a if (CONDITION) in the 4GL you would need a class IF and a class CONDITION in Java. And a REXX script for finding the points where to use the appropriated classes.

      Bottom line:
      Yes, the approach (as the asker has described it) can work. BUT: it also is easy possible that your contractor has no clue. Obviously you have a communications problem. You do not trust him and he does not give you a reason to trust him. The contractor is over budget and oer time. Thats a BAD sign.

      About the technical approach I can't say much, I sketched why a "simple approach" can work and also why it is in general not adviseable to use a to primitive approach. Regression testing should be a joined efford. If they would use a "4GL to Java" compiler you would not need a regression test (similar as you do not need it if you simply compile C code to a binary). BUT: as for every "porting project" you need a test suit of the original WHOLE code and run that on the new target. That is best done by the code owner.

      The other suggestions, using wrapping and reusing the old code surely has technical reasons why you did not choose them?

      My advice: getting soon into touch with the responsible technical persons on the contractors side. If they can not show progress and solve your bad feelings they are the wrong ones.

      Basicly: they should be able to show you in simple not to technical words: WHY THEIR APPROACH WORKS.

      If they provide only hot steam .... likely they have no clue.

      angel'o'sphere
      • I mean: not reusing the old code and wrapping it into a modern API(CORBA or whatever) surely has a good reason.

        And the reason is likely that somebody heard that Java is what everybody is using these days. If you haven't worked for a large corporation, you cannot believe the amount of money that gets spent on opinions based on zero actual knowledge. I've seen hundreded of thousands of dollars poured down sinkholes obvious to anybody on this site, and I've talked to people who have seen tens of millions vanish in similar ways. Dilbert is, if anything, optimistic.

        Unfortunat[e]ly the contractor CAN'T make a regression test suit[e]!

        Then they shouldn't even be trying to do the conversion. How else will they know when they are doing it right? Short answer: they won't. They took a contract like this only because they were fools or greedy.

        As you say, about the best they will be able to do is some sort of 4GL->Java compiler, that treats Java as an object code format. But that is guaranteed to be about as maintainable as patching binaries. As Martin Fowler says,
        Any fool can write code that a computer can understand. Good programmers write code that humans can understand.
        To have this code be maintainable, a human who understands the purpose of the code will have to do much of the porting.

        Personally, I'd favor the wrap-and-replace approach. Build an external Java API that is a good OO version of what the app is supposed to do. Then write a metric shitload of glue to the old code, writing unit tests and regression tests as you go. Then just start scraping out the glue and putting real code in place.

        • I wrote: (without the [e]s :-))
          Unfortunat[e]ly the contractor CAN'T make a regression test suit[e]!
          You wrote:
          Then they shouldn't even be trying to do the conversion. How else will they know when they are doing it right? Short answer: they won't. They took a contract like this only because they were fools or greedy.

          But the customer asked for that.
          I worked around y2k for a company which had specialized into "porting of code".


          As you say, about the best they will be able to do is some sort of 4GL->Java compiler, that treats Java as an object code format. But that is guaranteed to be about as maintainable as patching binaries. As Martin Fowler says,

          You are right. I had made the way how it is done clear upfront with the contracting company. But the core question was: "Is REXX sufficient?" I liked to point out: yes, it can be the case.

          Personally, I'd favor the wrap-and-replace approach. Build an external Java API that is a good OO version of what the app is supposed to do. Then write a metric shitload of glue to the old code, writing unit tests and regression tests as you go. Then just start scraping out the glue and putting real code in place.

          I would do the same. But the asker probably can't do that? DB or hardware on wich the 4GL stuff is running is no longer available (in house)? the 4GL stuff can't link to C and is because of that not wrappable... no idea.

          angel'o'sphere
          • But the customer asked for that.

            Hmmm... On the one hand, the customer is, theoretically, always right. But on the other, as a professional I feel an obligation to not take a customer's money and give them garbage, even if that's what they ask for. I've turned down a number of contracts because I felt the client was clue-deficient and unwilling to listen. Sometimes that's enough to wake the clients up, but often they're quite determined to go off and waste a lotta dough.

            When one does tech support, one learns to hear the real question behind the stated question, the real need behind the service requested. I think devlopers should do the same.

            the 4GL stuff can't link to C and is because of that not wrappable... no idea.

            Yeah, I wish the poster had told us more about what he was up to. I presume this is mainly a text/batch application, in which case even if there's no official way to link you can write a lot of screen scraping and file examination code. I've done this before and had it work adequately.
  • maybe aubit can help, it's GPL'ed 4GL compiler, well actually translator - it generates C code...

    check out: http://aubit4gl.sourceforge.net/

  • You have a large piece of code written in a proprietary language that probably noone will know about in 10 years and you want to convert it to Java. I don't know why but I bet that the word "web" is in it. The reality is that you have a WORKING and DEBUGGUED large piece of code. Why don't you just reuse it? Why don't you just bolt an adaptor on top on your project? Corba is especially good at it, it has mapping for most languages and integrates seamlessly with J2EE. Sure it won't look as good as a pure J2EE project, but it will be a lot cheaper, and will probably work a lot better too.
  • Admittedly I know very little about 4GL, but I was wondering if anyone could comment on the possibility of generating a parser/compiler for the language in Java, perhaps using a tool such as JavaCC [webgain.com]? There seems to be quite a set of grammars already available [ucla.edu] (not 4GL, but many other languages/file formats) for use as a starting point.
  • Wow, I've seen this type of insanity myself, managers reading too many tech magazines with lots of sun, oracle, and bea advertisements, really you need to just re-write the whole app, its much faster than trying to convert line by line, trying to convert a procedural program into a complex object oriented environment is one of those horrible ideas that makes one wonder just how much crack the managers are smoking this week.

If you want to put yourself on the map, publish your own map.

Working...