4GL to J2EE Conversion Tools? 38
FunkyGibbin' asks: "I've just started with a new company that has outsourced a project to translate a 4GL language - some 1.5 million lines of it - into J2EE. It's way behind schedule, and keeps on tripping over itself at acceptance test points (one item is fixed, others are broken). The outsourcing company chose as their conversion tool IBM's Rexx. They don't parse the 4GL to a syntax tree and then generate from that. They 'match patterns' which then generates the output. In addition, they're not even using the object oriented features of Rexx; it's in procedural Rexx! They have no regression test suite that runs through the original system and the newly generated system to ensure that they produce the same result [this apparently was 'our job' to do it manually]. Frankly, I consider the outsourcing company to have adopted completely the wrong approach, but I'm happy to be proven wrong if compiler construction or language translation experts can tell me that Rexx was their best choice, or even a reasonable choice! Any and all information gratefully accepted - this project has so far cost many dollars, and the board is unsure what to do with it."
Easy (Score:3, Funny)
it's hard (Score:2)
Consider the fundamentals of the problem: You're attempting to translate 4GL code into 3GL code. Think about why 4GL exists, and its relationship to 3GL. It's not hard to see that you face a fairly insurmountable problem. You might find it easier to just write a new 4GL system capable of running your old 4GL system's code. The process of churning 4GL into a generic 3GL is somewhat harder (but very similar to) the job of a 4GL compiler or interpreter.
Re:it's hard (Score:2)
The only thing I have seen that comes close is certain RAD/CASE tools like Oracle Designer. Designer can absorb applications written with itself, Oracle Forms, etc, and output them as Java, and to a certain extent can absorb other applications (say, VB) and convert them to Designer. It's far from perfect, but it might get you 80% of the way there. I'd wager that IBM, Sybase et al have similar tools, but he didn't say which 4GL he was using.
I'd say that the business case for doing so needs to be very carefully examined. If the 4GL is due to be discontinued and there's no-one other than the vendor who can support it (i.e. no contractors available who know it) then maybe there is a case for conversion, maybe there is a case for rewriting, particulary if it needs substantial enhancement. If it's to be buzzword-compliant, forget it, it's a waste of time and money.
Absolutely screwed. (Score:3, Insightful)
Dave
Re:Absolutely screwed. (Score:1)
Read anything at all by Mr. Yourdon, and wade through pages and pages about how the Japanese are going to eat our lunch.
How's that workin' out for ya?
Give Up? (Score:1)
1) Hire a team to write you the converter. If it comes out good, maybe you could sell it to other companies in the same boat.
2) Give up and hire a team to start over, from scratch, and do it right this time. Will take longer, but will be easier, smaller in size when done, and cause fewer potential headaches along the way.
Hope this helps.
Conversion (Score:2)
Unfortunatly, there generally is no quick fix to doing this that I know of... Your outside contractors are going to basically have to write a language parser and then output syntax to the new language you are moving to. It's very likely to take a lot of time.
Personally, I think trying to write a system to do this is going to take longer than going in and recoding for the new language, ie go through the system module by module and figure out what each one does, rewrite it as an object and see if things work correctly, move on to next module, repeat ad nausium.
As far as using rexx, other than being slow if they are trying to write an automatic conversion utility, it's probably not a terrible choice however I would probably use perl for syntax structuring since string manipulation is perl's strong point.
As far as object oriented design goes, REXX OOP is rather clucky and is not really OOP, been a while since I used it, but I think it basically provides OOP like interface by using linked lists and such... Nor does it provide for inheritance, better to go with a more OOP oriented language than to try to do so in REXX.
What you probably need to do is to go back to the company, spend a week or two finding out what progress they have made, and determine when the project is likely to be done based on what they have done.
Then decide if that is acceptable.
Also try to decide whether the project is more than they can handle... Do they understand what is going on? From your description, it appears that they are not checking whether the project even works as the are doing it, therefore they probably are in over their heads... If so, yank the project and get someone else to look at doing it... If they are making progress and can show you that they do understand what is happening, it will probably be better to work with them to complete the project since a new development team is going to have to relearn the application before starting.
It appears that your company wanted a silver bullet solution, ie an easy fix to convert the code from one system to another. They don't really exist. The other company promised to do so, but they weren't able to make it work either. In conversion was a simple affair then all windows programs would work on linux and vice versa... The conventional path would be to hire programers to do conversion line by line as suggested above...
REXX is fine (Score:2)
Why are you so keen on their using the OO aspects of REXX? While REXX can (it depends on the implementation) full OO facilities (contrary to what one other poster said), that's not going to make it go any faster either.
Information missing (Score:1)
- Why? Does the 4GL program work? Why change it?
- Which kind of 4GL? Some 4GLs are event driven, some are 3½GL, some are database-centric, some are GUI-centric, etc.
To me it sounds like a project either heavily underestimated, or a project doomed to fail.Have to agree, wrong problem is being solved (Score:2, Informative)
There is only one way to go about this that works and everyone will like:
Gather requirements. Get input from everyone. Give a small drinking budget to programers who know someone who left (someone who left the company won't officially do work for you, but for a friend and a drink might give some insiteful rants on what is wrong, and this is invaluable in a re-design). Figgure out what was right, and what was wrong. Document how it should be, based on expirence.
Get some sort of translation between the 4gl, and the J2EE. (ip sockets work), and re-impliment whatever needs to be done. Look at exteem programing, re-factor all the code, but make it work with the old stuff so you always have something working. Re-do the stuff that is buggy, slow, or has new requirements first. (Don't be surprized if some unchanging piece never gets re-factored)
I've never done 4gl or J2EE (much less coversions one to the other), but my expirence in language conversions is everyone wants to do them not because of the limitations of the current language (even though there often are serious limitations, you can code around them), but because there is a lot of baddly designed code in the old system, and the conversion to something else is an excuse to re-write the bad parts. So get your requirements right this time, and get the design right.
Vague Description (Score:1)
I'm confused. Are they writing a 4GL-to-J2EE translator, or are they porting the 4GL compiler to J2EE?
The big decision for you and your board is whether you think the contractor is going to get you where you want to go, faster and cheaper than if you fired them and hired somebody else (like me! ;-)
It sounds like you are unhappy with their design, implementation, project management, and QA -- and that's just about everything a contractor offers.
No way this is going to work... (Score:3, Funny)
See what you really want is a Peer-To-peer B2B Distributed Cluster of High Availablity C# Tablet-PS Servers running a Micro-Kerenel Database. Preferably using Wi-Fi over Firewire. Oh yeah, don't forget that you need some of that Linux in there too. I've heard that that's good for cross-platform open-source binary-only drivers.
OOP is sooooo early 90's, that I'm supriesed your project is still being funded.
This is a business problem, not technical (Score:2)
You say:
Then get on their ass. You're undoubtedly paying these people a healthy sum of money to do this job. Tell them to get it done or else. This is the singular advantage of outsourcing work: you spend money to rid yourself of some pain. Quit worrying about HOW they do it and demand the results agreed upon in your contract.
However later you say:
So, you didn't read your own outsourcing agreement? You can't blame this vendor for being this way. Stop letting the tail wag the dog and show them who's got the money. It's a buyer's market, use it to your advantage. If they keep this up, kick these fools out and cut your losses. I know some excellent people available right now who can get this done.
ask slashdot.. (Score:2)
so the slashdot crowd must know! *sigh*
You're in BIG trouble (Score:3, Informative)
For the transition from WebObjects 4.5 to WebObjects 5.0, the developers at Apple created a converter that transforms Objective-C source code to Java source code. This was done by parsing the Objective-C source, creating a parse tree, and then running scripts against the parse tree. Simple pattern matching was found to be inadequate.
This was a much smaller project than the one that you seem to be involved in. Why? First off, as computer languages go Objective-C and Java are very, very close in terms of structure and design. Second, even with the talent involved in this project -- whom I am completely in awe of -- they only created an 80% solution. The goal was to produce something that could get a developer well down the road on the way to a conversion, not do the entire conversion.
Then, the Apple engineers tested this tool in the harshest environment possible -- their own code. It was used to convert the entire WebObjects source code from Objective-C to Java; they made it work before releasing it to customers.
Management needs to take a step back and ask, "what the heck are we trying to do here, anyway?" Unfortunately, it sounds a lot like your management has their heads 'way up in the wrong place. A useful piece that I read recently is the Happy Valley Tax Authority case study [linuxworld.com]. <HINT>Consider yourself lucky that your resume is relatively current.</HINT>
--Paul
Huh? (Score:2)
Re:Huh? (Score:2)
Dave
Start looking for another job! (Score:1)
I would suggest checking the details of the original contract to see exactly what was included in it. My guess, however, is that the contract will be so vague that the contractors can get away with blue-murder. Ideally there should be performance clauses, milestones and details of the outputs required - if they're not there, try to see what out clauses you have and get out of it as soon as possible.
Unfortunately management can be very reluctant to pull the plug on a project. My previous company would not accept that they ever failed in any software development projects, despite running 200% over schedule and 500% over budget in one disastrous case! (The reason I left BTW.) It would definitely be worth while to at least put a freeze on the project and perform an audit of the progress to date - get an external party to perform this audit, and make it clear to all involved that the company doing the audit will not be eligible to picking up the the contract if you decide to ditch the original contractors. Be prepared to pay for GOOD quality auditors too!
It would also be worthwhile reassessing the need to convert the code. Is it really necessary or could you just modify/fix any broken bits? If you really DO need to move to a different language/platform, are you sure you have made the right choice with J2EE, and should you be converting or re-writing? These can be very difficult issues to answer in many cases - it is sometimes hard to determine where the trade-off point is between maintaining the existing code-base, converting to a new platform and re-writing from scratch...
I would suggest that you look at books such as Death March, and even The Mythical Man Month, as they can give an insight on how to play the politics required in such situations...
Whatever you decide to do, keep an eye out for a better job, but be prepared to come across similar situations... 8^)
Waste of Money (Score:2)
There's not a snowball's chance in hell that pattern-matching is going to do the conversion. REXX scripts are only going to futz the syntax and that's not nearly enough. As a bare minimum the conversion tool will need to understand the semantics, and as you correctly surmise this is going to require at least a 4GL parser. If your company is lucky they'll get 10% of the way with the REXX scripts - the easy 10%.
Afterall, porting is more than making it compile! You have to make it work too. That the contractors don't even have regression tests proves that they are inexperienced, ignorant, or con artists.
My advice is to bail ship now: the company is doomed because the managers don't know what they're doing. Even the choice of J2EE makes it look like they're playing buzzword of the month. They've sunk lots of money into a half-assed attempt at a port instead of just biting the bullet and doing it properly. And the outsourcing company is taking them for a ride: whether this is fradulence or incompetence doesn't matter. Either way you don't want to be anywhere near this stinky shit.
Expect failure! (Score:2)
And of course, the politics in the situation is a real problem. You obviously have the wrong contract, if they can keep going on without even rudimentary simple regression tests. And if you hired them to make a converter instead of converting your code, you are also doomed (unless you can really afford to pay up until their converter is finished, which it will never be).
First of all, you should stop paying them. You can also consider suing them for incompetence (although that is unfortunately not possible in most countries, so you should consult your lawiers and try to find a contractual break instead).
Then you should consider your options.
Personally, I would recommend option 4. If you use option 8, it can be combined with option 5 and/or 6 in the initial stages, you might be happy with the result. I would believe option 7 would be easier than option 2 or 3, and a partial solution would also be easier to maintain. But I would prefer both 3 and 1 to option 2.
But anyway, this is a high-risk project, and with high-risk projects, you don't put all your eggs in one basket. Whatever you do, make sure that you maintain and keep using the original implementation, untill you have something you are happy with (or at least think you can see the light at the end of the tunnel). Try different options to see what is likely to work before you put a lot of money into it. And if you already spend a lot in something that is likely to fail, stop doing that untill you are sure you want to pursue this direction at the cost of abandoning the others.
Re:Expect failure! (Score:2)
Several posters question the point about regression testing.
Unfortunatly the contractor CAN'T make a regression test suit! So its perfectly logical that he deffers that to their customer.
The original code alone usualy provids not enough informations to craft a test suit.
BUT: if the customer allready had one for the old code, the converter could use that to craft a minimum set of regression tests of course!
The question is here: who pays it and on what was agreed on?
(( I did some 600k LOC Y2K reengineering, we guranteed to find all Y2K problems, and we did, but testing after the conversion was the clients job. The reason is: the environment in which the converted code is running is the clients environment, not our test environment. We had about 20 sample cases in our whitepapers where we showed under which conditions our regression tests would be successfull but a client could have a case which would break them))
In general the best approach is to have a true compiler and to generate a "data base" with the model of the old code. OF EVERYTHING.
Creating such a data base could be done with simple REXX scripts, of course also with PERL and similar approaches. As you only need to find all tokens of the language and you only record their position in the source file. The compiling step or semantical analysis before the transformation is then done in the database.
*IF* simple REXX scripts are enough, that depends on their approach: do they generate a reporitory of all source codes to be converted? Or do they work file by file (or class by class, or what ever the smalest 4GL unit is) and directly try to put out Java in a single pass?
Working file by file sounds not good!!
How does the output look like? Do they try to craft good looking Java code? I mean: would the Java output look as if a programmer had done it from the same spec from wich the 4GL code was crafted? That would be the case if a simple 4GL/switch or 4GL/if is translated to a Java/switch or Java/if.
That is HARD, if not impossible to automate. Something like that can only be done half automated. A typical way is to have a work bench collecting all transformations in a data base and asking for each transformation step how to do it. (Probably with a set of proposals for the engineer to pick from).
A logical but not necessaryly the best way to transform such a program is to have for each language construct in the 4GL language a class in Java.
If that is the case a easy pattern matching approach can do 95%. BUT: you need for all language construcs a Java class!! If you have a if (CONDITION) in the 4GL you would need a class IF and a class CONDITION in Java. And a REXX script for finding the points where to use the appropriated classes.
Bottom line:
Yes, the approach (as the asker has described it) can work. BUT: it also is easy possible that your contractor has no clue. Obviously you have a communications problem. You do not trust him and he does not give you a reason to trust him. The contractor is over budget and oer time. Thats a BAD sign.
About the technical approach I can't say much, I sketched why a "simple approach" can work and also why it is in general not adviseable to use a to primitive approach. Regression testing should be a joined efford. If they would use a "4GL to Java" compiler you would not need a regression test (similar as you do not need it if you simply compile C code to a binary). BUT: as for every "porting project" you need a test suit of the original WHOLE code and run that on the new target. That is best done by the code owner.
The other suggestions, using wrapping and reusing the old code surely has technical reasons why you did not choose them?
My advice: getting soon into touch with the responsible technical persons on the contractors side. If they can not show progress and solve your bad feelings they are the wrong ones.
Basicly: they should be able to show you in simple not to technical words: WHY THEIR APPROACH WORKS.
If they provide only hot steam
angel'o'sphere
Re:Expect failure! (Score:2)
And the reason is likely that somebody heard that Java is what everybody is using these days. If you haven't worked for a large corporation, you cannot believe the amount of money that gets spent on opinions based on zero actual knowledge. I've seen hundreded of thousands of dollars poured down sinkholes obvious to anybody on this site, and I've talked to people who have seen tens of millions vanish in similar ways. Dilbert is, if anything, optimistic.
Unfortunat[e]ly the contractor CAN'T make a regression test suit[e]!
Then they shouldn't even be trying to do the conversion. How else will they know when they are doing it right? Short answer: they won't. They took a contract like this only because they were fools or greedy.
As you say, about the best they will be able to do is some sort of 4GL->Java compiler, that treats Java as an object code format. But that is guaranteed to be about as maintainable as patching binaries. As Martin Fowler says, To have this code be maintainable, a human who understands the purpose of the code will have to do much of the porting.
Personally, I'd favor the wrap-and-replace approach. Build an external Java API that is a good OO version of what the app is supposed to do. Then write a metric shitload of glue to the old code, writing unit tests and regression tests as you go. Then just start scraping out the glue and putting real code in place.
Re:Expect failure! (Score:2)
I wrote: (without the [e]s
Unfortunat[e]ly the contractor CAN'T make a regression test suit[e]!
You wrote:
Then they shouldn't even be trying to do the conversion. How else will they know when they are doing it right? Short answer: they won't. They took a contract like this only because they were fools or greedy.
But the customer asked for that.
I worked around y2k for a company which had specialized into "porting of code".
As you say, about the best they will be able to do is some sort of 4GL->Java compiler, that treats Java as an object code format. But that is guaranteed to be about as maintainable as patching binaries. As Martin Fowler says,
You are right. I had made the way how it is done clear upfront with the contracting company. But the core question was: "Is REXX sufficient?" I liked to point out: yes, it can be the case.
Personally, I'd favor the wrap-and-replace approach. Build an external Java API that is a good OO version of what the app is supposed to do. Then write a metric shitload of glue to the old code, writing unit tests and regression tests as you go. Then just start scraping out the glue and putting real code in place.
I would do the same. But the asker probably can't do that? DB or hardware on wich the 4GL stuff is running is no longer available (in house)? the 4GL stuff can't link to C and is because of that not wrappable... no idea.
angel'o'sphere
Re:Expect failure! (Score:2)
Hmmm... On the one hand, the customer is, theoretically, always right. But on the other, as a professional I feel an obligation to not take a customer's money and give them garbage, even if that's what they ask for. I've turned down a number of contracts because I felt the client was clue-deficient and unwilling to listen. Sometimes that's enough to wake the clients up, but often they're quite determined to go off and waste a lotta dough.
When one does tech support, one learns to hear the real question behind the stated question, the real need behind the service requested. I think devlopers should do the same.
the 4GL stuff can't link to C and is because of that not wrappable... no idea.
Yeah, I wish the poster had told us more about what he was up to. I presume this is mainly a text/batch application, in which case even if there's no official way to link you can write a lot of screen scraping and file examination code. I've done this before and had it work adequately.
Aubit? (Score:1)
check out: http://aubit4gl.sourceforge.net/
Reuse instead of conversion (Score:2)
Write a cross-compiler? (Score:1)
sounds like your managers are on crack (Score:1)