Hope For Multi-Language Programming? 371
chthonicdaemon writes "I have been using Linux as my primary environment for more than ten years. In this time, I have absorbed all the lore surrounding the Unix Way — small programs doing one thing well, communicating via text and all that. I have found the command line a productive environment for doing many of the things I often do, and I find myself writing lots of small scripts that do one thing, then piping them together to do other things. While I was spending the time learning grep, sed, awk, python and many other more esoteric languages, the world moved on to application-based programming, where the paradigm seems to be to add features to one program written in one language. I have traditionally associated this with Windows or MacOS, but it is happening with Linux as well. Environments have little or no support for multi-language projects — you choose a language, open a project and get it done. Recent trends in more targeted build environments like cmake or ant are understandably focusing on automatic dependency generation and cross-platform support, unfortunately making it more difficult to grow a custom build process for a multi-language project organically. All this is a bit painful for me, as I know how much is gained by using a targeted language for a particular problem. Now the question: Should I suck it up and learn to do all my programming in C++/Java/(insert other well-supported, popular language here) and unlearn ten years of philosophy, or is there hope for the multi-language development process?"
Languages (Score:1, Insightful)
Programming Languages are just tools. And, surprise, languages of the same type are equivalent.
Go study the fundamentals of CS.
The rest will follow.
StupidPeopleTrick (Score:5, Insightful)
Python (Score:5, Insightful)
You need to go no further.
Python gives you the Rosetta stone for a project combining any other languages you'd like.
It is a very high level development language, and does have a vast common library, able to "talk" tens of protocols, you can call directly any module compiled into a dynamic library with the CTypes module. .net? Ditto - there is ironpython! .CPP? Use libboost or ctyypes to interface python with the .cpp, and soem xmlrpc to talk with a module in the JVM (oh, it would take you 10, perhaps 12 lines of code to write two methods in python which use the standard library to talk back and forth between both running enviroments.
Also, if your application or parts of it run in the Java VM, no problem, python is there in the form of "jython", enabling you to use this dynamic, multi paradigm and interactive language directly from inside the JVM, with all its standard library, plus full access to any java classes you have in there. Oh..you do you use
Ah, you need to exchange data from parents of your app in the jvm with native code in
Plus, connectivity with the automation interface of hundreds of other software - including OpenOffice, GIMP, Inkscape, all of KDE software through DCOP (kde 3) and DBUS (KDE 4), easy communication to any windows software which does havea COMM interface - and, it even works under GNU/other unixes - just run your windows app and win32 python under the wine emulator (the apps "think" they are ont eh windoews, but sockets and network ports are on localhost across windows and native apps)
Anyway -- too much to try to write in such a shrot space. It obviously have all you are askign for and certainly goes beyond that.
And, pronably you don't know Python yourself , or you would not need to ask such a question - souyou might have the impression itr is a "script" language jsut like some dirty linear scripting tools around one have to sweat a lot of hacks to insert a "for" and a "if" statement. Not such. It is multi paradigm and de-bureaucratized, but it supports a full OO model, written in from scratch, not shoehorned in a later stage of the language like happened with PHP or Perl. Everything in Python is an object. Even integer numbers, and it can give you more flexibility with your classes and objects with features such as meta-classes, computed properties, and such than the majority of OO languages out there.
And before one says "ruby", just a thing: "readability counts"!
You can (Score:4, Insightful)
Do both. (Score:5, Insightful)
You already know one philosophy. You don't "unlearn" it by learning another. You just learn more. Gain experience!
Re:Languages (Score:1, Insightful)
Yes, it may be true that languages are essentially equivalent, but that doesn't mean that some languages don't make some tasks easier for the programmer than the other.
If you had a task that had serious text processing, surely you would use C++ over Haskell or Lisp. On the other hand, if you needed to do AI, you would use Forth instead of Pascal. Graphics means using a well supported library, so Perl/Tk is the better choice than Java/Swing.
So while programming languages may be equivalent at some theoretical baseline, they are designed to address specific issues and often provide better support for some tasks and worse support for others. Using a combination of languages is a very good idea, but it also takes a lot of discipline to keep interfaces well defined.
Re:Yes (Score:4, Insightful)
meh...
In my 23+ years of professional development, administrative and engineering roles, I've found that you stick to what you're best at, and learn new things in between...
Unless the command prompt disappears, there will always be shell-scripting and that strung together happiness of well written, single-purpose shell scripts doing their jobs together as though a well orchestrated symphony.
stay with the Unix philosophy (Score:3, Insightful)
I'm not sure why you say that there are no multi-language environments. Emacs is a great example of such an environment (and not the only one). There are modes which let you program in just about any language in separate buffers, all at the same time.
The only valid argument I can think of for choosing a single language and sticking with it for everything is that your collaborators may only know how to program in the one language. The big danger for you in that case is obsolescence in a few years, when the world decides that your language is no longer desirable.
Yep (Score:5, Insightful)
Becomes a real problem if the multi-language thing is more or less "You use what you like." So you get one dev who likes some obscure language and uses it for his little part. Everything works well so nobody cares. Then he leaves for a different job some day. Time comes around to update his part... and nobody has any idea how to. None of the other staff are skilled in that language. Now you are in for a world of hurt while that gets sorted out.
Multiple languages for their own sake isn't a good idea. Any time you choose a programming language for a project, there should be a reason. That includes the initial language, but certainly applies to any other languages also used. For example suppose you have a web package that's written in Perl. You chose it because your package deals with a lot of text processing and Perl does that well, and also because your target platform supports Perl. Ok, good choice. There are other legit choices, but that's a fine choice. Now you have a client app that goes with it. Perl doesn't work well for that, so you choose C#. You choose that because the client app needs only to run on Windows, isn't speed critical, and needs to be easy to develop a GUI for. Also a good choice.
However you then decide to add a new feature to your server side of the program. You chose Ruby because it is new and seems neat. Bad choice. The server portion is already written in Perl, why add a new language? The "Because it's neat," is a horrible reason. Will Perl not do the job? Then why add more complexity.
Just sticking in new languages at semi-random on a project will do nothing but make it much harder. If there's a real compelling reason to have multiple languages, ok then great. That certainly does happen. However just saying "Let's use more languages for fun," is just setting yourself up for hard times.
Re:Languages (Score:5, Insightful)
And where did you learn this "philosophy"??? (Score:3, Insightful)
Nothing has changed in that respect. Sure, you have more options and there is more interoperability today... and it is indeed a "philosophy" to allow such flexibility. But if you think that a hodge-podge of various scripts and languages (which is what you appear to be saying) for large projects was ever any kind of "computing standard", then you got a VERY wrong impression somewhere.
I would like to know about this "philosophy", because I have never encountered it before.
Re:Languages (Score:4, Insightful)
Graphics means using a well supported library, so Perl/Tk is the better choice than Java/Swing.
Um...I do not think "well supported library" means what you think it means. Tk is old and crufty. Swing isn't much better, but if you're doing graphics you're almost certainly doing it with SDL or some other accelerated system, and you wouldn't use Swing for that either.
Tk is not as old and crufty as you think. It has been updated allot recently
Java/Swing actually has a very fast fully accelerated OpenGL drawing pipeline that is even supported on Linux. And yes many graphics heavy applications are buildt using Java/Swing The problem with swing is that it takes some heavy study time to learn how to do it correctly and not make your interface suck. But done correctly Java can keep up with most things out there. It's the done correctly part that is hard.
Are you a programmer or not? (Score:4, Insightful)
Learn new things.
The more stuff in your toolbox, the faster you can solve problems. Yes, I know you can solve a lot of the problems you face with the tools you have. It's true you can solve any realistic problem in any Turing complete language. But would you really want to write a pinball simulation in Cobol?
Learn more languages, and you'll develop a feel for when they're appropriate. I've been known to spend a day looking at different languages before starting a project. It saves time later on.
Using one language (Score:3, Insightful)
Re:It's a big umbrella (Score:2, Insightful)
"Should I suck it up and learn to do all my programming in C++/Java/(insert other well-supported, popular language here) and unlearn ten years of philosophy, or is there hope for the multi-language development process?"
No, DON'T unlearn ten years of philosophy. DO learn to do programming in C++/Java/(insert other, imperative programming language here).
Simply by learning a new language, you should not lose abilities and ideals you gained with others (goes for scripting too). You should build on your previous experiences. Experience in functional programming languages (and scripting) goes a long way for your ability to efficiently and effectively use an imperative language. This is why a lot of colleges and universities are starting to teach Scheme as an introductory language instead of Java.
There is hope for a multi-language development process, just make sure that your capable with the languages that are more prevalent
Brittle glue code (Score:5, Insightful)
In my experience, using multiple languages in a project will force you to write sizeable amounts of glue code. These processes will have to communicate one way or another and they all have their particular way to do so. So, your glue code is often not that simple but deals with interprocess communication for which a protocol will have to be devised (could be simple, but nevertheless).
Now if that was all, then so be it: write glue code. However, I found most programmers do not heavily check for errors in their normal code, where you have things like exceptions and return values at your disposal.
This becomes much worse when doing interprocess communication. The normal language constructs aren't there, you're communicating using a self-defined protocol. Often this is invented on the spot and does not include a proper way to catch errors and deal correctly with them.
So in my opinion: don't shy away from using multiple languages, but remember that you need a decent amount of time for what could be quite complicated glue code.
Re:Python (Score:4, Insightful)
I'd agree with this completely. I've worked on a few large multi-language projects, and had to combine flavours of asm, C, C++, perl, prolog, bash, maple, haskell, java and a few others together to get what I want. After years of working this way, and stumbling over the questions posed in the article (integrating build processes, dev environment etc) I had my arm twisted into using Python. What a joy it has turned out to be.
As someone with a background in functional programming (most of my PhD was implemented in Haskell) the "functional" programming support in Python is a bit barren - but list comprehensions and bound methods are powerful enough for most tasks. The shell integration is nice, and the regexps are simple enough that I hardly touch perl these days (thank god...). Sage replicates most of the functionality that I would use maple or magma for.
A sibling (AC) poster complains that Python is not "the one true language" (tm) that the parent asserts it is. Well the parent makes no such assertion. I would assert that if you want a single True Language then Python would be a good candidate for you. But if you like multi-language programming then Python is good glue. Personally I like it as glue, shockingly other people may have different uses for it. Lastly, a general word on the whole monolithic application vs small tools in multiple languages. This isn't a new debate that has sprung up in the past decade. It has always been there. There isn't even really a debate, because the two approaches are building tools for disjoint sets of users.
If you eat your own dog-food, it's vital that you have the quickest turnaround time possible on adding new features and trying out new ideas. In a sense you are continually prototyping, never finishing. You are your own end-user, although occasionally other people may pretend. As a result your target audience is highly literate in the languages that you are using, and wants to maximise the power of the tools with a programmatic interface. The unix-philosophy was constructed to service this case.
If your target user is a non-programmer, who simply wants to be able to perform a task then you will need to wrap up the functionality in a non-programmatic way. GUIs are the best way (so far) of doing this. Because there is no intention of exposing an external call interface the overhead of mapping from language to language becomes pure overhead and there is a natural tendency towards a single monolithic language approach.
The interesting gap between these two areas is when your target audience spans the non-technical and the technical. This is where monolithic applications with APIs and scripting interfaces appear. I've got to say that AppleScript is quite cute in this area, purely because it's pervasive enough that it reminds me of Arexx all those years ago...
The Unix Way was never good at everything (Score:4, Insightful)
The Unix Way is a perfectly valid method to develop administrative, text-based tasks targeting a single, well-known platform, but does not scale well toward the development of other types of applications.
First, compared to modern languages like Python or Java, shell scripting sucks. The syntax is awkward and it can only manipulate bits of text. The world has moved on from text. Today, I want to be able to process complex structures, which in many cases cannot be converted to a simple text format.
Second, modern languages have huge libraries, so usually there is no need to use anything but those libraries. Furthermore, using those libraries reduced compatibilities issues. When I develop for the Java 6 platform, I know the code is going to work on every single platform with support for Java 6: Windows, Linux, Solaris, you name it. With the Unix Way, you have to make sure that every single function of every single tool you use is going to behave in the same way on every single platform. This is of course a huge pain in the ass.
But there is no need to fret. You quote Python: from my point of view, it is one of the platforms that can be used exclusively, so your experience is perfectly valuable. Regexp are pretty much the same everywhere.
But I do not really understand your problem. If you're developing applications, you should know all that already. If your software development experience is limited to administering systems, shell scripting is always going to work for you. I guess that in this last case, you may want to try to pick a singe platform (say, Python) for all your dev needs and see how it goes.
Re:Languages (Score:2, Insightful)
Yes, .NET may actually be the best bet (Score:4, Insightful)
What good are multiple languages if all inter-language integration have to be funneled through some narrow C-like basic model?
The problem is not with the languages but rather with the platform on which they are based. As long as the common denominator is C memory blocks or opaque text-serialized objects, integration is always going to be a pain.
Sometimes language pairs are supported by an asymmetric relationship (one of the languages often being C). Using objects/features from one language is usually easier in one direction than in the other. Often there are severe restrictions on the layout of objects/functions. Examples of this is how PHP can use some C based functions/objects, provided they adhere to the "PHP way" of laying out objects. Using PHP functions from C is considerably more involved and certainly does not qualify as minimalistic.
The problem is that these bindings are often just pairs, excluding other languages from richer integration. Sure, you can integrate other languages with C as well, but rarely will the object layout of one language align with the layout of another language. This makes multi-language integration a pain. Going (down) to C level loses a lot of meta information and complicates going up the abstraction chain to another language. Essentially this is a hub-and-spoke problem with the hub being too weak. Another example is Java/JavaFX. It's quite easy to use existing Java classes from JavaFX, but going in the other direction requires a number of ceremonies. And this is still just a pair. Even though the Java platform has been used to implement a rich set of languages, the basic problem is still there: The "hub" (Java) is too weak to describe objects/functions on a the level which would let you pass objects from Jython to JRuby to Groovy to JavaFX to Scala. Java's generics are incompatible with the generics used in Scala. Java (and also the VM) does not support unsigned integers.
Java was always designed to be an abstraction over the underlying platform not to integrate with it. This philosophical difference was actually at the center of Microsofts dispute with Sun over the direction of Java. Microsoft wanted the language to evolve in the direction of facilitating integration with the platforms (one in particular) while Sun wanted to abstract away the world outside Java and have everything remodeled in Java. Microsoft enhanced their Java implementation with the "lacking" features, Sun sued and won and Microsoft completely left Java and went on to design .NET.
Your example illustrates that at least in the area of multiple programming languages, the .NET platform is much more evolved. Unlike Java, Microsoft designed .NET as a multi-language platform from the get-go (although initially for multiple statically typed languages).
The .NET Common Type System (CTS) and common language runtime (CLR) were always (and still is) more advanced and feature rich than the primary driving language, C#. The CTS already supports generics variance (C# and VB don't yet) as part of the platform. (aside: Scala generics while being define-site with variance are merely an internal Scala thing and not compatible with anything else; Java generics based on type erasure are merely a compile-time only and as such doesn't exist for anything else).
The latest development in .NET is to incorporate support for dynamic languages into .NET. C# 4.0 and the CTS will lay out a common principle (a "dynamic" interface) which theoretically can accommodate most known static languages, hybrid languages and dynamic languages on a very high level.
This will enable any language which adhere to this specification to participate in a high fidelity hub-and-spoke architecture. You will essentially be able to pass a IronRuby object to IronPython, C#, VB, F#, Eiffel etc. and interact with it there while still relying on the Ruby
Re:One time..... (Score:5, Insightful)
And the main problem I have found with young guys is they have absolutely no appreciation for talent and can't stand it when they are in a position to hire because quite frequently it eclipses their own by an order of magnitude (that's a gross generalization, of course, and not always true). But the by far the biggest problem I have found to be more and more true these days is that some kid manages to get himself hired because he has a degree in java, manages to keep his nose clean and then gets to hire more talent as the project gets bigger and in most every case they turn down anyone with a vastly superior skill set that was born more then 3 years before they were because they prefer a quiet well lit office that doesn't stink of BO and stale junk food. I might be wrong, but somehow I doubt it very much.
Now you are probably thinking, "Must be some old fuck" and you would be right I am part of the over 50 crowd who has a very current skill set and has 30 years of programming experience to back it up. I am a consultant and I am frequently hired by CEO's and CFO's who are at their whits end with the "kids" that got hired by the other kids that got the job then decided the lights were brighter and more sparkley someplace else after talking management into using the latest and greatest hair brained library written in a language that uses tabs for scoping.
Mostly what I am is a janitor, a very very well paid one, but a janitor never the less, cleaning up messes left by kids.
Don't take offense because one of these days you (if you are any good) will be doing the same thing in 30 years and why? Not because you "need" the money but because you, like me, will hopefully have some ethics about our craft.
Re:Languages (Score:3, Insightful)
What you describe about Perl is quite correct. And for many odd-ball functions and forms of processing, someone has already published a CPAN module for it, and in many (not all!) cases it's good enough.
But the level you mention, almost _everything_ is written in C. Many software packages that are allegedly in C++ could be switched to ANSI C in a single afternoon with s competent programmer, and in fact gcc's quiet compilation of both without error has led to a lot of people writing mostly C when they think they're writing C++. And anyone who thinks Java, Ruby, or Python have enough power to write themselves has not looked carefully at them.
Re:One time..... (Score:2, Insightful)
There's an important distinction between being highly skilled and being inflexible, bull-headed, or argumentative. If someone lacks good judgment in how much arguing a particular issue is worth before letting it go, or if they lack the skills to convince other people of their views - much less recognize when their own views are wrong, then what other expertise they have as an engineer becomes much less worthwhile to most employers.
Re:One time..... (Score:4, Insightful)
using the latest and greatest hair brained library written in a language that uses tabs for scoping.
I'm with you for all but this. And, BTW, I'm 40, with 20 years of experience. I was skeptical about Python, but I've recently started working on a significant project written in it, and I think it's a fine language. It does require a decent editor to keep the indentation clean (and the recommended practice, BTW, is not to use tabs at all for indentation, only spaces), but assuming you don't abuse the dynamic nature of the language, Python code tends to be some of the clearest and most readable of any language I've used.
Python is a fine language (and not a particularly new one, being roughly contemporary with Java). Like any sufficiently-powerful language, inexperienced programmers -- and especially very *clever* inexperienced programmers -- can make a huge mess.
I'm sure that's little consolation when faced by such a mess, but (and I'm sure you're fully aware of this), the problem is the messmakers, not the toolset.
Re:Languages (Score:3, Insightful)
Re:One time..... (Score:3, Insightful)
Like any sufficiently-powerful language, inexperienced programmers -- and especially very *clever* inexperienced programmers -- can make a huge mess.
True enough, but I've found that because Python is not yet that popular, it's still drawing mainly above-average programmers. Maybe 2% of the Python code I see is awful, versus over 90% for Perl. I think part of this is due to Perl's inherent flaws, but surely another part is due to the fact that every Joe learning to program is pointed at Perl (which can't end well, and doesn't).
MultiLanguage Programming thoughts... (Score:3, Insightful)
I have a toolbox at home that contains a pair of pliers.
These pliers have a formed handle that can be used as a screwdriver. I can pry with it, I can pound with it, I can pull small nails, grip, twist and an assortment of other jobs.
It's not particularly good at anything.
I also have a set of combination wrenches that work great for loosening nuts and bolts without buggering them. The pliers just can't do the job.
I have a hammer that can cleanly drive a nail and pull it out too. The pliers would be hard pressed to drive a finishing nail much less pull it.
Programming languages are tools.
Some are better at some tasks than others.
Recently, I have begun using Python. It is a very nice pair of pliers and the proper tool for one specific task - prototyping. It is so good at prototyping that the prototype often BECOMES the finished product.
Python code is so close to pseudocode that often the code documents itself.
Python is pervasive but not as much as C/C++/Java.
However, if I wanted absolute speed of computation, I'd make the finished product using C or C++.
If I wanted cross platform - Java is king.
Python is an excellent glue language for MultiLanguage Programming. Black box your components and call em with Python to tie it all together.
Or use another pair of pliers language that meets your needs.
What I'm getting at is:
There is a proper tool for the job.
Use the proper tool for the job.
Have more than just a pair of pliers in your toolbox.
Use the pliers if it's the right tool.
Re:Yes, .NET may actually be the best bet (Score:3, Insightful)
The latest development in .NET is to incorporate support for dynamic languages into .NET. C# 4.0 and the CTS will lay out a common principle (a "dynamic" interface) which theoretically can accommodate most known static languages, hybrid languages and dynamic languages on a very high level.
This will enable any language which adhere to this specification to participate in a high fidelity hub-and-spoke architecture. You will essentially be able to pass a IronRuby object to IronPython, C#, VB, F#, Eiffel etc. and interact with it there while still relying on the Ruby member resolution principles.
Really. So how are you going to represent classes with multiple or "mixin" inheritance in a language that only has static single inheritance? Or how are you going to map classic objects with modifiable state to stateless, side-effect-free languages like Haskell? I'd imagine that that's not going to be pretty, no matter what the underlying runtime can supposedly do, because the unpretty-ness is a function of the semantic gap between the languages themselves, not of some feature set (or lack thereof) in the runtime.
Re:Yes, .NET may actually be the best bet (Score:3, Insightful)