Do Scripters Suffer Discrimination?

TheTheologian writes "In his InfoWorld column, Chad Dickerson says 'there is a level of quiet discomfort between the "scripting" versus "programming" factions in some corporate development environments in which I have participated. In some instances, executive-level technology management has held scripting languages in disdain as not being "real" languages for day-to-day problem solving, which has discouraged highly talented scripters on staff from practicing their craft. In such an environment, scripters are relegated to the lower ranks ... ' He goes on to say that some companies will assign Java and C++ programmers tasks that take them weeks but could be done by Perl or Python programmers in a few hours. Is it true that some companies are so overcome with code bias they'd assign weeks of unnecessary work rather than give it to the scripting untouchables?"

Yes

Yes, often scripters are biased against.

No, it is not fair.

Programming is programming; solving problems is solving problems. What tool you use is just as pointless of a reason to express bigotry as the color of one's skin or one's gender is.

PHP scripting/coding/whatever

I have written [blacknova.net] a web-application (game) in PHP... and a friend who is a java snob (he feels no other language is worthwhile any more... and I have to listen to it... :P) constantly is saying thing like "well in java - that problem doesn't exist because [insert long winded arrogance]", or "loose types are a short path to hell - and that's where you're headed with PHP" and "PHP isn't a real language anyway - no one would use it at an enterprise level"...

Pointing out that Yahoo is now using it as their default language - and that Rasmus (author of PHP) actually was hired by Yahoo as a result is simply dismissed as bad judgement on their part.

It's like arguing religion or politics... :P

So I just sit back and listen to the tirade - and try not to egg him on...

Re:PHP scripting/coding/whatever

Just tell him that both PHP and Java are both interpreted languages, and thus are "morally" equivalent. Only languages compiled into assembly are worthy of being considered "real" programming. :)

It just astounds me that anyone can be snobby about Java. I mean, it's not a terrible language, but...

Re:PHP scripting/coding/whatever

It just astounds me that anyone can be snobby about Java. I mean, it's not a terrible language, but...

The problem isn't the language, or anything remotely to do with programming. The problem is that most programmers are as arrogant as all get out. They find something they like, and because they are convinced that everyone is their intellectual inferior, they need to point out the error of their ways.

Re:PHP scripting/coding/whatever

The problem is programmers that are insecure because they aren't confident in their ability to move between languages as needed. Programmers usually have their favorite tools for any given job but the ones that get really nasty are the programmers that are only comfortable with the few tools they use.

For me I'm pretty confident in my ability so I can move between any language that exists or is just invented as the job goes along (happens sometimes) so I don't especially get snotty. Python is one of my favorites but it certainly isn't perfect. I have done a lot in PHP but have grown unhappy with it for large projects. It is good for small to medium sized projects. Java is okay for programs that are going to run on servers with lots of memory and that won't be restarting the program often but is to heavy for most of the things I do. C/C++/Asm are good for low level stuff that needs to be fast but IMO should not be used for the bulk of things they get used for.

Re:PHP scripting/coding/whatever

Only languages compiled into assembly are worthy of being considered "real" programming. :)

You are too tolerate. Only ASM itself is real programming. Everything else is a joke, no matter how it gets to ASM, via compiler or JIT-compiler.

Re:PHP scripting/coding/whatever

Ha ha! You get to use your hands? I have to use my tongue!

Re:PHP scripting/coding/whatever

Real coders use this [rulez.org] as their keyboard ;)

Re:PHP scripting/coding/whatever

Yes, very true! Take your try at any nontrivial perl-program. Rewrite it in C. Now, pick your alternative: maintainability, speed

Hmmm, the choice is rarely as simple as that. Perl is not inherently more maintainable than C. As far as picking a non-trivial program... project size is usually directly proportional to maintainability in Perl. Additionally, Perl encourages poor coding practices. Note that I'm not saying that Perl forces poor coding. It simply makes it easy for immature coders to be sloppy and lazy. I've seen countless newbies come and go who wrote some god awful code that wouldn't be possible in a more structured language. In a business environment, reliability is important, and the gains made when a project manager knows that a certain class of problems are eliminated by choosing a structured language (though C is not generally the best choice for this) is invaluable.

And I would reverse your statement about a poor programmer. A poor programmer is one that doesn't know to choose the right tool

That's hardly a reversal of what I said. We're talking about execution time and you implied that I said C was always the right tool. Untrue, and if you go back through my comment history, (if the last n comments happen to contain one) you'll find me saying the same thing... the right tool for the right job.

If you feel like spending months writing in C when I can spend a few hours on a Perl script and get the same result

...
It's not hard for a below average programmer to write a Perl script in a day with a performance and reliability that would take weeks if not months to achieve if it was written by an above average programmer in C

That's bordering on FUD, and is most certainly hyperbole. Hours versus months, no way, weeks vs. a day, uh uh. A good coder who knows both languages can finish them in comparable time. I know from experience I can beat an average Perl programmer on the same task when writing the code in Scheme, C, or Java.

Perl is incredibly fast, and incredibly useful for a commonly occurring class of problems. Sometimes it's the right choice sometimes it isn't. It's a great language, and has replaced shell scripting for me personally. When I need to whip out a relatively small application I do it in Perl, for the main project it's generally C or Java.

Re:PHP scripting/coding/whatever

With no judgement on php in this post...

There's some problem that some java developer sees with php. Some of it being it's OOP implementation. A lot of enterprise stuff that follows OOAD, which a lot of developers and analysts are getting to like, relies on OOP. Strong data types make things easier for automated tools, for say, reverse engineering (by tools, not human).

Ruby on the other hand, is strongly typed and OOP Python too (I *think*). If he has no problems with those, then he's just an OOP zealot. If he still does, just find everything possibly wrong with java, like large memory footprint, slow startup.

Re:PHP scripting/coding/whatever

Erm, well, in Java, you wouldn't have most of the problems associated with PHP. That said, you'd have a whole slough of new ones.

Java is made for applications. PHP is made for dynamic web pages. I think, by default, this gives PHP an extra layer of planning that you have to do, and restricts you to a different set of options. And for your game, I think PHP was a far better choice.

Remember Barren Realms Elite? If I wasn't so lazy I'd be about half done with my PHP rip-off of it.

Re:PHP scripting/coding/whatever

what about the performance of .jsp and the like on non-massive hardware?

It depends on how well you code.

JSPs are translated to Java exactly once, so the performance is equivalent to regular Java servlets. For a good coder and using a modern VM with the run-time optimization, you can get performance reasonably close to what you can get with C.

For one of my clients I built a pretty complex dynamic site. On the generic P733 box I have handy, I can easily pull 20 megabits, even though I've spent almost no time optimizing the site.

The reason you see a lot of slow JSP sites is that Java is the language of choice for large corporate shops. On average, these outfits have many drawbacks:

• Many of them pull data from large, slow databases or mainframe systems.
• Many of them employ mediocre programmers who value job safefty above all else.
• A fondness for process means that making big changes is hard.
• A fondness for high-headcount teams makes it hard to get a coherent design and even harder to change anything later.
• Throwing money at a problem is generally favored over thinking, especially if said thinking would result in opinions that would make somebody look bad.

And that's just getting started. One of my clients had a team of twenty spend two years developing a site that was deployed on $2m in fancy hardware and software licenses. With three top-notch developers, I could have redone the whole thing in four months and run it all on maybe $10k of generic Intel hardware and free software.

Of course, they never would have let me; even if one development manager had taken the risk, the others would have done everything they could to sabotage the project, as it would have made them look bad.

Re:Yes

i even see bias within scripters (e.g., perl scripters are higher up the ranks than bourne scripters).

in a lot of cases this bias is justified: shell scripts have more portability problems as, say, the location and vendor for awk differs from system to system, or the behavior of "echo -n" changes. this carries over to, say, C vs perl as well: in most cases a C program will run faster with a lighter footprint than a perl script, so when either of those are a big concern then how you solve the problem is as important as the fact that you solved it.

i'm afraid i share the bias for this reason. i think you should pick the right tool for the job, not just do everything in perl because you're a "perl guy" (or a "C++ guy," for that matter). sometimes that means spending weeks writing a program in C that you could do in a few days with perl.

Re:Yes

Amen to this. I'm sick of hearing how people can do anything in "insert-language-here". Well, sure, but just because you CAN doesn't mean you SHOULD. I think a lot of this has to do with the maturity of the programmer, and they're willingness to learn new things.

...like they say, when all you have is a hammer, everything looks like a nail.

Re:Yes

Don't look at it as *their* unwillingness to learn new things, but rather as their maturity to recognize that a current tool is "good enough" to get the job done. When it comes down to it, is the (performace gain/footprint/portability/insert favorite reason here) a justification for the increased cost? If it is, then by all means, kick the programmer in the butt and make him learn a new language. If not, save everyone some grief and just get the job done.

Sure, but

I wholeheartedly practice discrimination on scripters, and in scripting languages in general.

But ya wanna know what? I *love* scripting. When I'm building a large scale application, I'll often embed a scripting engine, just so that (a) I can modify smaller bits of functionality, without a recompile, and (b) others can do the same.

Heck, often I'll write unit tests in a scripting language, so that I can drop them into a production system in order to test, or do some ad-hoc/on the fly debugging.

Scripting is a type of programming, that to compiled-languaged developers, somehow fails to feel like programming at all. It's often viewed as some sort of "configuration language".

When it boils down to it, I don't know *why* scripting/scripters are discriminated against, it just seems that they are.

Alright, you scripters! Back of the Bus!

Re:Sure, but

John Ousterhout (the guy who did Tcl/Tk) wrote a wonderful article about scripting which changed my view about scripting languages : http://www.tcl.tk/doc/scripting.html, I don't know if this is the original one, but I think it still contains his main ideas.

His main point is that all the basic infrastructure API and so on, what he calls system programming should be done in compiled language (like C, C++, etc) and offer clean API to applications aprogrammers, because experience shows that people are much more productive in interpreted languages, and you can change much easily. Well, he has has many other points but this the most important I recall.

One of the nice way you could to this is by using SWIG : http://www.swig.org which helps you all your API from many scripting language (Perl, Python, Shell, etc)

You all are missing the biggest thing..

Scripts are by nature open source. Any program that is distributed as a script MUST be open source. Few companies are willing to spend the money on developing a program if anyone can copy it. In addition, the installation must include installing the interpretter as well as the script, no choice of a binary only distribution. Even fewer companies are willing to do installations of their programs in this.

Scripting limits the choices of the developer in terms of licensing. It can cause incompatibilities when the user needs different interpreter versions for different scripts. And god forbid a pointy haired boss accidentally edits the program in ms word and it no longer works. I can just imagine the support phone calls..

Re:You all are missing the biggest thing..

Isn't there obfuscators for perl to buy/download?? THought it was. Not that I agree with obfuscating...

I believe Perl is obfuscated by default.

On the contrary -

- many in my company believe that scripting languages are often more suitable for all applications except those where processing power or speed is absolutely critical. The added performance overhead is paltry compared to the development overhead involved in writing code to the more exacting specifications of compiled languages.

True,

but often scripts are seen as quick and dirty solutions to problems that should have been solved by the inital program. Not to mention documentation, scripting is SO free form that it often intimidates management...

Legitimate concern

I'd have to say that that's a legitimate concern.

Most programming languages are designed around keeping a codebase usable even at large sizes.

Most scripting languages are designed around letting small problems be implemented quickly.

They each have a place. Using one in the place of the other really is a bad idea.

I disagree 100%

I've done my fair share of Perl, C, C++, Java, etc. programming, and I have to call BS on your comment.

There may still be a small amount of truth to what you said, however, modern scripting languages are every bit as maintainable as C, C++, or Java. In fact, an incompetent C programmer probably is the most likely to create unmaintainable code, as scripting languages require less total code, and therefore it's easier to absorb quickly.

Most scripting languages are designed around letting small problems be implemented quickly.

True, but most scripting languages that are still widely used today have evolved beyond that.

But in any case, you're certainly correct that they each have their place.

Cheers.

Re:I disagree 100%

There may still be a small amount of truth to what you said, however, modern scripting languages are every bit as maintainable as C, C++, or Java. In fact, an incompetent C programmer probably is the most likely to create unmaintainable code, as scripting languages require less total code, and therefore it's easier to absorb quickly.

What do you mean 'maintainable'? Sure, an incompetent programmer can screw up the best languages. But the programming languages aren't designed to help incompetent programmers -- they're designed to help competent ones. I remember reading about a study done in the 80's that suggested that experienced coders wrote as many bugs as inexperienced ones -- they just found more of them before the ship date.

With that in mind, there's a hierarchy of places that bugs exhibit themselves, going from good to bad. The best bugs don't get written; the next best are caught at compile time. After that, are bugs which cause the program to crash immediately (fail-stop) and the worst are bugs that cause random, non-evident behavior much later down the road. Anything you can do to push errors up the hierarchy will make programs easier to debug and maintain. Hence strong typing languages, OO, things like that.

Sure, all decent languages have comments, functions, ways to structure the code that make it somewhat easy to read. But last time I checked (which was a while, granted) Perl didn't have strong type checking to make sure you didn't pass the wrong kind of thing to a function. You have a handful of data types that do everything; it doesn't allow you to make assumptions about what other bits of code are/aren't doing, as you can with a properly-organized strongly-typed language. That's the next step in maintainability -- partitioning the thing into littler bits and making sure they work right, and moving errors up the hierarchy to compile-time errors.

Re:I disagree 100%

I think, you're wrong. First, you mix strongly typed languages with statically typed languages. Perl is stronlgy, but dynamically typed language, while C++ is weakly, but statically typed language. But even assuming you meant statically typed languages, your reasoning is flawed. There are 3 things that are pretty much orthogonal:
a) language is OO
b) language is statically typed
c) language is 'scripting language'

Examples:

Java: a+ b+ c-
CLOS: a+ b- c-
Perl 4 (non-OO): a- b- c+
Ruby: a+ b- c+
etc.

You can have any combination of these 3, but none of them correlates directly to maintainablilty.

Re:I disagree 100%

modern scripting languages are every bit as maintainable as C, C++, or Java.

Maybe not quite true for Perl, but for Python, this is an understatement.

I've never seen code written in any low-level lanugage, much less in Java (!) that was half as readable as the equivalent code written in python

The only real disadvantage of interpreted/scripting languages is raw power. They are just a greater abstraction from pure machine code than lower level languages like C, etc., which are themselves abstractions from that machine code.

Re:Legitimate concern

"Most scripting languages are designed around letting small problems be implemented quickly."

Isn't that the core philosophy of Microsoft's Windows Update service?

I'm curious

When was the last time you saw a manager reading code?

Re:On the contrary -

The added performance overhead is paltry compared to the development overhead involved in writing code to the more exacting specifications of compiled languages

I beg to differ. Consider a program script that sits on a web server and does something reasonably complicated for each person that hits the server.

Now consider that said program runs .5 seconds faster when written in C, than when written in perl.

Now consider that the website gets 100,000 hits a week. Thats almost 14 hours a week wasted. So we're up to almost 60 hours a month. How much time did you save using perl instead of C?

Re:On the contrary -

Not every script sits on a webserver. Not every script runs 100,000 times a week. You have to distinguish between performance path and non-performance path.

Re:On the contrary -

It saves time in implementation. It would probably be quicker to roll out the perl solution than the one written in C. Also 1 hour of computer time is worth less than 1 hour of a programmer's time.

Re:On the contrary -

At 100,000 hits a week, that's one hit every 6 seconds if my math is right. At that level of performance, an extra half-second per hit isn't going to cause any problem at all. You'd be foolish to optimize out that half-second for that volume of hits -- particularly if it took an $80K/year programmer two weeks to do it.

Flip side

Is it true that some companies [osdn.com] are so overcome with script [slashdot.org] bias that they'd assign years [slashcode.com] of unnecessary work rather than give it to the coding [sun.com] untouchables?"

Re:Flip side

That's funny, except what's funnier is that I consider Java a scripting language.

If it ain't compiled into assembly language, it ain't real programming.

I personally do more programming in Perl nowadays ... I don't pretend that it's real programming though.

You have a very warped view of what "real programming" is. Compilation v interpretation has nothing to do with it. Besides virtually nothing is "compiled into assembly language" these days. As for Java - it is compiled into machine code, it's just that all the platforms it runs on emulate the target machine. And there are also plenty of Java verions that produce native executables.

Re:Flip side

There are things in Java that will NEVER allow Java to be useful as a general purpose language. The lack of an unsigned datatype is probably the most egregious flaw.

Be careful of making predictions in the computer world. It was once said that only Assembler could ever be used to make an operating system. Compilers would never be able to make code efficient enough. Then along comes these guys, Dennis Ritchie [bell-labs.com] and Ken Thompson [bell-labs.com] who proved them all wrong.

Re:Flip side

Er, no. Java is compiled into an intermediate form, just like every other scripting language such as Perl, Python, etc. Calling that "machine code" shows an ignorance of programming in real assembly language. Java bytecodes are just a numeric version of the Java language.

Wrong, wrong, wrong.

First, your precious native-code compilers compile into an intermediate language as well. No modern CPU runs a program as-is -- they all have tricks like microcode, out-of-order execution, register renaming, and other hand-waving that make the actual program run by the CPU quite different than the one sitting on your disk. I'm sure "that's different" for some reason, of course.

Second, Java bytecodes are a machine language. Admittedly, no 100% complete implementation of the machine in question exists, but I fail to see how that makes a difference. Are you saying that if I extended the picoJava CPU core to natively handle the last few instructions that are currently emulated, suddenly Java would switch from being a "scripting language" to a "real language"? That's asinine.

That's the primary reason that Java is so slow. The bytecodes cannot be efficiently interpreted.

The primary reason Java is "so slow" is that most of the people claiming that haven't used it in years. Java 1.4.1 is pretty damned fast as I see it. The other reason that Java is seen as slow is that its GUI libraries are not as fast as the native libraries. That doesn't have a thing to do with bytecodes, but rather with how they were designed.

There is nothing special about bytecodes that makes them any more difficult to run efficiently than any other programming language. In fact, they open the door to a lot of optimizations that are all-but-impossible with other languages.

There are things in Java that will NEVER allow Java to be useful as a general purpose language. The lack of an unsigned datatype is probably the most egregious flaw.

The only reason that unsigned datatypes matter one iota is in interfacing with someone else's code that does use an unsigned datatype, in which case nasty conversions must be done. If you don't need to interface with such code, you find that they are completely unnecessary. I fail to see how that is such a serious flaw.

I'm not saying "Java is the bestest language EVAR!!!", but please get your criticisms right.

Re:Flip side

Yeah. Better yet, if you don't actually hand-craft each of the electrical impulses for the CPU, in order, one by one, with a tiny hand-powered generator, you are not a real programmer.

What kind of logic is it that makes people think that invoking a compiler on their textfiles is the step that turns them into "real" programmers?

gcc file.c
woo, i'm a programmer!
perl -e file.pl
doh, i'm not a programmer.
gcc file.c
woo, i'm a programmer!
perl -e file.pl
doh, i'm not a programmer.

Re:Flip side

"If it ain't compiled into assembly language, it ain't real programming."

Interesting definition. My distinction has always been that if I can fuck up memory, then I'm programming. Otherwise I'm scripting.

--Richard

I once heard

In reference to perl vs. C that scripting is good for a quick and dirty "proof of concept"

IT's called a standard

At the very large company I work for there are standards. And if they were followed we wouldn't be in the trouble we are in now with over 16 different databases, 24 different programming languages, 8 different OS's.

The reason a company wants you to develop in Java or C++/C or whatever is to maintain the standard, do you have any idea how much money is going to have to be spent to maintain the employee knowledge to support so many different databses, OS, Languages, etc...

That's what standards address. Now the real question is what is the process to create a diviation from the standard, and is it justified?

Thats what this questino should address.

Ted

Re:IT's called a standard

No kidding!

We had an intern who wrote a bunch of stuff in Python and Ruby. He was all gung-ho on those languages and made a big deal about how they were "it". When he left, no one had the time to learn how to support these languages, so we ended up re-writing them in Perl so that everyone could support them.

FYI: his scripts sucked, too. He'd make lots of dumb mistakes like assigning a variable called "retval" and then checking "ret"!!! Duh. gcc would have caught this immediately, so would "use strict".

Don't blame the intern!

Why did you let an intern deviate from company standards??? I don't blame the guy/gal for being a beginner and thus writing "sucky" scripts in whatever language. But you guys have been so plain DUMB for letting the intern go ahead with Python and Ruby knowing full well that you couldn't support these languages. It's sometimes too easy to just blame the intern... YOU (experienced script guru familiar with company policy) should have instructed him/her (fresh out of school newbie) to use Perl and nothing else. And if that weren't an option, why did you hire this intern in the first place?

I agree, but...

...at this point, wouldn't it be a good idea to pick ONE of the scripting languages, and make it a co-standard? Sure, allowing anyone to code in language du jour isn't a great idea, but taking forever to do code simple programs because C takes forever to develop with...well, that ain't so great either.

This is why I use VB and ASP.

One language, one platform, one big piece of shit.

Nevermind I just forgot my point...

my belief

is that it stops being 'scripting' and starts being 'programming' based on the scope of the project. Processing a web form is scripting. Writing a GUI app (be it in Win32 or wxPython) is 'programming'.

Re:my belief

Sounds good, but I'd break it down more specifically: Scripting is interfacing, tying things together on a higher level. Programming is functionality, algorithms and such. This still has nothing to do with language choice, as many languages can handle both to a degree.

A better definition

Would be to say that 'scripting' is a subset of 'programming'.

Never had that problem

I've been lucky. The management at my previous job was all tech-savvy, so they knew to use the right tool for the job. The management for my current job are completely un-tech-savvy, so they don't know the difference ;)

Wrong Person, Not Language

Typically these jobs that take weeks instead of hours are assigned to the wrong people, not the wrong language. The right person should figure out the best solution for the problem and tackle the problem correctly. The wrong person will go after it in his favorite language and ignore the best way if it includes any amount of work before he begins coding.

Re:Wrong Person, Not Language

That doesn't work if The Powers that Be have decided on a solution ahead of time. If TPtB decide that you must use an in-house language that takes a few thousand lines to code what Perl can do in a few dozen, you can't use the right tool. You have to do what TPtB and the PHB have decreed.

Re:Wrong Person, Not Language

I agree. I started using wxPython for a project I'm working on in OS X. It's great, and beats the poop out of Swing, but I decided that using the native (Cocoa) toolkit, while it may take longer, will give the program the spit-and-polish I desire from the app.

Re:Wrong Person, Not Language

And the "right" language is not just a technical question. If the company only has Java, VB, and COBOL experience and the permanent staff isn't very flexible, the right language probably isn't perl, no matter what the problem is.

On the other hand I've worked in companies that could grok any language. We even made them up when we needed to.

Joe

Mountains and molehills..

Call that "discrimination" is hardly justified,
what it most likely is, is good old managerial incompetence,
perhaps with a dashing of conservatism as well.

Anyone who claims that one programming language is superior for all and any purpose is obviously incompetent to make such decisions.

Personally, I wouldn't stay long at a company like that. Unfortunately these kinds of things are very, very, common. Bosses know one way of doing things, and they want it done that way, no matter if its not a good way or not.

Re:Mountains and molehills..

I think you're over symplifying. Managers realize that the more different languages are used means that, most likely, the harder future support becomes. Instead of just giving the programmers free range in what language they should use, two or three languages should be selected that provide good coverage of various functionality, and development should be limited to those languages.

I've worked places where the developers use whatever language they want. Guess what? Every time one of the developers leaves, their stuff gets rewritten since no one else likes their choice of language. That's not good business.

The title of idiot manager should not be placed on anyone that wants to reduce the choices of the developers. Instead, it should be placed on managers that don't recognize that at least more than one language will be needed and force everyone into C++. Unfortunatly, it seems that if management makes a decision that limits the "freedom" of the developers, they are labeled idiots irregardless if their decision makes sense business-wise.

Re:Mountains and molehills.. (Python apologia)

One little piece of common sense to remember, though, is that it doesn't matter that e.g. Python would only take 10 lines and is easier to read, if there is only one person at the company who knows Python, and the other 30 developers only know C/C++/Java. You can argue that Python is easy to learn, and easy to use, and I will agree with you to the ends of the earth, but that doesn't mean that a particular individual will find it easy to learn or use.

The additional factors of training expenses and/or recruiting and hiring someone who knows the language should be taken into account when evaluating the tools used on a given project. This is a basic thing in managing a project. It is only my personal opinion that sending all 30 developers out to learn Python is the obviously correct solution, that will save the PHBs (and developers) time, money and frustration in the long run. ;)

Re:Mountains and molehills.. (Python apologia)

it doesn't matter that e.g. Python would only take 10 lines and is easier to read, if there is only one person at the company who knows Python, and the other 30 developers only know C/C++/Java.

If those 30 developers can't decipher all of 10 lines of python (or any language) it's time to get some new developers.

vs programing?

Hmm. I thought scripting *was* programming.

Scripting vs Programming

I think the only difference, generally, between the two, is nomenclature. Although scripting languages are generally interpreted, all in all, there isn't too much difference.

So the name comes up as the big deciding factor. You call yourself a scripter, you're actually limiting yourself in the eyes of those who want to see a difference between scripts and "programmed" software. I've actually found a lot of resistance among people who write in scripting languages to call themselves programmers, even when, by rights, they do the exact same tasks.

Of course, no one ever stops to question when a programmer writes in a scripting language... except maybe to say "why are you bothering with that garbage?"

Not Quite

Script kiddies, by definition, do not write their own scripts.:)

Absolutely.

There seems to be this mindset in large corporations that all "programs" have to be written in C, Java or another "compileable" language. In my job at a very large company (Caterpillar) we especially see ancient VAX-based apps or newer web applications that months are spent on, when a simple Perl script would do the same job in a matter of weeks or days.

Use BOTH!

On a project I designed [p25.com], I deliberately designed the system to have TCL built-in, for a very simple reason.

Scripting has its place, as does more conventional compiled code.

Use compiled code to do the heavy lifting - in my case, things like FFTs, signal analysis, and such.

Use scripting to tie it all together.

That way, when you are trying to figure out the problem domain ("Now, what does the radio expect me to do when it sends a GTC message - maybe it wants a CASSN message? Clicky-click - No, doesn't seem to be it. Maybe a IDN message? Yep - that's it.") you can try things out very quickly.

You can also very quickly string together smaller functions into larger blocks ("Ok, to test the radio, first I do this, then that, then the other.")

I cannot even begin to imagine how long simple things would take if we didn't have an embedded scripting language.

All in how you say it.

"I am good at scripting." == lame.
"ph34r my l337 skr197x0r sk1llz, f44g0rz." == cool.

Hardware is so fast and cheap...

..that the only time that really counts is programming time. Execution time is trivial. And this saving continues to be true over the entire lifecycle of any product. [as an assembler and C/C++ coder I will admit certain exceptions do exist in hardware dependent areas, but these are rare & getting rarer -- which is why I'm looking for work ;D

Blame the dot-com goldrush...

Hey, if all those art majors and wanna-be fashion designers hadn't decided to become "web developers", maybe someone who can write an actual program in Perl might get some respect.

Seriously, scripting languages have been "tainted" by the Web. "If it's a script, it can't possibly be worth anything" is a pretty common mind-set these days.

While I've seen some pretty awful C and C++ code out there, it's nothing compared to the horror of amateur Perl or (shudder) Shell scripts.

It's interesting to consider that scripting languages have been able to ride Moore's Law to the extent that you can reasonably implement things in a scripting language hat would have really needed to be compiled a short time ago.

-Mark

Re:Blame the dot-com goldrush...

"It's interesting to consider that scripting languages have been able to ride Moore's Law to the extent that you can reasonably implement things in a scripting language hat would have really needed to be compiled a short time ago."

No kidding. Example: Apache webserver: 700 pages/sec on Dell Poweredge 1300 PIII 550MHz (in 2001). Perl webserver: 700 pages/sec on my beige box Athlon XP 2000+. Kind of funny to do pattern substitutions on the fly on webpages ;).

At my prev company, due to a request from my boss to filter out various sorts of email, I configured some dot qmail stuff to call a perl program. Sure C could have been faster, but while with Perl I was introducing a performance hit, I could be pretty certain I was not introducing a security problem - no risk of buffer overflows, and if memory usage gets too high, ulimits kill the process. The code was short and simple - return different exit codes depending on what sort of patterns matched.

Nobody noticed any performance slow downs (the final windows based mailserver was usually the problem ;) ). Boss happy. And it was easy for me make my own custom filter to bounce off a fair amount of junk/spam from my own work account :).

For fun I recent wrote pop3, smtp and plug proxies in perl. AFAIK stuff like this would be fine for small orgs. By the time most small orgs double in size, I'm sure PC hardware would have doubled in power or more, and they would need to replace aging hardware anyway. They probably won't even need to upgrade for the first few doublings.

A rose by any other name

Scripter, programmer what's the difference? The thought process is the same whether you are using cshell, java, assembler or any other programming tool. This is like saying that speaking another language will make a difference in mathematics.

Certainly

I don't know about 'weeks', but there's little doubt in my mind that tasks are often assigned to C or other 'proper' languages that could more easily be tackled with a so-called scripting language. Whether this comes down to 'prejudice' or mere ignorance to the potential of perl and the like is open to question.

And, without wishing to develop too much of a flamewar, this same issue comes up -- more frequently, even -- with the battle between 'traditional' web development languages that use CGI -- notably perl and C -- and more modern languages like PHP, ASP, etc. It's my view that a truly experienced and effective developer, whatever the particular circumstances or decisons to be made, will be sufficiently open-minded to consider multiple alternatives: those who show a propensity for platform elitism, or for discounting certain solutions out of hand, often seem to prove poor developers - for the very reason that they show a lack of imagination, an unwillingness to consider different options, and so forth.

Also, people often only consider one side of the equation -- and it's the least important side: the particular language used often has vastly less impact upon the success of a development than does the ability of the developer to write clean code, to think in a sensible fashion -- and to get a *full* picture of what's going on. Take Slashdot -- perl-driven, perhaps, and working reasonably well in its way -- but betraying a lack of understanding of modern web development techniques such as the use of XHTML/CSS in place of kludgy tables and the like.

Long story short: the language won't make the difference, and the developer or manager who thinks it will is deluded -- and will pay for it in the long term.

Re:Certainly not

CSS is in a wierd niche - unneeded for simple pages, and too weak to do what Flash can do. Most of what CSS is usually used for can be done on the authoring side, with Dreamweaver templates or something similar. CSS also interacts badly with firewalls and proxy servers that edit out hostile content. If you really need exciting animated graphical effects (and you usually don't), Flash has far better capabilities.

Wow, such a complete misunderstanding of CSS... CSS is intended to separate content from presentation. That's it. It has nothing to do with Flash or "exciting animated graphical effects".

It's unfortunate that CSS is so misunderstood, as it is really a quite elegant model for web presentation.

• http://developer.apple.com/internet/css/introcssla yout.html [apple.com]
• http://www.alistapart.com/index.html [alistapart.com]
• http://www.meyerweb.com/eric/css/edge/ [meyerweb.com]

Re:PSST

According to your definition, assembly is also scripting, because the mnemonics have to be translated by an assembler before your code will run.

C is not a scripting language, because the end result, after compiling and linking, is an executable that can be run by the OS w/o a separate runtime (I'm including linked-in runtimes, such as the old dbase runtime kit, as 'separate', b/c the end result still goes thru the run-time interpreter).

Oh, and assembler is not 100,000 times harder to code. I actually found perl made me cross-eyed for quite a while before I grokked the mind-set behind it, and now I use it whenever I need a quick-and-dirty script to fetch some data, process it, and give me the results.

It's about stability

When you are building a software application, you try to get everything synchronized, so all programmers will be able to understand and feel confident in each other's code.

Many times programmers, in charge of maintenance, have had to search through code only to find the bug related to a script which does not follow the norm of the project.

Therefore, in a serious project, with millions invested, scripting can be a dangerous shortcut that may plague the project a year later.

My point is not that scripting is a waste of time or an unneccesary technique, since it can indeed be useful, but it is likely that an average manager's gut instinct to avoid the technique unless it is the only way to achieve something, because the more it's intermixed with C or Java code, the less standardized the project becomes.

A concept may be easier to express in Chinese, but you don't see many novels written in English with Chinese added here and there. Uniformity often leads to quality.

Re:Oh please!

Any such distinction between them is better explained along the software programmer versus system admin dimension (programmers do more programming, admins more scripting).

That's the misunderstanding that leads to problems. Scripting is programming and scripting languages can be used for software programming. I mean are you going to say that the task of building slashdot is "system administration" not "programming"?

I guess I am biased against scripters as well...

I guess I would be labelled as biased as well. Scripters often are talented, home-grown and self-taught but true enterprise systems require more enterprise-capable features and capabilities offered by RDBMSs, tranaction coordinators, asynchrnouse messaging, distributed computing, etc. I'm sure some or all of those things can be accomplished with scripts as well but vendors and products in these categories tend to API their products to programmers (Java, C++, .NET)

Also, I find scripts like Perl/PHP/ASP and other harder to maintain for larger projects. And, if the original scripter is fired/laid off how much easier is it for a new scripter to jump in and successfully maintain that code base? I think people in OOP-land work really hard to creating standards and methodologies that make code maintainable over the long haul (just attend an OOPSLA conference some time).

As far as hiring biases, it depends. I've seen people hire scripters because they can get their site up just as good or even better than a programmer. That works great in small organizations, but if you are working on products with 100+ developers then scripting becomes pretty painful, hirers of large teams would probably rather like to stick with tradidional business development tools, languages, platforms, products, etc.

Flame away...

There are no scripters.

There's no such thing as a 'scripter;' there are merely those who use just-in-time or per-execution compilers.

Does the end user know the difference?

10 Echo Starting Application
20 system "start iexplore -k http://localhost/index.php"
30 goto 10
40 profit

Re:Does the end user know the difference?

Warning: profit unreachable.

How true :)

Programmers vs amateurs?

I think it's not about compiled vs scripted languages but programmers vs amateurs. Lots of scripting languages have an easy learning curve and many people who are hired to write scripts have not really been trained as programmers. This, also why we often hear "perl code is such a pile of mess". Well, most of it seems to been written by people who learned it from tutorials and have no idea about basic concepts of software engineering, algorithms, programming styles and such. Most companies just can't afford to hire a person with a computer science degree from a respectable CS school for every job that requires coding.

my $0.02

weeks vs. hours

An author loses all credibility to me when he asserts things like "developers spend days and weeks writing Java and C++ code to solve problems that those talented Perl or Python programmers could have knocked out in a few hours", with absolutely no substantiation. I guess that with anecdotal evidence, you can prove anything.

I'd challenge anybody to come up with a problem that could be solved within a few hours in Perl or Python that couldn't be solved within 2 or 3 times that length of time (longer, but not "weeks") by a competent C or Java programmer. Certainly, there are jobs where Perl is absolutely the right tool. But I have a very hard time believing that there can be that much of a difference.

Happens all the time

A quick aside: I HATE the term "scripting", as if it were some degenerate form of "real programming" - especially with feature-rich languages like perl that never have to call other applications.

Anyway, first-hand experience: thanks to the concept of perl modules and the incredible CPAN archive, writing applications that have to go to the network for things like HTTP or (especially) LDAP are trivial in perl but seriously heavy lifting in C.

You also get string parsing, regular expressions, and garbage collection built right in. Not to mention the incredibly powerful (from a code legibility standpoint) associative array or "hash" data structure.

Believe it or not, correctly written perl is orders of magnitude more legible than C or Java, because it works at a higher level of abstraction.

I wrote an LDAP->LDAP replication program, with schema and data format translation, in a couple of hours using perl.

Doing stuff like comparing the contents of a database dump (provided as a CSV) against an LDAP directory is trivial in perl.

C is best used when you won't have a perl environment availible and need the binary to stand alone. For pretty much every other task I've encountered in the last 6 years, perl got the job done faster and with much better maintainability.

DG

Re:weeks vs. hours

I'd challenge anybody to come up with a problem that could be solved within a few hours in Perl or Python that couldn't be solved within 2 or 3 times that length of time (longer, but not "weeks") by a competent C or Java programmer.

I want a program that

• recursively reads in a bunch of web pages.
• validates the HTML [cpan.org] for those web pages
• if the HTML is valid, parse the HTML [cpan.org] and make a list of all images on the pages
• for each image, create a Morse code audio clip [cpan.org] from the relevant ALT text.
• if the HTML is invalid, parse the HTML to determine who maintains that page. (Let's assume there's a corporate standard verbiage at the bottom of each page, which includes maintainer info.) Lookup the maintainer name in the corporate LDAP directory [cpan.org]. Generate an interoffice mailing label (complete with barcode [cpan.org]) so you can mail a seppuku knife to the relevant webmaster.

It took me 20 minutes of browsing CPAN to come up with this (admittedly stupid) example, I'm sure I could throw in lots more freaky CPAN modules to make life harder for the C folks.

CPAN is what forced me to learn Perl. I'm sure a lot of these libraries exist for C, but it's much harder to find 'em, and who knows if they work on your platform? Let's stipulate that our program will be deployed on a DEC Alpha running WinNT...

It's all about the gory details

One of the purpose of a programming language in general is to hide gory details from the programmers, to allow them to better concentrate on the task at hand. Scripting languages excel in this!!

Otherwise we'd all be programming in Assembler..

Anyone who's ever programmed Microsoft VC++/COM/ATL/STL/MFC will attest that that particular environment does not do a very good job of hiding gory details.

I choose the language based on the task, if a scripting language is good enough and performance is not an issue, I'll be the first one to use Perl or even VBScript.

There really is a difference

There really is a difference between scripting and programming. Scripting languages tend to be heavily dependent on compiled code. Where would perl be today if all the modules had to be written in perl? Instead, getting a module from CPAN, there's a good chance you are actually getting C code and a perl wrapper.

Another difference: type safety, programming languages have more stuff being caught at compile time than in runtime, then scripting languages like perl do.

Another differene: scripting languages make the common things easier, while programming languages opt for generality and extensibility. Compare writing to a file in perl, versus Java.

There are indeed differences. But that doesn't mean one is better than the other. I remember a joke that circulated around the internet about the evolution of a programmer. In the beginning was the beginning programmer with "10 HELLO WORLD". Then came C, with #include's, a main function that printed "hello world", etc. Then C++ with a #includes, a class, a main function. Then came COM with about 5 pages of code dedicated to making a COM service that outputted "hello world". Finally, the last stage, a grand master programmer: "10 HELLO WORLD".

Scripter Discrimination....

I can vouch for this viewpoint, and I'm not even in IT, per se.

I work in industrial information management, which is often accomplished using various types of information technology. Just from exposure, I've picked up and used some scripting tools to solve problems.

The fact that I use scripting tools some of the time was held against me in a review. Remember -- I'm not even *in* IT, not required to know programming *or* scripting -- but the value of "script kiddies" has been so diminished that the management here heard I could use scripting, assumed I was another devalued dot-com remnant, and assumed my value would be much lower. I was able to explain the difference to them eventually, but at the time, I thought their zeal to devalue scripters was noteworthy....

scripting "cowboys"

There are multiple facets to why scripting is descriminated against. Some of it is justified and some is not.

For starters, the biggest myth of scripting languages is that they don't perform well. The bottom line is that there are very few applications where the overhead of the scripting language is going to outweigh the performance cost of a bad design or poorly written code.

That said, the biggest problem with scripting languages is that they are so easy to use. The tends to create a coding cowboy type environment where folks solve a problem really quickly in a script but that script is never kept in version control, or it is written in a language that noone else in the company is trained to use, or it contains hard coded entries for database passwords, or there are hundreds of scripts and it becomes a nightmare to make a change to the way things work because the scripts don't share any codebase...

Note that none of the above problems are the fault of the scripting language. They are more the fault of developers abusing them. In a sense, scripting languages leave a lot of rope for folks to hang themselves with. And because lots of folks do hang themselves with them, there is a lot of ammunition that people can use to spread FUD on scripting languages.

But perhaps most importantly, there is this goofy thing called human nature. For some reason, we silly humans are easily duped into thinking that "you get what you pay for". It's marketing/sales 101, and it happens all over the place. For example, if you see two bottles of wine, one for $2 and another for $20, odds are that most people will be convinced that the $20 bottle is a better wine, even though there is no evidence whatsoever to base that decision on.

Well, scripting languages are typically free, so the natural inclination of people is to think that they aren't as good as products for languages that sell for tens or hundreds of thousands of dollars. Unfortunately, I don't see this ever really changing, but then I've never been accused of being an optimist...

Oh Hell Yes!!!!

Absolutely this is done, and the bigger the company, the more stubborn and thinking!

I've been sitting here at my little pathetic cube banging out perl scripts in a few hours to run diagnostics and spot problems in the day to day operations of the company.

The IT monks recently approached me and informed that I was practicing sacrelidge by using Perl instead of C or Java. In order to save my soul they would have to assimulate all my work and do it in Java.

That was nine months ago. They are still working on the first 3 of 50 scripts that I've put together in about one years time.

And don't mention the following words to any of them:

• Open Source
• GPL
• Freeware
• Shareware

or they will start screaming, running around the room, and hitting themselves over the head with boards asking the IT gods for forgiveness.

Seriously, the notion of standards in todays IT industry is rather fucked up. They select one tool for every problem and go from there. Hell, if that was the case, then we would all be running Visual Basic and be happy. After all, there isn't anything VB can do that anything else can't.. right!

The Question is Discriminatory

Implied in the question is that Scripting knowledge is equivalent to Programming knowledge. My years of managing techs of various sorts has taught me that's just not true. It's not that the problems can't be solved just as well, but that generally they won't be... because of a looser design and implementation style.

Many (not all) programmers will analyze the problem, design a solution, perhaps have their design checked, implement it, and run some tests. This is based on education, experience, and the requirements for programming, but they will do this EVEN WHEN SCRIPTING.

Many (not all) scripters who aren't programmers will simply start writing the code. Lots of iterations. No formal design. Very quick to first test, but not nearly as likely to be the right solution the first time.

The typical result is that the same problem, solved by a script, will often work better and solve the problem completely sooner coming from a programmer than from a scripting non-programmer.

Typically I would expect management to be asking only scripters with exceptional track records at the same time as they ask the programmers. Because management isn't interested in which tool was used to solve the problem, or even in minimizing cost of solving the problem. Management is interested in solving it once and having the solution work. If that means using a developer who implements overkill at double the initial cost, but with much lower ongoing support and revisiting costs, that's fine.

The best

The best is if you know both scripting and programming, then you can make a sensible decision on what will do the proper job. It is all about balancing effort and needs. Does it need to be lightning fast? Does it need to work in certain environments? For how long must it work?

But regardless of what solution you choose, document. Even one-liners.

Equal Opportunity?

If you work for a college (especially one with a high number of foreign students) you might be able to seek recourse if "language" has been added to one of the non-discrimination criteria in the Equal Opportunity policy.

Just hope you get a geek judge who sees no difference between human and computer languages :-)

The right tool for the right job, and maintenence.

I code both, and I agree with the columnist, although the column was a bit lacking in useful information or original opinion (although he did give a decent analogy), so here's my take on the subject.

When I have to decide what language(s) to use in a project, there are many factors entering the decision, beyond a simple analysis of mile hike vs. Mt Everest. As he touched on, some languages have specific strengths and weaknesses. I wouldn't use java for parsing large text files unless I had other really good reasons to do so.

The only place this breaks down is maintenence. I think that, and the low entry point actually one of the big reasons scripting laguages are looked down upon. You end up with a lot of scripts in place that were poorly written by inexperienced programmers, which have gotten even worse as other programmers applied patches and bug fixes. ASP is particularly offensive in this way, as, while it is possible to write clean & readable code with it, most people will find it much easier to write nightmarish spaghetti code.

What the initial programmer expected to be a mile hike, turned out to be something much longer, as scope creep and unforseen bugs turned it into an expedition. Rather than turn back and resupply, the stubborn programmer kept going, marvelling at how clever he was to keep himself alive with only a swiss army knife. Unfortunately, this lack of sufficient tools carries over to every other trip up the mountain to fix a bug or add a feature, and clever hacks turn into brutal kluges.

There's not always a right answer, but everything has its strengths & weaknesses, and refactoring or restarting from scratch is an often overlooked option at any stage in development.

Hammers and nails

If you have a hammer, everything starts looking like a nail.

A lot of managers don't know about scripting - so they see programming as the solution to everything. Let's face it - scripting is often behind the scenes, doesn't fit well into standards at times, and isn't as flashy on the surface.

On the flipside, I've often seen scripting-heavy people not communicate the necessity of their work, whereas more coders I know seem to have developed that skill.

The result? Management doesn't know about the options, so they'll go with what they know.

Just my 2 cents at current exchange rates.

Since you ask...

Yes. *grin*

To expand on that: I have found that a lot of people fail to realize that you should first identify a PROBLEM that a static computer program can solve repeatedly. If a process is temporary and won't be used several times yet still requires a lot of processing for the whole 1 time it'll be run, then perhaps a simple script will do.

Where I worked before, the order of the day was ad-hoc reporting. Management failed to understand that a static processing unit cannot produce very different sets of output - it can handle a lot of different branches of execution, but only those that are explicitly defined. Whereas some simple scripts to get the job done for a short while would have sufficed, they demanded that all output be generated by static compiled binary programs that they could run locally on their computers (no, they wouldn't shell out for a webserver or database server until much, much later, when they hired a new project manager with clue).

Yes, it was maddening. No, I will never return to work for them under any circumstances (save a large 6-figure salary). And no, they never learned their lesson - they're busy making a giant funding model in a binary program, where the model's implementation is CONSTANTLY changing & being tweaked, having modules added & removed. The program will never get off the ground, but for the past 2 years they've been plugging away at it, desperate to come up with that holy grail of "Static process Ad-Hoc Reporting."

May God have mercy on their souls.

Law of leaky abstractions

I think a lot of this discrimination has to do with the Law of Leaky Abstractions [joelonsoftware.com]. In short, the further people get from the metal, the less likely they are able to fix any subtle problems that may arise when the abstraction breaks down. High level script languages are generally themselves just abstractions to lower level systems.

Of course, some people who specialize in scripting DO know the lower levels too, and thus the law doesn't apply to them, but many people whose jobs rely around scripting activities would be stuck if their abstractions leaked...

Yes, scripters get short shrift

A topic near and dear to my heart.

In places I've worked, the CM system (build, defect-tracking, patching, etc.) was written in scripting languages.

The people who worked on it were never really considered to be "developers", even though the systems could have benefitted from requirements analysis, design and code review and modular development practices. That had two effects: the good software engineers who were scripters got frustrated, and the crappy hackers were able to slam in crappy code that worked fine but was fragile and hard to maintain.

It's even easier to produce crap w/a scripting language than w/a compiled, statically-typed language. (Not that you can't produce crap with C/C++, don't get me wrong.) This ties in w/the preceding paragraph, but it's also a good standalone point -- w/out rigorous code review, Bad Stuff is going to accumulate more rapidly on the script side.

That might be more a reflection of people's attitudes towards the kind of work that gets done w/scripting languages (quick-n-dirty) than a reflection of attitudes toward the programmers who do the work.

Standards

Well if you can be sure that the same person who initiates the program will be around in 10 years to maintain it -- then go ahead and use 40 different scripting languages and 20 different compiled languages.

However -- in the real world (where we have to request specific skillsets for our contractors upon one person leaving and another person coming) we have been forced to standardize on a handful of languages to ensure that we could get the job of 2 people done with 2 people. Not having to employ an extra Python or Perl guru just in case those few programs that the one guy who thought Python was cool and could do it "tons" faster than Java or C need to be maintained or added to.

Sure -- we have extra up front time writing Java programs in a week when Perl could do it in a day.....But at least if we replace our "Java" resource with a "Java" resource -- we can ensure that future maintenence and enhancments to existing programs has a fighting chance.

Do Scripters Suffer Discrimination?

I'm not sure, but in my building, there are three bathrooms -- Men, Women, and Scripters.

What is the difference?

I use VB SCript in my ASP development- am I not a programmer? I thought I was. That's what I told my Mom I was. She'll be so disappointed.

That means I'll have to chnage my business cards :-(

Seriously, what is the difference? Depth of the manguage? I don't know.

System Administrators and a scripting culture...

I followed the programming (as opposed to hardware) branch within the Computer Science degree program at my University. At the university there was no stigma placed on one language over another, and so armed with my previous experience with basic, pascal and fortran, I dove into classes on perl, sed, awk, and Unix shell programming, as well as C++, Java and Lisp.

My first job was as a Unix systems administrator/technical support weenie on an proprietary embedded system. The system did not have (and it was not legal to add, without breaking our maintenance agreement) a compiler. So, any automation we needed to perform was in the form of shell scripts.

I ended up building a full blow interactive application that hundreds of people use on a daily basis to this day. The last bug for this system was found in 1999. Scripting allowed us to extend the functionality on that system, and all of the design tasks and lifecycle considerations were the same.

I have been in several projects since then, big and small. In every case I always was able to make the decision to use a scripting language if I thought it appropriate (for example, we needed to perform remote administration on hundreds of machines; what better way to automate this functionality than with Perl and Expect.pm - so I did). As a developer I always keep my eyes open for the most efficient means of getting the job done.

Perhaps being a system administrator for a time helped me avoid the stigma associated with 'scripting'. To me it is all just programming - plain and simple. Those that limit themselves and don't grok as many languages and methods as possible are selling themselves short. Today I am extending my abilities by teaching myself python, and extending my perl repetoire with perl/Tk.

Holy wars are only an overt attempt to subjugate other's ideas to your own. Its wrong - so, STOP IT!

Scripting Is Becoming Programming

In 1993, programming was done in C and scripting was one in .bat files or shell scripts. It would be fine, even good to descriminate between scripters and programmers in that environment. Then HTML came along, which was more of the same thing. Any old joe can write HTML, but real programmers use C++.

In 2003, however, the difference between scripting languages and programming languages is not so clear cut. C can be used to script the CGI that holds up a simple website, and perl can be used for writing programs.

Is Java a scripting language? It has constraints similar to that of any other programming language, but technically runs on top of a virtual machine and is thus a scripting language. Scripting languages will continue to become more powerful and more difficult to use, and this will further blur the line. With perl even gaining the ability to be a compiled language, it's often hard to tell a programming language from a scripting language.

In this way, how can you really look down on a scripter because of the choice of programming language when C and perl are almost interchangeable for many tasks?

Caveat to the Small Fish

The problem I run into with scripting (and indeed, other languages) is that I am one of three programmers at my business and the most experienced in a diverse number of languages, both programming and scripting. I try to use the right tool for the job....Perl for quick string manipulation, handling webpages, PowerScript to ease the pain of banal Windows programming, Visual C++ to handle the lower-level, API-humping apps, and pure C to do fast work when I need speed.

However, it has come around to bite me on the ass. For instance, I am the only programmer that knows Perl. As good as the tool may be, the company now regards me as an enigma -- something to be dealt with by procedure, policy, and backups. I am now being forced to document my code to a level at which a non-programmer could figure out what's going on and stumble through it. The same with the IDEs (if applicable). My code was well-documented and written before, any competant programmer should be able to pick it up. I am not being forced to do this for languages for which we have other people that know them...just the ones I am the sole intellect on.

So, as a warning to all of you trying to use your scripting or programming abilities for the good of your job. Good idea. But watch your ass or you'll end up writing n00b manuals for the rest of your days.

Nope

I write software that handles a lot of Citifinancial's processes - 90% of it is in perl.

When I have to write something in C it's just never as easy or bug free.

sh + sed vs. Java

I watched someone spend an entire week writing a Java program to parse a text file even after I told them a one line sed script could do the same thing.

It isn't so much about discrimination in the racial or sexist sense, it's about technical ignorance coupled with a reluctance to learn. Fortunately, a person doesn't have to learn the 5 billion different scripting languages out there to resolve this--just sh plus sed/awk or PERL would save weeks of time. The ROI on scripting is at least ten-fold and often much more.

"Executive-level" management

If "Executive-level" management is making decisions on what programming language people should be using, they're sticking their nose were it doesn't belong. Lower level management (if even that) should be making that decision. Deciding what to use based on biases is always a mistake. I like Java a lot, but using it when something else would be better is a serious mistake....and visa versa.

The way out: "Prototyping Language"

Tell Management that the scripting language is being used only to quickly develop a prototype, and that then the prototype will be re-implemented in a "real language" later, with profiling tools used to determine the priority order of which modules get re-implemented first.

Once the thing works, if it performs well enough, the resources to re-implement will dry up.

Once enough of it has been done in a compiled language (that is, about 20%...remember the 80%/20% rule). resources to re-implement the rest will dry up.

Soon Management will realize this new "Prototyping Paradigm" saves them resources, but gives them plenty of busy work (re-implementing scripts in compiled code) for when they need to look busy, and it will turn out to have been Their Idea All Along.

Government Contracts

I don't know why they make the disctinction, but certain contracts let by the government contractually define a difference between 'coding' and 'scripting'. On any given contract, some roles may allowed to do both, some one or the other, some neither.

As an example, project managers are not be authorised to code or script, software engineers may both code and script, technical leads are not allowed to 'code' but are allowed to 'script'.

My only experience with this policy cones second hand over lunch. It is the case of a small project that consisted of a project manager, a tech lead, and an a small number of junior engineers. The engineers were allowed to write 'code', the tech lead was allowed to 'script', and the project managers duties were restricted to scheduling and budget. Though it sounded like a good idea, schedule concerns required that the tech lead contribute to the project. Since the tech lead was not allowed to bill for time spent 'coding' it was decided to write the project in Perl (since it was considered to be a scripting language).

I don't want to get into a Perl flamewar, but I don't think anyone can disagree that Perl is not an appropriate choice of language for production systems. Perl _can_ do everything that a more structured language can do, but it doesn't necessarily do them well (it doesn't encourage good software engineering practices, has a steep learning curve, can be cryptic).

I've probably dis'd Perl too much already. flamewar is certain to follow. I'll stop more before I incite a holocaust. Suffice it to say that Perl wasn't the best choice for that project, yet the distinction between sripting and coding effectivly made it a requirement.

It's not entirely unreasonable

I'm basically a C++ programmer, but I like and use Perl for smallish text-processing tasks.

However, the main reason I see for preferring C++ for long-lived projects is one that has not been mentioned here: the stability of the language specification. The specification of C++ is extremely thorough, and changes glacially slowly. That's a big advantage for software that will have a long life. Remember, folks, that the main work that programmers do is not developing code. It's maintaining code. I've only ever used Perl 5.x; I'd hate to have to maintain something written in an earlier version that didn't have references. And in a year or so, I wonder how someone who started with Perl 6 will like MY code ... probably not very much.

All languages have this problem but C++ has it much less than Perl.

As for the boundary between "real" programming languages and the wannabes: for me, the test is whether it's well enough specified that you can determine from reading the language spec whether a piece of code is valid, and if so, what it does. Perl passes this test. (well, 99%). Others, Ruby for example, don't. For this reason, I regard Ruby as a waste of time. But I'm very results-oriented. If you have a more playful disposition, YMMV.

Actually the right way is to use both languages

This "either or" stance is false. Scripting *is* programming and to be a decent programmer you need to know both a compiled language and a scripting language. Programmers who know only one language can not be called professional programmers in a true sense of the word (BTW in old days one needs to know assembler and a high level language to be called a professional programmer; Programmers who can program, say in only Fortran, or PL/1 were often called suckers ;-)

Moreover in complex systems it's much better to use both.

The main advantage of a scripting language it that it permits writing five or more times less lines of code. For a large system this is a tremendously important consideration. Many projects died just because the codebase size exceed a reasonable limit and thus IQ of the development team and the resources of the organization to maintain it.

When you have that much less code, it's not only easier and cheaper to maintain the codebase, the design itself can be more better. This is the same consideration that eventually killed usage of assembler language for writing compilers. Moreover the time to create the first version and cost of the development can be considerable less. That's why scripting implementation is often done as a prototyping phaze.

But for most complex projects the development team can benefit from using both scriptnng and a regualar compiled language from the very beginning to the very end of the development cycle and coding different parts of the system in the most appropriate language

In this case you need a scripting language that links well with your base compiled implementation language (for example TCL+C ) but that gives a lot of possibilities to structure the system more flexibly.

One important possibility is to have an internal scripting language for the system that you are developing. That is an important advantage for a large class of systems.

All-in-all scripting language is more important on the initial, exploratory part of the system life cycle. As the system became more mature and design stabilize, it might make sense to rewhite some parts of the system in a high level language. If speed is of primary importance all the system can be rewritten, but this is a pretty extreme and rare case.

One can consider Java as a language sitting between two chairs: it's too verbose and low level to compete with scripting languages and it's too slow and inflexible to compete with classic compiled languages like C and C++.

But still using Java is a compromise that helps to achieve some benefits of scripting language and some benefits of compiled languages while using a single language. The main problem is that you often need to write 5-10 times more lines of codes in Java and that's a huge cost difference.

See http://www.softpanorama.org/Scripting/index.shtml for more inforamtion

- Nikolai Bezroukov

Technology choices are seldom rational

Quite often the technically best technology for a job is not chosen. Many times who is available to work on the code, how sustainable management believes the resulting code will be, and, quite frankly, a plethora of non-technical issues that management views as more important will have more impact than any technical criterion.

After all, if technology selection was rational, everyone would be using Lisp or Smalltalk.

The problem isn't scripters

It's VB scripters :) Actually, I'm somewhat serious on this one. I think the bad reputation that 'scripting' gets comes from the influx of incompetent programmers that come in from temp agencies when any given company decides to outsource some part of in-house programming.

Scripting isn't the problem. Slop and using the wrong tool for the job is. Ever try to write an 'application' using Microsoft Access? I was forced to when working in the temp industry...now that I use Apache/DBI/Embedded perl to do much better applications I am so much happier!

Pretty much all of my sysadmin and desktop customization on my linux boxen are done with a combination of bash and perl. Same with my web databases. Right tool for the job and whatnot...

Supportability

I have dazzled many enterprises in an emergency by delivering Perl scripts in hours or days that do amazing things. BUT once the emergency was addressed and they began to look under the hood and saw it was Perl script they had me re-engineer it in C++ or Java (weeks to develop...) because they had no one on staff (besides me) that could support the Perl. They spent the money for the increased amount of time for development to reduce cost in long term support.

Guys...cant we work this out?

Scripting provides the imperative edge to tasks that are essentially declarative When this barrier gets hazier, you have scripting lanaguages that look like programming languages, and vice versa. The real danger is that of scripters or programmer failing to understand the implications of moving to the other domain. Thats when you have, for eg, the web page from hell - asp/jsp that creates a page full of javascript that creates html which embeds a call to another jsp/asp which opens a socket to include the contents of another asp/jsp. Programmers and scripters live for this kind of masochistic fun.
Here're my tips to keep both sides productive and respecting each other:

• Have one predominant style:scripting or programming:That way everyone agrees on the pecking order. if you dont like it, jump ship.
• Define roles that compartmentalize scripting or programmingThat way nobody gets in anybody's way.The web scripter sticks to jsp/asp/whatever, and calls compiled code developed by the programmer to do the heavy duty stuff.this code does not have any UI code in it.

The Difference

OK, so a big question is what is the difference between scripts and programs? To me, a script is something you just write. A program is something you design, then write. I don't really care what language its in. A 10 line Java program that does some simple operation on args is a script. A huge multi-module Perl file/script is a program. There are other terms to differentiate what most people are talking about, its simply compiled vs. interpreted.

Re: The Main Topic

This basically means the difference is that programmers can script but scripters can't program. *ducks* Seriously, if you are writing complex enterprise-critical applications in javascript, you aren't a scripter, you are a programmer (who probably made a bad language choice). Conversely it you are just running search and replace on open source C code to suit some minor business requirement and compiling it, you aren't a programmer.

Wtf is a 'scripter'?

Anyone worth their salt should be able to code in either scripting languages or compiled languages. If they can only handle a few scripting languages like Perl or Visual Basic then of course they should be discriminated against. They're 'real' programmers, sure, they're also bad programmers.

He goes on to say that some companies will assign Java and C++ programmers tasks that take them weeks but could be done by Perl or Python programmers in a few hours.

No see, what the hell is this? Why couldn't a Java or C++ coder write the same Perl or Python script? If Python is a better solution, you should bring it up with your boss. If they don't go for it spend the extra time and collect the extra cash (assuming your hourly)

And secondly, I seriously doubt that a Python program could be written in hours that would take weeks in java, unless the coders are completely incompetent. Java has a rich API and is pretty easy to use.

The race is not always to the swift...

Dickerson wrote:

If you put the world's most talented Java developer and the world's best Perl programmer in a room and gave them an unstructured textual document to parse, I would put my money on the Perl programmer to finish first.

There is no such thing as an "unstructured textual document".

The person who finishes "first" does not always produce the "best" program.

What are you going to do in a year when all the developers are gone, and you need to update the program for some reason?

If you're going to create situations where your pet language will win, let's talk VSAM file manipulation. :]

Finally, as Dickerson seemingly fails to understand, choice of language should be as close to the programming staff as possible, not with the buzzword-laden clueless managers.

HEY! Who has the Authority here?

As a senior technical consultant for my company, I do not generally recognize any the Authority of any person, outside of the Customer, to specify which programming language I shall use. If the manager types think they know so well, I ask, why aren't they writing code?

Leave the technical decisions to those most closely coupled to the technical problem. Perhaps a few companies should learn a bit from Demming.

C//

God is in the chip tracings

It seems like the deeper you have to go to get something to work the more immaculate you are. Like everyone is hovering somewhere above laying down silicon, the further away from tracings and transistors the less holy.

In this regard machine language programmers spank assembly coders, who spank compiler builders, who spank those who use compiled lanagues, who in turn spank scripters, who would spank spreadsheet macro writers if those people ever came to the party. Of course everyone is aiming at getting particular patterns of electrical potentials established across specific etched wires and via arrays of transistor gates. But some of us are closer to God and everyone knows it.

I figure it is just like any other religion. Closest to God are the self-flagellators, ascetics and grazers, those who abuse the flesh and the mind in order to get to the bare naked truth of God. They would dream in machine code but speak not a word anyone could understand, just mumble. Then the mendicant monks and wild holy men, clinging at the gates of the city, begging alms, pitifully beseeching to God; assemblers. Less mentally scattered beggers with pens would write very terse, almost insane ramblings about how the world is actually made, their searing visions what we would call compilers. Those who would actually take those insane ramblings and teach them as a path to truth? They use languages that rely on the compilers and most people would call them preachers and spiritual leaders and merely pity the others, if not fear them.

I take my religion easily. I don't preach, and I am not a missionary; nobody is gonna be saved by me anytime soon. I conduct the rare bit of working sorcery, often for personal gain but not always, and my relationship with God (or Goddess as the case may be) is functional and laid-back (obviously). And I'm a scripter. I code to please myself as well as the higher powers. Mostly myself. If it works, groovy God is happy too. Hey I got other things to do besides obssess about Truth and my navel, OK?

It's those Nancy boys writing spreadsheet macros that are wasting their time. Rookies. ;-)

Engineering.

How about that? I'm an Enterprise Systems Engineer. I use both Scripting and Programming languages. I also design systems that incorporate scripts, programs, applications, physical components, etc. That's what engineering is, creating something using many different components. I'm a programmer, a sysadmin, and a 'scripter' if there is such a thing.

Back in the day the difference between a script and a program was simply how it was run. A script was a interpreted (usually line by line), and a program was compiled into the native machine language. Hense scripting was writing scripts, and programming was creating a program. Back then scripting languages were usually very very simple, such as shell scripting. They would execute a repeated amount of statements and become the glue between programs.

Today however technology has progressed so much that the line between a script and a program is blurred so much it's become irrelevent. What is Java then? You compile Java into bytecode, and then the bytecode is interpreted into native machine code, sometimes constantly, or in the case of a JIT, once.

Luckily I'm in a company with a manager who doesn't care how it's done, as long as it's done to specification and done by the set due date (which is flexible within reason). I use all kinds of languages, C, Perl, Java, PHP, Shell scripts, flat out SQL and PL/SQL.

Of course this is my take on it. And in Slashdot fashion I'm sure at least 10 people will point out 'flaws' in my comment and how it makes me stupid.

Languages are just tools people!

Languages, be they scripting languages or compiled languages are just tools. You chuck them all in a toolbox and pull the right one out for the right occasion.

In the same way that you don't use a hammer to remove screws, you don't use c++ to do some quick text munging. You pick the tool that will let you get the task done as efficiently as possible.

You also take into account whether this is a long term app or a once off problem solver. Do you really need a spec cycle, architecture, etc, to retrieve some data from a postgres db to a csv file? No.

doubt this is true:

"He goes on to say that some companies will assign Java and C++ programmers tasks that take them weeks but could be done by Perl or Python programmers in a few hours"

If it is, they need to get some different C++/Java programmers.

Future Reality of Software

I make these comments from the business world, not so much what you do on your off days or as an academic excercise.

So with that, here begins my tirade:

In the 21st century, languages for business have to meet the following criteria. If your company is using a language that doesn't meet this criteria, you are in trouble, and probably don't know it.

Why? Because more than likely your competitor is using a language that does meet the following criteria, and you soon won't be in business.

As a past CIO, now a CEO, I won't get technical, I will just ask these criteria in the form of a series of questions. If you run a company, it is going to become clear, which language and OS you should be using by the end of the article.

Here are those requirements:

1) Software your business invests in, and owns outright is an Asset, not an expense. Obviously this doesn't include any shrink wrap software.

Interesting point isn't it?

If you build software or buy it, and toss it out the window because you change hardware platforms or upgrade because your vendor says you have to, you are bearing costs that you don't have to bear, and are throwing your money away.

I gurantee your competitor won't make the same mistake, because one of my sales people will be explaining it too them real clear like on the telephone.

More than likely, because you didn't want to listen.

2) Software is not only an asset, but it is your intellectual property which represents a unique way on how you run your business.

Software enables this idea. Good ideas are unique, not commodities. When a good idea is applied to a business process, you do more with fewer people, less money and out manuver your competitors as a result in price and service.

Software built by companies who acknowledge that software is an asset, also understand it is an investment that is to be protected and furthermore acknowledge that as part of the IP capital of the business, represents something a competitor can't BUY ANYWHERE ELSE.

So with these two points in mind, think about these little diddies

Why would I buy SAP for example, and Windows 2000, when my competitor can buy the exact same thing?

What does buying a business process API that anyone else can buy get me? Does it give my business an edge over my competitors if they can buy the same consultants and produce the exact same thing for my competitors?

Why? Why not?

If Joe Tool and Die down the street can choose a Shrink wrap software desktop/server system for File/Print and Office Suite from Company A, and I can do the same thing for my end users if I use the exact same.

What does that get me? Am I beating Joe Tool and Die down the street following his every move?

Can I somehow make or modify shrink wrap Office Suite Word Processor A, for example, to the point it can make me a unique business process as I invest money and time into growing my infrastructure that my competitor can't duplicate in a way that makes me more money than who my competitor is?

Especially if Joe Tool and Die decides to woo some of my IT people away from me?

Can I modify File and Print server shrink wrap software from company A for my users in such a way that my competitor can't, that saves me money?

Or perhaps, something my competitor can't buy off the shelf and do the same by adding it to company A's file and print server software?

If Joe Tool and Die can't own his software A, but I can own my own software B.

What does that get me?

Does that give me an advantage over my competitor if both sets of software have the exact same features, yet I can modify A and Joe can't modify B without a License?

Company A has As/400's and Company B has Sun/PC hardware and decides to merge with company A, yet it is decided that company A's software is the real advantage to merging with B.

If A has to totally scrap its As/400's to rewrite its software on Company B's Suns/PC's, what does that do to the shareholder value of the merger?

What would have happened if Company A had software that was written to be hardware independant like company B?

Do you think the merger would be of more value?

I think it is extremly obvious what I am getting at here, and why software as we know it, is going to radically change.

Many IT professionals never EVER ask these sorts of questions, Historically. Why? Primarily because until quite recently, the technology wasn't available in any practical sense, to make such decisions very very obvious, and very very easy to do.

Anyone have thoughts on those arguements and what language and OS do you think I am talking about as I pose these arguments?

-Hack

Because the word "script" is too lightweight.

For PhBs how the words sounds does matter much.
If we say "virtual-machine portable languages" instead of "scripting languages", I think the valuation of that languages will rise in the eyes of PhBs

Scripting vs. Programming

I used to work for a company who insisted that everything be done in Java. Now I work for a company who is in bed with C# and other .NETedness. I can understand standardizing on a set of tools, but I think this attitude is kind of dumb in some respects. Sometimes it feels like hammering a screw into the wall with a somewhat stale loaf of bread.

I just finished writing a front-end application at work using Python and wxPython (which is incredible I think). It would have taken me at least a week to do it in C, C++, C#, Java, or any other buzzword language, but I finished it in a little over a day using Python. My app has the added benefits of being cross-platform (Windows, Linux, and FreeBSD), it has a native look on each of these platforms, and it runs a lot faster than a Java/Swing app would.

Ideally, such a time saving technology, and those who know how to use it, would be valued. Yet somehow those pointy haired MBAs that seem to run most companies don't seem to get it.

Real values vs marketing clout

As an avid PERL hacker (for the past 7 years), I find myself increasingly at odds against the onslaught of Java newcomers. It seems that these days, what really matters to IT directors is the "fashion" value of a language and not its real merits.

Where I work, I see this happening everyday. New projects are, by default, assigned to Java adepts even if they are relatively inexperienced or even fresh out of college/university. The whole market, here in Greece, is quickly veering towards this direction. The funny thing is, that these people quickly discover that doing productive work in Java means you have to have someone with at least a few solid years of coding behind him. So you have a large number of softcos who are looking in the market for people with 2-3 years of EJB experience and the market simply cannot supply them.

So when we scripters go to them and propose a working prototype in a few weeks (vs a few months) with object orientation, proven performance and plan for future maintenance all we get is a smile, a hint of irony and a short dismissal. No arguments, no discussion.

Makes you wonder, how these people got to be IT directors at all...

That said, it is true that scripting, with all the freedom it offers, requires discipline to write maintenable code. Java on the other hand, with it's huge APIs provides a strict framework which sort of guides you through. An inexperienced coder is bound to write better code in Java than in Perl, most probably.

And that is the crux of the thing. Experienced programmers are hard to get by and command larger paychecks. Once again a financial decision is made opposing technical considerations.

And the suits win once more.

It's inevitable

Scripting is an artificial distinction. Both scripted and compiled languages are turing-complete.

Scripters vs. Programmers

A programmer is someone who has (in)formal education in information sciences including common data structures, design patterns, algorithm, and algorithmic analysis.

With the above skills, one should be able to master any language--compiled or interpretted. Often, the term "scripter" is used to describe someone without an understanding of the above who limits themselves to scripting languages.

Obviously, the later is inferior to the former. Most people tend to favor a language but even if that's a scripting language, that doesn't make one a "scripter."

Now as to whether a scripting language could solve problems more effectively, that's simply not relevant. The largest cost in code development is maintaining and expanding existing code bases. Therefore, it is more economical to use languages that are more widely known.

More people know Java or C++ than python or perl (at least, in enough capacity to do something useful). Therefore, in most circumstances, they are preferrable.

Re:Right tool for the job

To the untrained eye, perl looks like line noise, and may be rather difficult to maintain.

To the untrained eye, English doesn't make any sense. When hiring someone to maintain Perl scripts, one should look for the trained eye, yes?

Re:Solution

So, write it as a script, put it in a crontab, and make a VC UI that just fetches a "results page" of text and shows it to the end user whenever he/she/it hits the "refresh" button. Pretend you're working on it for the next two weeks, and spend the time you save doing something useful, like reading '., downloading pr0n/mp3/movies/whatever :-)

Re:Script kiddies should be fired

"Faster" in what sense? Tight programs in compiled languages can provide faster execution, sometimes by an order of magnitude. And if that's your measure, a hand-coded tight program in assembler can be two or three times faster than an equivalent program in a compiled language.

On the other hand,

grep '^foo*bar[a-zA-Z]' *.text

can be written in 30 seconds, where the equivalent C code would take all afternoon. A C program that evaluates just that one finite state machine will run at least an order of magnitude faster than grep will ... but since most of the time will be spent opening and closing the files and doing read(2), the speed of the program itself won't make much difference. So while the program may be faster in some sense, it will cost ten times as much without doing anything more or shortening the wall-clock time to run.

To a first approximation, the time it takes to write a program is proportional to the number of lines of code in the solution, whether you're writing assembler or perl. The cost of a program is directly proportional to how long it takes to write it. So if you're going to opt for a compiled language over a "scripting" language, you should be sure that the additional cost is justified by the gains that come from performance.

In an awful lot of cases, it just isn't.