Is There a Free Software EDI Solution?

pi_rules asks: "I'm by no means an expert on EDI software (stuff used to communicate purchase orders, invoices, ship notices, etc., electronically) but from a programmer's point of view I'm dismayed at what the market for this stuff looks like right now. Looking at the specs to the documents it doesn't seem like an overly daunting task to implement a solution on a business-by-business basis; however the only commercial solutions out there are huge, expensive products. I'd like to know what other people who have worked with EDI software can tell me about what they need their software to do; and if they have a solution if they could give me a general over-view of how it even works. My intention here isn't to beat up on existing EDI solution providers really; it's just that I want to see a solution that's 'geek-friendly', and beneficial to businesses by setting up a system that will provide them software that not only functions, but does the job lickety split."

"After shelling out $10,000 for one of these products we found it to be: slow (incredibly slow), unintuitive, and it didn't even work! Of course, the company is clueless as to how to fix the problem. Granted we've only tried one vendor, but this was the one -recommended- to us by our customer. After asking around, it turns out that other businesses with similar needs had to hire in programmers to write their own custom solutions.

I'm entirely capable of writing the software to handle our situation here but I'd like to start a Free Software project out of whatever I make. The problem is that I know little about this massive standard and the Right Way to go about constructing software to do the task. The standard is absolutely HUGE, and can vary significantly from vendor to vendor I'm told."

it depends...

[djweis]

You can implement any parts of the standard. Some people just do the PO (can't remember the number, maybe 581?) form, maybe the invoice message.

I think it's a neat idea. Go for it!

Why EDI?

[cancerboy]

I know EDI has been around a long time, but why EDI?

These days XML is as capable, or more capable than the EDI systems, and for that matter there are a lot of tools for doing XML based messaging.

I've been working for a year on a project (contract) writing client-side interfaces to CLEC OSS interfaces based in HTTP and XML. Development is fast (it's convincing my client to take the time to design the client side APIs and do the integration work correctly the first time - man they wanted quick one offs that don't scale).

Re:XML by itself can do nothing

[cancerboy]

Granted that XML is not a solution in and of itself. The point I was trying to make is that using EDI specs and proprietarty VANs is really inefficient. A number of the companies I've worked for have been taking their EDI solutions which are software with transfer maechanisms and document translation to multiple formats, and moving them to HTTP based XML transfers. Simplifies the software considerably. I was asking, why bother with the EDI spec (OK legacy apps are one reason). But if somenone is building a new "extranet" solution, it seems that it may a bad idea to latch onto EDI, when cheaper more efficient tools are out there.

XML Irrelevant -- Standards Cabals are the problem

[david_goldstein]

I have embarked on this attempt to many years ago. EDI historically has used flat file fixed-width formats that were easy to process with COBOL. Any fool could write an EDI parser in those days (and this fool did). Now all EDI packages are moving to be XML based. This makes sense because XML is even easier to parse than fixed-width flat files and XML parsers are available universally. Now the problem is exactly what one other reader pointed out -- you need to tell the EDI package what the fields mean. So any person can pull out the field named "PO number" or whatever, but how do you know that the other guy didn't call it "Prchs. Nbr." The answer is that standards bodies such as ANSI X12 and EDIFACT define this. The specs are huge, and they change periodically. When I tried to get a subscription to the specs, they wanted to shaft me for big money. They also change the specs every 4-6 months, so you have to buy a subscription, and electronic format costs more (at least when I tried). I wanted to release a freeware EDI processor, but realized that those guys who charge $10,000 are probably just breaking even because of the fees they pay the standards mafia. The whole idea if oasis and biztalk.org is that they are hoping standards bodies will publish their specifications (schemas) in machine readable format for free download. Unfortunately, it looks like there is still alot of resistence and the standards mafias are attempting to control access to schemas. If products like mikula's, commerceone, biztalk, etc. all agree to work with each others schema formats, then they will be able to put pressure on the standards groups. Otherwise, if they stay fragmented then EDIFACT and all of the other industry groups will be able to play the vendors against one another and fragment the market..

Re:Good to see other interest...

[david_goldstein]

Contrasting EDI and XML as if there is a difference shows a lack of understanding. EDI can be done using XML (many companies do) *or* using flatfile formats like X12 or EDIFACT. Of course XML is nicer than flatfiles for input, but that is fairly irrelevant. GM, Walmart, and others will still require flat files for awhile, people coming on board today all use XML formats now. Even with flat files though, it was common to need to translate between flat file formats (for example, P&G would send a version x PO to WalMart, who needed a version y.) As far as the business is concerned, the EDI package should be able to handle whatever EDI format they want

Re:Go to it

[david_goldstein]

I think you all miss the point by thinking the translator is the hard part. You can convert just about any EDI format (flat file, XML, or other) just using a well-constructed XSLT transform. Yes, virginia, XSLT will transform flatfiles, too! The part that is hardest, IMO, is getting the standards cartels to tell you what their particular format is. You should start a "free standards" movement to get these people to describe their formats in a machine-readable format and make it freely available instead of charging thousands like they do now. If the standards are available, any idiot can write a translator, and many will. Starting a free-standards movement might be a waste of time, though. Look at http://www.biztalk.org if you want to see one attempt. oasis is another. The problem is, you need to get EDIFACT, X12, and all of the other cartels to provide the standards they use for free if you want this to work. IMO, you should be applying your pressure within your company to use only EDI formats that are freely and publicly available. After that, it is trivial to write your own translator or buy one off the shelf..

Everyone wants to write an EDI Translator

[wake2sleep]

The "rub" with EDI is not the technology, but the particular business. I've worked on a couple different EDI projects in the utilities industry. Despite the "standards" for various forms, each company still has its own version of each form, or its own opinion of the way things should work. EDI Analysts need to know the business, the EDI standard, and the way EACH of their business partners interprets the "standards". And XML is no better (yet, as a replacement for EDI). At least it includes metadata which should help (eventually?). But the technology of EDI is just as simple as, if not simpler than, XML. So writing an EDI Translator in theory is a simple task. But whizzing contests between various vendors makes EDI a nighmare. Maybe it's different in industries where both vendors benefit from the exchange of information. But in the utilities industry, the only benefit of giving another company your data is that it satisfies government regulations. Good luck on your quest! But I leave EDI off my resume ;)

XML by itself can do nothing

[Alex Belits]

Saying that XML is a solution is similar to saying that binary files, or coma-separated lists, or lisp are solutions -- there is nothing actually defined in the standard except some data formatting and possible means of verification that data indeed is formatted as specified. "Tools" can either verify that formatting is valid, or convert between XML and internal representation with no semantics attached, so software should still implement semantics in its code with no "automation" or standards whatsoever provided by the tools.

Good to see other interest...

[runswithd6s]

I've submitted this very same question to Ask Slashdot once before. It's nice to see it actually surface and get a little attention, although I see not a lot of attention.

My first impression of EDI was why? But, as I read in to it a little more, I was increasingly impressed with the concepts, just discouraged by the implementation. EDI, for people who don't know, stands for Electronic Document Interchange. It has been around for quite a while and developed out of a military need for efficient, standardized documentation during WWII. No, it wasn't electronic then, but it didn't take long to adapt it.

If you've ever looked at a raw EDI document, you'll find that it is simply an ASCII flat file whose datafields are defined by a standard document template. It's amusing that they use the word "standard" to describe the template, because with each company you interface, you must have a copy of their "template" to tranlate the document with your EDI software. If you use the same software, GREAT! Just pass the template around. If not, you will normally have to hire an EDI consultant to create the template for you. The company we were going through would charge between $500 - 750 a pop (per business-to-business exchange) for 3 or four document templates (Purchase Orders, Order Acknowledgements, and Invoices are the basics).

The "markup" for the EDI document is supposed to be "universal", but my impression is that they've perhaps standardized on a few notations and styles. You could probably be able to pick out key fields in an EDI document by browsing the raw source file, but you would more than likely need to have your tranlsation specs sitting next to you to confirm.

Frankly, EDI is lightyears behind XML in readability/parseability but quite a bit ahead as an established business-to-business document exchange standard. Perhaps we'll see a migration of EDI to XML happen in the near future. It seems like the next logical step.

This leads in to my next comment, and a brainchild yet to be born. I would love to see an EDI to XML translation library pop up out of the Open Source community. (Which leads, of course, to the XML to DB import/export library...) Unfortunately, I'm quite busy and can't lend my hand in this venture, and as he previous post suggests, it's quite expensive to obtain copies of the "standards" documents.

I did have some links relating to EDI, but since I left my last job, I've had little need to follow up or maintain the list... IOW, I've misplaced it. Look into the XML&EDI scene on google, you should be able to find a few links.

Re:Why EDI?

[JordanH]

• These days XML is as capable, or more capable than the EDI systems, and for that matter there are a lot of tools for doing XML based messaging.

Because a lot of companies, particularly very big companies, like Walmart, all the package carriers (UPS, FedEx, DHL...) and the Department of Defense use it heavily.

They aren't going to change overnight. If you want to do business with these guys you might be advised to get some X12 EDI capabilities.

-Jordan Henderson

Heh...

[cr0sh]

I've worked with a bit of EDI myself - mainly the products from St. Paul Software [spscommerce.com] - I don't know if these were the guys you worked with (it doesn't sound like them) or not (it wasn't the Gentran software from Sterling Commerce [sterlingcommerce.com], was it? I've never used their products, so I can't comment on them).

I've had good luck with SPS's solutions in regards to EDI (first with spEDI*tran on a sco box, then with Evision on NT) - they had excellent tech support, which is one thing that has stood out in my mind. However, their product isn't cheap.

To be honest, portions of the standards for EDI (which come from the UCC [uc-council.org], a company we all love, right?) are convoluted, while most of the others are straight forward (most of the warehousing/inventory/order forms are easy - but the insurance related forms can give you nightmares).

Good luck in finding a free project, or starting your own - the "standards" tend to be the issue. It seems like every year they change significantly, to the point where it is hard to keep up with the standard. On top of that, the fees to buy a copy of the standards (in order to code a solution) can be pretty high. Plus, anybody working with the system would need a copy of those standards as well.

Most of the EDI solutions offered by companies (such as SPS's Evision product) are actually development environments, that allow you to create the templates to do the translation (basically, all EDI comes down to is data translation from one obscure format to another "standard" format to interchange data). The company may also offer services to create these templates as well, for a fee (many times it is cheaper to do this than doing the templates in house with the software).

I looked into the possibility of creating my own EDI software at one point - a template development/translation environment. I realized pretty soon that such a task would be near impossible by myself. I figured to even have a hope, I would have to have about 10 programmers working and designing for about a year before we would even get something (this was before I heard of free software, open source, and the GPL). Maybe today it would be possible to do what you want - but it would be akin to creating a programming language (in fact, SPS's software started exactly that way back in the early 80's - as a form of interpreted BASIC - you essentially wrote BASIC code to do the translation - the tools that came later GUI'ized the system, but still, under the hood of Evision, lies BASIC code and interpreter, chugging away). With the head start the other companies already have (10-20 years in some cases, like SPS), it might be possible, but it would be an uphill battle.

Still, if you have the need and development time, and you are well versed enough in EDI to consider it, I say go for it - create a simple system to handle creation of simple forms, or the portions of the forms thereof (so you can combine the pieces to form full forms), then go from there.

BTW - if you get something going, anounce it on /. - I would love to follow it's progress.