EDI in Good Form
Beyond just compliance with the EDI standards, good form is using good practices in planing the format of your EDI. Below are a set of rules or guidelines that will help you create EDI in good form. Good form helps others to received your EDI and process it without encountering common exceptions.
Eliminate trailing delimiters.
When EDI was first created, modems were much slower, and high speed internet and Wi-Fi didn’t exist. Moving large, verbose files over a network took time, and e-commerce solutions charged by the kilo-character. This means that having unneeded characters cost money.
EDI, in good form tries to eliminate extra characters.
The rule is to keep your segments as compact as possible by eliminating optional trailing data elements.
Choosing Effective Delimiters
It is illegal to have a delimiter as part of the data in any data field. There is no escape character that will allow it. That being said, many systems will allow EDI that breaks this rule to be formed, as it does not harm the system sending the message.
However, it becomes impossible for the receiver to read the message properly. (I know that some of you have encountered and scripted solutions for this type of thing, but this is about good form, not work-a-rounds for bad form.) If you are lucky, this violation only errors out of translation. If you are not so lucky, the translation succeeds and you have corrupt data in your system.
The rule is to select delimiters that will never appear in your output data.
Avoid Unnecessary Repetition
Some segments can be found in multiple locations in an EDI document. This allows for a flexible and context sensitive usage of segments and data.
Lets take the N1 segment and group for example. In a shipping notice you can use the N1 in the header section before any of the line items, but you can also use it in the line level. This gives you power to specify specific shipping locations for the line.
This is great if each line is going to a different locations, but if most of the time this means that the N1 group repeats the same information over and over, then we are wasting space and are not in good form. It would be best if we place as much information where it will only be displayed once if it is the same. And in our shipping example, we can specify that N1 only appears in the line if and when there is an overriding shipping locations. (this is where having effective usage specs comes in handy.)
The rule is to avoid repetition of any data that can be represented in a non-repeating structure.
Remove Excess White Space
Just like any other character spaces take up room in the file. Outside of the ISA segment, no EDI fields should contain leading or trailing white space.
The rule is to trim or remove any whites space in an EDI field.
Send EDI in Wrapped form
When EDI is wrapped it looks like one continuous line of text characters. When EDI is unwrapped each line appears on its own line. Normally people do this by inserting carriage returns and line feeds after each segment delimiter. Humans have an easier time reading an unwrapped EDI files. So in most of the examples of EDI that I use here are unwrapped EDI.
Wrapped and unwrapped EDI are both EDI and most parsers don’t have a problem with either one. But line feeds and carriage returns are characters and take up space. And because in good form EDI eliminates excess characters, these should be eliminated as well.
The rule is, always transmit wrapped EDI.
Some people try to have the best of both worlds and try to use a line feed as a segment delimiters. This is a bad idea. First, delimiters are supposed to be a single character, second, different operation systems treat these characters differently. And some systems pre process files and remove carriage returns and line feeds. Basically, its not a nice way to do it and will eventually cause someone trouble.
What all these rules have in common?
What all of these rules have in common is saving space. Creating EDI that uses extra or unnecessary characters is bad form.
–
Looking for something else relating to EDI? Check out the EDI Primer post
–
Subscribe to "The Integration Engineer" by Email
Find out about the tools and services available at The Integration Engineer's Consulting site.
August 23rd, 2008 at 6:34 pm
There are many kinds of “bad form” in data, I once worked on a database with 1,500 columns of data in a single table. CRLF as a delimiter is just as bad.
Nice Blog.
July 23rd, 2009 at 8:15 am
If EDI in good form is wrapped, then at what point would there be a carriage return? If there should never be a carriage return, don’t we end up having to deal with a reading in a file that potentially has the entire file on the first line?
July 23rd, 2009 at 8:57 am
Sean,
Yes, in a well formed EDI document, the entire contents is one line of data is viewed that way. This is a cumbersome way for humans to read EDI, (especially if said humans are new to the EDI format), and so un-wrapping the EDI file is a useful thing when looking for errors and such. I have another post that deals with how to wrap and un-wrap EDI, and you can find it here. http://www.theintegrationengineer.com/edi-wrapped-and-unwrapped/
So your observation is also correct in that there is no point in having a carriage return or line feed. As I said, “some systems pre process files and remove carriage returns and line feeds” and when those systems are encountered, EDI that is CRLF delimited fails.
The rules of good form are not like the rules of the standard. Violating the standard, and you have a corrupt file. But violating the rules of good form, and you may get away with it. These are more like guidelines to keep integration engineers from falling into common mistakes, and to recognize them when they see them.
I hope this helps clarify, please let me know if it does not.