The Integration Engineer
When it just has to work.

Starting off work on my On-line Status Repository, one of the things that I will be starting with is uploading and downloading SEF file from a data repository.  SEF stand for Standards Exchange Format.  SEF files are repositories of standards information that can then be exchanged between people and applications to define the format of EDI documents.

If you have used a standards editor, you probably know what an SEF file is, or have used it.  Some applications and EAI even use SEF files as part of their document creation and validation processes.  It becomes useful to describe briefly what SEF files look like, and what type of information they contain.

(more…)

When two trading partners agree to send each other electronic documents.  And they begin to describe what EDI documents they will exchange and how the documents will flow, they should also exchange EDI specification documents.  EDI usage or specification documents describe what fields and what segments a trading partner will send or expect to convey the information necessary to complete a transaction.  It doesn’t matter if we are ordering widgets, or invoicing, or transmitting catalog data, or checking insurance claims eligibility, the EDI needs to contain the data that the two parties need to communicate.  To explain this, and document it to that both trading partners know what is expected, we create an EDI usage specification.

(more…)

ACK is an Acknowledgement segment. It is commonly found on response transactions to Purchase Orders or 855. It is one of the lines found in the PO1 loop or group.  As this line modifies the PO1 line data from the Purchase Order, what it can contain and how that data is handled is important for the stability of the supply chain integration.  For clarity, we are going to use the 4010 x12 standard.

(more…)

Like the PO1 segment, IT1 is another line item segment used in the supply chain.  This is the line item segment found in invoices.  And since invoices make up an important part of a supply chain, I wanted to spend another article on this segment.  (IT1 is a group or loop so it is more than just a segment.)  In this article we will discuss what data is found on the IT1 segment, and how it should be handled on both inbound and outbound document.  For clarity, we are going to use the 4010 x12 standard.

(more…)

When we are dealing with supply chain or e-commerce related EDI, we probably will deal with line item data.  On a Purchase Order, (PO) or 850, the line item data is contained on the PO1 segment or the PO1 group.  (PO1 is a group or loop so it is more than just a segment.)  In this article we will discuss what data is found on the PO1 segment, and how it should be handled on both inbound and outbound document.  For clarity, we are going to use the 4010 x12 standard.  The PO1 hasn’t changed in content for a while, but it has gotten longer, (you will see how this works when we talk about the PO1 elements). (more…)

A reader asked me a question, and I want to pass it along: “Do you know of any DOT NET frameworks for working with EDI?”

I didn’t, and a search of the web didn’t reveal anything that I could, in any confidence, recommend.  So I bring this question to you.  Do you know of a DOT NET framework that can be used for generating and or parsing EDI data?  If  you do, please comment below and let everyone know.  I will also pass it on to my friend who is looking for this.

(more…)

From time to time I am asked if I know where such and such standard can be found.  This is more than idle curiosity.  People are trying to integrate, or update integrations, and the standard documentation is not available or has been lost.  From the perspective of an integration engineer, such standards are more precious than gold.  I keep a collection of the standards that I have used over the years, (something that I recommend) but what I really encourage is for companies to publish their standards in a publicly accessible location.  Booknet Canada is an example of a company that has done this.  Lets take a look and see what it says….

(more…)

I have sitting on my desk, a very expensive book published by the ASC that contain the guidelines for the X12 3010 EDI standard.  I have rarely used them.  The EDI standards tool that I have used the most and can highly recommend is EDISIM from Foresight.  The last version that I used was version 5.0 and they are on version 6.8 as I write this post.  But for a long time, Foresight has nailed the conceptualisation of working with EDI standards.

One of the things that in invaluable is the ability to share your standard documentation both internally and externally.  EDISIM is a tool that has allowed me to do that.  They produce a very professional looking document for external consumption and the standard is exportable so that everyone using EDISIM can share the same standards file in a format called SEF.

(more…)

As part of our series “EDI Primer” we are going to go over the various parts of EDI. EDI is neither truly delimited, nor fixed width.  EDI has some fixed position components, and uses delimiters in a different way than a truly delimited file.  This post it going to overview some facts about EDI and its parts in an introductory fashion.  If you are unfamiliar with EDI, you might want to start with the post, “What is EDI?”

(more…)

This is a mapping exercise that will go through the process of creating a paper map, or mapping document.  We will start with an empty paper map that you can get here.  And we will end with a completed paper map document that documents what data from the source goes into what fields on the target.  This process will take more than one post, and I will link them together so that you can follow from one to the next.  Along the way, we will discuss the things that we are doing so that you can apply this technique in your mapping using the target and source in your own mapping tasks. (more…)

In EDI there is a construct called a Loop.  But this is not like you think of a loop using an while  statement.  EDI is a file format, not a programming language.  When we speak of Loops in EDI, what we are really talking about is repeated structures, or segments.

EDI is not a flat file, but it does have repeated lines or segments.  Some times this repetition comes in groups and sometimes its just a line that get repeated in part of the document.  Repeating structured data is fundamental to EDI, but can become difficult to interpret at times.  Understanding the types of repeating structures and when and where they may occur can save a lot of time. (more…)

The 3rd layer of the EDI envelope is the ST segment.  The ST segment contains a Document Identifier, and a control number. The ST and its partner the SE segment define the beginning and ending of the Document.  The SE segment contains a counter of segments within the document, and the corresponding control number to the ST.  Relatively simple compared to the ISA and the GS. (more…)

EDI in its proper form is wrapped.  This means that there is nothing between the segment terminator and the beginning of the next segment.  Sometimes this is hard to read, for humans anyway.  And at those times people will unwrap the EDI so that it looks more like a flat file.  This is not hard to do, and is useful for debugging and trouble shooting processes.  A good parser or validator should be able to read either file. (more…)

What is an Element?

In the context of delimited data files, an element is the contents between two delimiters.  Like in a spreadsheet, the element is the contents of a cell.  In EDI, we have delimited segments, and just like a delimited file, the contents between the delimiters is the element.

So in this segment “PO1*1*8*EA*10**SW*H11510.REF*CT*089480-00-B-0336~”, the 5th element or PO1_04 is “10″, while the 6th element or PO1_05 is empty.

Empty elements are not necessarily null.  Calling something null when the data type is know like an empty element can cause some data theory debates.  I don’t want to start one here, but may write about that philosophical discussion at a later time.

(more…)

What is a flat file?

Files are called “Flat Files” when they contain a single data structure.  Generally this structure is the column and row structure like a spreadsheet or table, but a file in binary or encrypted with a single encryption key could also be called a flat file.  Files that are not flat; marked up files like XML or HTML, EDI files, other formats like HL7 or SEF files and others.  Here I am going to briefly discuss two flat file types; Delimited Files, and Fixed Width Files. (more…)

When EDI was born, (in the 1970s),  the committee did its best adopt a forward thinking plan and architecture.  But there was really no way for people living before the birth of e-mail to make place for what we use today to do business.  So the early versions of EDI had no place for e-mail addresses, and other things that we use in e-commerce today.

The goal of EDI was to provide a way to support communications between dissimilar computer systems.  EDI was intended as a light weight, reliable means to transmit data. (more…)

EDI is a standard, and having standards in e-commerce or any type of integration, is important in achieving success in your integration efforts.  I found this old video of Bill Gates talking about this very topic, and I decided to share it here. (more…)

What is the GS segment?

The GS segment is the second mandatory enveloping segment.  It shares some properties with the ISA segment.  There is a sender and receiver, version Identifier, time stamp and control number.  But one thing that the GS has that the ISA does not is a Functional Identifier.

That’s right, Functional.  This means that within the GS envelope are only EDI messages or documents that have the same function. (more…)

EDI stands for “Electronic Data Interchange”  It is a file based format that goes back before the modern Internet and our current concept of e-commerce.  The problem that EDI was trying to solve was, “How do we get data from one system to another reliably, cheaply and in a common format?”  These problems are still with us today, not because EDI didn’t provide a workable solution, but because it was not implemented on a wide scale.  And because competing solutions also exist so that no one has a corner on the market of data formatting. (more…)

What is in the ISA?

The ISA Segment is the first segment in any EDI document.  Once you understand what it is saying, This long random seeming string will make sense, and be very helpful in helping you relate EDI documents to the real world task that you need to do. (more…)

An EDI file contains a structure known as enveloping.  Everything that is necessary for the file transmission and parsing, is part of or inside of the envelope.

The first segment in and EDI file is the ISA segment.  The very last segment in and EDI file is the IEA.  For those of you familiar with XML, you may be thinking of this as <File> and </File>.

This approach may make EDI seem familiar, but XML was not around when EDI was being created.  It is more like a Beginning Of File (BOF) and End Of File (EOF) marker.  However, there is no reason in EDI not to place multiple EDI envelopes in the same file. (more…)

What are Delimiters

In short, delimiters are characters that separate the elements of data so that one piece of data can be distinguished from another.  EDI files don’t have externally set delimiters.  This means in a pure sense, that an EDI parser may not know what the delimiters will be until it has begun to parse the file.  This may sound chaotic for someone familiar with strict delimited files.  But we need to remember that in and EDI file, the first segment is fixed position.  This is how EDI addresses the problem. (more…)

Beyond just compliance with the EDI standards, good form is using good practices in planing the format of your EDI.  Below are a set of rules or guidelines that will help you create EDI in good form.  Good form helps others to received your EDI and process it without encountering common exceptions.

Eliminate trailing delimiters.

When EDI was first created, modems were much slower, and high speed internet and Wi-Fi didn’t exist.  Moving large, verbose files over a network took time, and e-commerce solutions charged by the kilo-character.  This means that having unneeded characters cost money. (more…)

Why are segments Important?

The segment is what makes an EDI file an EDI file.  They are at the very heart of EDI.  There are hundreds of different types of segments and each holds a different type of data.

N1^BT^Bill Gates^01^123456789~

For instance the ‘N1′ segments holds some common data that is used in many EDI documents.  First, the ‘N1′ type.  This is the name type.  The first elements following is the qualifier for the second field.  If this is were an invoice it might be the code ‘BT’ indicating that the value in the second field is the name of the “Bill To’ party, in this case ‘Bill Gates’.  Following the name is another qualifier that determines the type of the account number in the fourth filed.  In this case we will use the DUNS Number, and so on.  We stop now with the segment terminator to indicate to the parser that the next character will start a new segment. (more…)

For those who may be unfamiliar with the EDI standard, and those who may have used EDI but have encountered a new, unfamiliar question.

This introduction aims at addressing the basic questions of convention and implementation of the EDI technology.  I have a list of topics, and hope that over time I can cover all of them.  I also hope that people reading here may have questions that I haven’t thought to write about.  Please let  me know what your questions are and I will see what I can do about providing answers.

• What is EDI?
• Parts of EDI
• Segments
• Delimiters
• Conventions of EDI
• Why are there more than one version of EDI?
• Usage of EDI specifications
• Standards and Usage
• Encoded Values
• EDI in Good Form
• Element Notation
• Wrapped and Unwrapped EDI
• Enveloping Part One
• Enveloping Part Two – ISA
  
• Enveloping Part Three – GS
• Enveloping Part Four – ST
• Loops in EDI
• Hierarchical Levels

Of course understanding how EDI presents and represents data is the first step.  From here we may want to go to our Data Mapping Guide, or if we are familiar with mapping there are some examples in our mapping case studies of EDI to and from other formats.  Again, if you have an interesting problem or solution, I would love to here about it.  Please send it to
roy@TheIntegrationEngineer.com and i will see what I can do.