The Integration Engineer
When it just has to work.

Like the exchange of data between Sellers and Suppliers, the exchange of data within the company is vital to successful processing of transactions.  This could be the Seller or the Supplier, it doesn’t really matter.  As data passes from the processing application, to the external interface.  We have a concept that is called a Canonical form of data.  Canonical data is the data that is required and essential to completing your business.  I have written about this in passing before, but today I want to talk about what goes in it, and what it looks like.

What is Canonical Data?

Canonical Data is the data that you or your business needs to complete the job you have to do with it.  If we are talking about order processing, then the Canonical Data will comprise the data that is used to complete the order.  This is not the minimal data set, this is the whole data set.  This is not necessarily the EDI enveloping data, it may be Identifiers that relate to the TPIDs, and it may not contain control numbers and other things that may only be found in the EDI file.  But it will contain information that you need for the order, PO Number, Line Item data, and so forth.  All of this data will need to be passed from your integration, translation, and or communication software, to your order processing systems.  And it may be passed around between systems as the order is filled and processed or what ever else you are doing with it.

What this data is, becomes defined based on the requirements of your internal systems, software and data needs.  One company may take shipping data and stick it into a data base outside of the canonical data.  It gets retrieved later, but the internal applications don’t have to deal with and preserve it.  Others may include it and leverage it on some internal aspects of their order fulfilment.  But the Canonical Dataset that your business needs and uses will be defined by the internal systems so that they can do what you need them to do.

What form should it take?

This is largely a choice that is up to you and you organization.  There will probably be some formats that are native to some applications.  As long as these fill the requirements of encompassing the data that you need to process on, these are fine choices.  Also, looking at working with these internally through upgrades to 3rd party applications, and possible inclusion of new applications, a flexible format that can be updated is important.

For me, in my experience I have seen flat file and XML used with a good deal of success.  EDI may have a comprehensive set of data, but can become really clumsy when working with internal applications that need to be extended by adding data to the canonical file.  So I would discourage EDI or a canonized XML like cXML to be used at the canonical

Who sees the Canonical Documents?

Please keep your Canonical Documents for internal consumption only.  It is not that they contain secret information, (they may have confidential data)  but they are designed just for you and  your applications.  I worked with a supply chain company once that developed their internal XML format, and then published it out to a couple of trading partners.  These partners integrated to this.  Eighteen months later, when the company was doing some growing, they were stuck supporting this one-off format that was no longer used as the internal canonical.

Developers, and those supporting the applications are going to see and deal with the canonical data and data files.  Monitoring and reporting systems may represent the canonical data, but it is rare that you need to invest time training and supporting bringing people outside of this group up to a familiar speed on your canonical file format.

What have you seen done?

Does your company have a formal canonical dataset?  Or what have you seen done that works to fill the need of canonical data?

Subscribe to "The Integration Engineer" by Email
Find out about the tools and services available at The Integration Engineer's Consulting site.

Bookmark It

This entry was posted on Tuesday, September 29th, 2009 at 3:18 pm and is filed under canonical. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.