The Integration Engineer
When it just has to work.

Ever have a customer contact you because they forgot to place their order with their registered profile?  Well I have.  Alot.

And Magento doesn’t have an “out of the box” way to modify the order to customer profile relationship even though the solution is really simple.

Simple that is, if you are comfortable writing queries in the database.

(more…)

Sometimes a customer will input a typo when entering their email address in Magento.  If they do this they won’t get their email notifying them of the order.  (if you have this setup.) And other things may not happen as expected and or desired.

Out of the box, the Magento admin does not have the ability to change this.  But if you are comfortable in the database, then these steps will let you make this change in seconds.

(more…)

Definition:  A database is a structured collection of records or data.

Many moons ago, I was shown an database that was constructed back in the 50s.  It was hand held and consisted of a stack of cards that contained information about plant biology.  There was a series of holes that wrapped around the cards, some of the holes were notches, and others were not.

To query the database, one placed a pin through one of the wholes and let the cards that had a notch fall out of the stack.  You could continue this process by removing and adding pins to select different combination of data.

Once you had narrowed your search, you could read the cards, and retrieve the data.

This was a very manual process, and I doubt that anyone today would take the time to learn to use such a tool, let alone update or add records to it. (more…)

As someone that has been involved in the supply chain industry, this questions has come up more than once, and at more than one place of business. For both vendors and retailers, having access to better, broader and more accurate information is worth money.  And is sometimes the difference between life and death of a business in a competitive market.

When businesses realize that the catalog data that they have may be worth as much as the products or services that they sell, they may be tempted to sell that instead.  Before they do, they need to think about who owns the data.

(more…)

If only it were really that simple.

Well, maybe it should be. The evil robot found this video, and thinks it is good enough to share. But don’t get your hopes up. This lays out some foundational information, but there are so many ways to use XML that this is well short of a complete overview.

But it was still fun to watch.

Once. (more…)

Continuing Mapping Exercise

Today we will identify our data source, and begin mapping the source data to the target data.  We identified our target format and placed that in the paper map last time.  If you didn’t read that post yet, you might want to review it quickly before continuing.  (read part One)

(more…)

This is a mapping exercise that will go through the process of creating a paper map, or mapping document.  We will start with an empty paper map that you can get here.  And we will end with a completed paper map document that documents what data from the source goes into what fields on the target.  This process will take more than one post, and I will link them together so that you can follow from one to the next.  Along the way, we will discuss the things that we are doing so that you can apply this technique in your mapping using the target and source in your own mapping tasks. (more…)

There are job or task specific tools that will have a high importance to each integration task.  When working on an SAP system, your SAP tools will be very important.  But there are tools and skills that are also important regardless of the systems and technologies that you are working on.  For me, these are the top 5 tools that an Integration Engineer should be able to use proficiently.  Do you use any of these?  Do you have others? (more…)

Introduction: This is a quick exercise to familiarize you with mapping from an EDI file to a Flat File.  If you are new to mapping, or want an idea of what mapping EDI will be like, this exercise should be a good place to start.  If you are familiar with mapping this should be a quick review with a few tips.  I use Target Based mapping.  Check out my post on Target Mapping here.  If EDI is unfamiliar and you need some basic information, my EDI primer is here.  If you are ready to map, and understand EDI basics, then lets get started.

(more…)

I found this video on YouTube.

Theoretical discussion of integration of data and information.

Speaker is Alan Nash

(more…)

I found this video on YouTube.

RFID is exciting technology for integration engineers.  All of this data coming in will have to be aggregated and integrated by someone.  Also, one of the focuses of integration is to provide more and better information to people and systems.  With more and better information people, businesses and systems can make better choices and decisions. (more…)

What is a “Character Set?”

A character set is a collection or library of characters, (letters and symbols), and their identifying number.  Included with the printable characters, (letters and punctuation) are some unprintable yet important characters.  Characters are used to form messages.

Characters are not fonts.  Characters exist under the font that represent the definition of the character the  font is attempting to display.  When you change the font on a document the A is changed to an A, but the underlying character that identifies its meaning remains the same.  The font identifies how the character is displayed.  You can even convert to Wing Dings and the underlying character remains the same. (more…)

Back in the day, I used to use a program called Windows made by a little company called Microsoft.  I liked windows, and for the most part it did what I needed it to do.  But it also had a nasty habit of crashing when I tried to do something complex.  And this was really annoying.

Well as time went on, and I became more adept at using windows, I noticed that it tried to tell me what was wrong.  Windows was writing out a crash report.  And it had a message on the blue screen of death.  (a mostly incomprehensible message, but at least it was trying.)  And I discovered that if you looked for the error codes in Microsoft’s knowledge base, you could find the errors had corresponding articles for many of them.  (some of these articles were also incomprehensible.  ) (more…)

A circular file is not a nickname for the waste can. Circular files, sometimes called round files, are useful in some applications and support tasks.  With a normal log file or repository, the log grows as logged events are added to the log.  The obvious danger is that if the space where the log is located becomes saturated when the log grows to fill this space.  Many applications will shut down and refuse to restart if this happens.  For some applications, having the log write to a circular file is the answer.

(more…)

What is a flat file?

Files are called “Flat Files” when they contain a single data structure.  Generally this structure is the column and row structure like a spreadsheet or table, but a file in binary or encrypted with a single encryption key could also be called a flat file.  Files that are not flat; marked up files like XML or HTML, EDI files, other formats like HL7 or SEF files and others.  Here I am going to briefly discuss two flat file types; Delimited Files, and Fixed Width Files. (more…)

“Is not the beginning of wisdom the words: ‘I do not know’?”

– Data, Star Trek: Next Generation: “Where Silence Has Lease“

If the beginning of wisdom is to realize what it is that we do not know. NULL, by its definition is this not knowing.  We do not know what NULL is, this is why it is NULL. (more…)

In the beginning we start with the data target.  This may be strange if you have not done any mapping, but the first, best thing that you can do to make a mapping project successful and fast is to start with a well defined target for your data.

I believe that Steven Covey says it this way, “Begin with the end in  mind.”  We do the same thing in data mapping, except that we begin with the end or target if you will.  The data target naturally leads us to the data sources that we need. (more…)

“So, how long will that take?”  Is a question that some of us have grown to hate.  And it seems that it is a question, that in various forms, we are asked daily.  (If not more frequently)  And if you have been doing this for any time, you have probably come up with a way to answer these questions.  There was a time when I started answering this question with a standard answer of, “2 weeks.”  And let me explain why.

(more…)

Its time for another mapping exercise.  This time we will receive an EDI 832 in 4010 format, and map the data to a DB or flat file.  Getting catalog data into your procurement system is an important task.  And creating  variety of mapping exercises provides us with a better understanding of how mapping projects work than just having one that tries to be everything.  And mapping is one of the fun things that we get to do.  Following this exercise we will have worked through the the common issues and demonstrated how this process comes together.

Mapping Steps

(more…)

One of the basic tasks Integration Engineers do is to compare files that we use or receive.  There are some interesting and useful tools that people can get out there to DIFF files.  But on Linux and Unix machines around the world there is a native tool that is almost always present.  Amazingly it is called DIFF.

Like some other command-line tools, its interface is not really intuitive.  Lets walk through the basics of how to get use out of this handy file comparing tool.  (If you are working with and comparing EDI files, you might want to look at the post on how to unwrap your EDI file so that our line by line comparison is more meaningful.)

(more…)