I often see people use the two terms interchangeably, leading to dilution of the message. Often, causing confusion.
In this post, I attempt to bring some clarity given the context of data movement.
Data is used in two contexts: operational and analytical.
The operational context is part of a bigger business process e.g. order placement. Here the applications need to talk to each other, share data to achieve/contribute to a business process e.g. for the customer to place an order, the order system needs to know what products and inventory exists
The analytical context may or may not be part of the business process, and usually involves extracting some insight from the data. e.g. measuring demand based on the order history of products can be used to either feed back into pricing process (business process) or simply used to report on growth
In the earlier days, and to some extend even today, data movement in both the contexts would involve some kind of extract, transform and load process (ETL - more on it soon). More often than not, the medium of transport would be (flat) file.
In modern architectures, however, the process may involve push or pull, synchronous or asynchronous exchange of data over the network. The medium of transport then usually is APIs or Messages (more on them soon).
ETL - ETL is an umbrella term used for a process that utilises a middleware to perform the data transfer. The middleware may directly connect with systems/databases and relay the data over the network or use file as a medium of transport.
APIs and Messages - APIs and messages (in a simplistic view) allow systems to talk directly with each other. In the real world, there will be an integration tool and a message broker that facilitates this.
The theoretical difference
While application integration deals with connecting systems for the purpose of achieving a business process, data integration deal with combining data for the purpose of making it available for an operational or analytical purpose.
The practical difference
Application integration operates on a single (or few) object/row/line/message/event at a time, while data integration operates on multiple/batch of data at a time.
My simple analogy is this:
- If two people are talking or discussing with each other they covey an idea and wait for a response - operating on a single object at a time. that is application integration
- If you are reading a book and summarising a chapter in your own words - operating on a batch of data. that is data integration
Top comments (0)