Medium-sized businesses sit in an awkward place for integration. They run enough systems for point-to-point connections to become a tangle, but not enough to justify the enterprise integration platforms built for very large organisations. This article describes the patterns that work at that scale, and how to decide between them.
The problem with point-to-point
The first integration most organisations build is direct: system A calls system B's API, or exports a file that system B imports overnight. It works, it is cheap, and for the first few connections it is the right answer.
The trouble starts around the tenth connection. Each system now has several interfaces that nobody documented, changes to one system break integrations to others, and the person who understands the whole picture has become a single point of failure. Typical symptoms are duplicate customer records, reports that disagree, and finance closing a week late while someone reconciles spreadsheets.
The answer is not necessarily a platform. It is a set of decisions made deliberately.
Decide who owns each piece of data
Before choosing any technology, agree which system is the source of truth for each type of record. Customers may live in the CRM, products in the ERP, employees in the HR system. Every other system holds a copy, and copies are updated from the source, never edited locally.
Writing this down is the single highest-value integration activity, and it costs nothing but meetings. It settles most later arguments about which direction data should flow.
Four patterns that suit this scale
Scheduled synchronisation
A job runs on a schedule, reads changed records from the source and updates the copies. It is simple, easy to monitor and tolerant of downtime on either side. It suits data that does not need to be current to the second: product catalogues, price lists, employee directories, master data generally.
The discipline required is idempotency: running the job twice must produce the same result as running it once. Design for it from the start and reruns after a failure become routine rather than risky.
Event-driven updates
When a record changes, the source system publishes an event and interested systems react. This gives near-real-time updates without each consumer polling the source, and it decouples the systems: a consumer can be offline and catch up later.
Cloud providers offer managed queues and event buses that make this practical without running your own messaging infrastructure. The discipline here is designing events carefully. An event should carry enough information to be useful and a stable identifier so consumers can fetch the rest if they need it.
API composition
Some processes need information from several systems at once, for example a customer portal that shows orders from the ERP, tickets from the service desk and contracts from the document store. A small integration service can compose those calls behind one API, applying access rules consistently.
This pattern keeps the source systems simple and puts the business logic in one testable place. Its risk is that the composition service becomes a second monolith; keep each service focused on one process or one audience.
Managed file transfer
Files are not obsolete. Banks, logistics providers and many older systems exchange data as files, and doing so reliably is a legitimate integration pattern. What distinguishes a good implementation is management: automated transfers over secure channels, validation of every file received, and alerts when an expected file does not arrive.
Choosing between them
Ask three questions for each integration:
- How current does the data need to be? Overnight is fine for most master data; minutes for orders; seconds only for a few operational cases. Choose the simplest pattern that meets the real need, not the theoretical one.
- What happens when one side is down? Scheduled jobs and event queues tolerate outages; synchronous API calls do not. If the consumer must keep working while the source is unavailable, choose a pattern that buffers.
- Who will operate it? A pattern your team can monitor and fix at two in the morning is better than a more elegant one they cannot.
Interface contracts before code
Every integration should have a short written contract before anyone builds it: the fields exchanged, their formats and permitted values, which system owns each field, how errors are reported, and what the retry behaviour is. Contracts are what allow one system to be upgraded or replaced without breaking the rest.
They are also the basis for testing. An integration test that checks the contract, not just that the code runs, catches the changes that matter.
Make it operable
An integration that nobody can see is an incident waiting to happen. Every pattern above needs three operational properties:
- Monitoring. How many records moved, how many failed, how long the job took, and whether the expected run happened at all. Missing runs are the failures most often overlooked.
- Reconciliation. A periodic check that the copies match the source, with a report of differences. This catches silent failures that monitoring misses.
- Runbooks. What to do when a job fails, a queue backs up or a file does not arrive, written for the person on duty rather than the person who built it.
When a platform is justified
An integration platform earns its cost when the number of interfaces, the volume of data or the compliance requirements outgrow what a small set of services and jobs can handle. Signs include more than a few dozen interfaces, several teams building integrations independently, or audit requirements for every data movement. Before that point, the patterns above, applied consistently and documented, deliver most of the benefit at a fraction of the cost.
Summary
Good integration at medium scale is less about technology than about decisions: who owns the data, which pattern fits each flow, what the contract is and who operates the result. Make those decisions explicitly and the technology choices become straightforward. Leave them implicit and no platform will rescue the outcome.