Keeping an AI Document Processing Platform Online: 17 Months and Counting

The problem

A Dutch software company runs a production platform that reads invoices, orders and order confirmations and pushes them into customer ERP systems. Their customers feel every minute of downtime. They needed someone accountable for the infrastructure, with visibility their own leadership could see rather than a status report they had to ask for.

• •The platform is customer-facing, so an outage is visible to the client's customers and not just internally
• •The database ran inside a container without a persistent volume, which is a data-loss risk that had to be handled without a disruptive re-architecture
• •The engagement is deliberately low-hours. Everything had to be built so it keeps working without someone watching it

How we ran it

1. 01Started with backups, because an unmonitored system that is backed up recovers, and a monitored system that is not backed up does not
2. 02Three layers rather than one: local, off-site object storage, and a separate archive, so no single provider or credential failure loses everything
3. 03Built the observability dashboard for the client to read, not for us. Trust in a low-hours retainer comes from the client being able to check for themselves
4. 04Alerts sent to the client team directly, including leadership. If something breaks at 2am, the people who care hear about it at the same time we do
5. 05Documented the risky parts, in particular the containerised database without a persistent volume, so the risk is a known and tracked item rather than a surprise

Ongoing since December 2024

months of continuous engagement

17+months of continuous engagement

independent backup layers

3independent backup layers

alert recipients, client leadership included

7alert recipients, client leadership included