Issue with Transactions service
Incident Report for Tink
Postmortem

Introduction

Between November 17th 16:50 (CEST) and November 18th 17:14 (CEST), some customers experienced degraded functionality in Transaction Reports, resulting in some user reports returned as empty, or incomplete, even though the user had transactions.We know how important it is to our customers that we provide a highly available and reliable service and we sincerely apologize for the disturbances this incident caused.

Root Cause Analysis

The root cause of this outage was a faulty code change that was deployed, which could cause Transaction Report to be completed before bank side processing was actually completed. This would lead to empty or incomplete transaction reports. This issue did not impact all users:

  • For some users the bank side authentication completed fast enough for the issue not to occur.
  • Customers with specific configuration were unaffected and reports were generated as expected.

Due to gaps in our end to end testing of the transaction flow this was not caught in our release process. Because of improper monitoring it was not detected fast enough.

Remediation

The error was detected, and the error was fixed and service got restored.

Follow-up actions

Tink’s standard operating procedure is to continuously improve our reliability and learn from incidents. As a consequence of this incident we will focus on the following actions over the coming weeks to prevent a similar scenario from occurring again:

  • Review and improve our end to end tests of the Transactions product
  • Review and improve our monitoring of the Transactions product
Posted Nov 21, 2022 - 19:18 CET

Resolved
This incident has been resolved.
Posted Nov 18, 2022 - 18:00 CET
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Nov 18, 2022 - 17:38 CET
Identified
The issue has been identified and we are currently working on a solution.
Posted Nov 18, 2022 - 15:11 CET
This incident affected: Transactions.