·
13 min
Testing in Regulated Industries: Compliance, Audit and Automation
Roman Kirchmeier - Autemos
In regulated industries, what matters is not only whether software works, but whether you can prove it without gaps. Banks, insurers, pharmaceutical and medtech companies do not test for themselves — they test for supervisors, auditors and, ultimately, their customers. A single test run that cannot be traced can sink an audit. This guide maps the regulatory landscape and shows how to build a test strategy that is both auditable and economically automatable.
In brief: Testing in regulated industries means documenting every check so it is traceable, reproducible and tamper-evident. Since 17 January 2025, DORA requires banks and insurers to run a risk-based testing programme with at least yearly tests of all critical ICT systems — making automation and audit trails a duty, not a nice-to-have.
Figure 1: Testing in regulated industries – burden of proof, not just a function check.
What does testing in regulated industries mean?
Testing in regulated industries is the discipline of checking and documenting software so the results withstand an external audit. The difference from ordinary software testing lies not in the test techniques but in the burden of proof: every test case must map to a requirement, every execution must be logged traceably, every result must be reproducible.
This applies to organisations whose software is supervised — financial services, insurers, pharma and medtech manufacturers, critical infrastructure, and increasingly operators of high-risk AI systems. For them, a passed test without clean evidence is worthless. We cover the foundations in Software Testing: Fundamentals, Process and Methods; this guide builds on them and focuses on the regulatory context.
Why regulated industries need different test standards
Figure 3: The cost of poorly tested software – the TSB fine and the CISQ damage estimate.
The reason is the consequence of failure. Poor software quality cost the US alone at least USD 2.41 trillion in 2022 (CISQ, 2022). In regulated sectors, fines, reputational damage and the loss of operating licences are added to the direct costs.
A telling example is the 2018 TSB bank migration: UK regulators FCA and PRA fined the bank GBP 48.65 million in December 2022. An independent review noted that two data centres "weren't tested at all" before the migration (FCA, 2022). The problem was not the code alone, but the missing, demonstrable test assurance.
The regulatory landscape: DORA, FINMA, EU AI Act and GxP
Figure 2: The regulatory landscape and its testing duties at a glance.
The biggest shift of recent years: in Germany and the EU, the old IT supervisory circulars are obsolete. BaFin's BAIT, VAIT and KAIT circulars have been replaced by DORA, directly applicable since 17 January 2025; BAIT will be fully repealed by the end of 2026 (Bundesbank, 2025). Anyone still citing VAIT is working from outdated requirements.
Framework | Scope | What it requires for testing |
|---|---|---|
DORA (EU 2022/2554) | Banks, insurers, financial services | Risk-based testing programme, at least yearly tests of critical ICT systems, TLPT every 3 years |
FINMA Circular 2023/1 | Swiss banks | Separation of test and production, change management, protection of critical data |
EU AI Act (2024/1689) | High-risk AI systems | Conformity assessment, automatic logging across the lifecycle (from 2 August 2026) |
GxP / 21 CFR Part 11, Annex 11 | Pharma, medtech | Validation of computerised systems, audit trails, data integrity |
Solvency II (2009/138/EC) | Insurers | Data for technical provisions must be accurate, complete and appropriate |
DORA sets the pace. Article 24 requires a "sound and comprehensive" testing programme; Article 25 explicitly names end-to-end testing, performance testing and source code reviews as expected test types; Article 26 mandates threat-led penetration testing (TLPT) at least every three years (EUR-Lex 2022/2554, 2022). Regular, documented testing is therefore no longer an internal best practice but a supervisory obligation.
Tamper-evident test documentation and audit trails
The heart of any compliance test strategy is the evidence. An audit trail must show who tested what and when — and that the result was not altered afterwards. In 21 CFR Part 11, the FDA explicitly requires "secure, computer-generated, time-stamped audit trails" (eCFR). The principle that connects all industries is ALCOA+: data must be attributable, legible, contemporaneous, original and accurate — plus complete, consistent, enduring and available (MHRA, 2018).
How to produce such evidence without drowning your team in documentation is covered in depth in Audit Trails in Software Testing: Tamper-Evident Documentation.
Computer System Validation in pharma and medtech
Pharma and medtech follow their own logic: computer system validation (CSV). Here, something fundamental changed in 2025. The FDA published its "Computer Software Assurance" (CSA) guidance in final form on 24 September 2025 — so it is no longer at draft stage (Federal Register, 2025). CSA shifts the focus from exhaustive documentation toward risk-based, critical testing and explicitly permits unscripted testing.
The details — including GAMP 5 (Second Edition), the IQ/OQ/PQ model and IEC 62304 for medical devices — are covered in GxP Testing and Computer System Validation (CSV).
GDPR-compliant test data
An often underestimated compliance risk lies in the test data. Copying personal production data into test or QA environments usually breaches the data-minimisation principle under Article 5 GDPR. The Spanish data protection authority AEPD warns that pre-production environments are often less well protected than production — and explicitly recommends synthetic test data (AEPD, 2022).
Important: pseudonymised data remains personal data and stays in scope of the GDPR (EDPB, 2025). How to provision test data that is both lawful and realistic is shown in GDPR-Compliant Test Data: Test Data Management in Regulated Industries.
Test automation in banks and insurers
Banks and insurers face a double burden: high regulatory demands meet historically grown core systems. In the World Quality Report 2024-25, 64 percent of respondents named legacy systems as a barrier to automation (Capgemini, 2024) — a vendor survey, but one that matches the reality of these sectors.
For the banking side, we worked out the requirements in Test Automation in Regulated Banks: audit-ready under DORA and FINMA; the insurance perspective — Solvency II, legacy cores, premium calculations — is covered in Test Automation in the Insurance Industry. AI can noticeably reduce the effort for repeatable tests here, as the practical guide to AI-powered test automation shows — but it replaces neither strategy nor expertise.
How to build an audit-ready test strategy
Figure 4: Five steps to an audit-ready test strategy.
An auditable test strategy comes not from more documentation but from structured traceability. These steps have proven effective:
Anchor requirements: Build a requirements traceability matrix that maps each test to a regulatory or business requirement (definition per ISTQB).
Prioritise by risk: Test critical and important functions most intensively — DORA requires it, and it saves effort on non-critical components.
Automate audit trails: Have test runs, versions and results logged automatically and tamper-evidently, instead of collecting screenshots.
Defuse test data: Replace production data with synthetic or anonymised test data.
Eliminate flaky tests: Non-deterministic tests undermine the evidential value of your records; self-healing locators help keep tests stable.
How to turn these steps into an end-to-end, automated workflow is shown in our overview of test workflows.
Frequently asked questions
What does compliance testing mean?
Compliance testing is checking and documenting software so the results satisfy regulatory requirements and withstand an audit. The focus is on traceability, reproducibility and tamper-evident logging — not just whether the software works.
Which rule applies to IT testing in German banks since 2025?
Since 17 January 2025, DORA applies directly. BaFin's BAIT and VAIT circulars have been replaced; BAIT will be fully repealed by the end of 2026. Banks and insurers therefore align their testing programme with DORA, no longer with the old *AIT circulars.
How often must critical ICT systems be tested?
DORA Article 24 requires at least yearly tests of critical and important functions within a risk-based testing programme. Threat-led penetration testing (TLPT) is additionally mandated for significant institutions at least every three years.
May you use production data in test environments?
Usually not. Using personal production data in test or QA environments generally conflicts with the data-minimisation principle (Art. 5 GDPR). Supervisory authorities recommend synthetic or anonymised test data.
Is the FDA's CSA guidance still a draft?
No. The FDA published "Computer Software Assurance" as final guidance on 24 September 2025. It supersedes Section 6 of the 2002 guidance and follows a risk-based, less documentation-heavy approach.
Conclusion
Regulated industries demand proof, not assertion. Anyone testing in banking, insurance, pharma or medtech must be able to evidence every check traceably, reproducibly and tamper-evidently — driven by DORA, FINMA, the EU AI Act and the GxP rules. The good news: these very requirements can largely be automated today. Structured traceability, risk-based prioritisation and automatic audit trails turn compliance from a brake into a competitive advantage. If you want to automate your test strategy across web, mobile, API and desktop in an audit-ready way, talk to the Autemos team about your specific use case.
