·
13 min
Defect management in software testing: process, roles and metrics per ISTQB
Roman Kirchmeier - Autemos
Defect management in software testing is the process a team uses to log reported anomalies, analyze and classify them, decide whether to fix them, and close the defect report. That is how section 5.5 of the ISTQB Foundation Level syllabus describes it (ISTQB CTFL v4.0.1, 2024). The official glossary lists six activities: “recognizing, recording, classifying, investigating, fixing and disposing of defects” (ISTQB Glossary, 2026).
In practice, every defect has a clear path: who reports it, who decides, who fixes it and who runs the confirmation test. Defect management is one sub-process of test management. EU banks and insurers face a second layer: ICT-related incidents in production fall under the DORA regulation, which does not regulate bug tracking in test.
TL;DR: Defect management has four activities: log, analyze and classify, decide, close. Any report may turn out to be a real defect, a false positive or a change request. Defect detection percentage (DDP) measures the share a test level catches. A study of 171 TSP projects found no evidence that late fixes are consistently more expensive (Menzies et al., 2016).
Figure 1: The four core activities of defect management per ISTQB
What is defect management in software testing?
Defect management in software testing is a process that moves every reported anomaly through a fixed workflow from report to closure and classifies it by agreed rules. Per ISTQB, it includes “a workflow for handling individual defects or anomalies” and “rules for their classification” (ISTQB CTFL v4.0.1, 2024).
The terms form a chain. A person makes an error (mistake), which produces a defect (fault, bug) in code or a document. When the defect is executed, it may cause a failure: behavior someone can observe. Per the syllabus, environmental conditions can cause failures too (ISTQB CTFL v4.0.1, 2024).
The syllabus advises handling findings from static testing, especially static analysis, in a similar way to findings from test execution.
Is every report a real defect?
A report starts as an anomaly, and analysis decides what it is. The syllabus says reported anomalies “may turn out to be real defects or something else (e.g., false-positive result, change request)” (ISTQB CTFL v4.0.1, 2024).
This split has practical consequences. A team that counts every report as a bug inflates its defect statistics and sends developers hunting for faults that don't exist in the product. Good defect management keeps separate end states for duplicates, rejected reports and change requests.
How does the ISTQB defect management process work?
The ISTQB workflow has four core activities: log the reported anomalies, analyze and classify them, decide on a response (fix it or keep it as it is), and close the defect report (ISTQB CTFL v4.0.1, 2024). Between the decision and closure, real projects add the fix, confirmation testing and regression testing.
Figure 2: Workflow from logging to closure, with typical roles
Log: The reporter records the anomaly as a defect report with status “open”. Tools often fill in identifier, date, author and initial status automatically.
Analyze and classify: Can the failure be reproduced? Does a report already exist? Is it a product defect, a false positive or a change request? Severity gets set here.
Decide on a response: fix, defer, reject or pass on as a change request. This step sets the priority to fix.
Fix: Development changes code or configuration and references the change in the report.
Confirmation test: “Confirmation testing confirms that an original defect has been successfully fixed.” (ISTQB CTFL v4.0.1, 2024)
Regression test: It checks that the change, including the fix, caused no adverse consequences elsewhere.
Close: The report gets status “closed” and stays on record as evidence.
Steps 1-3 and 7 come from syllabus section 5.5, steps 5 and 6 from section 2.2.3; step 4 is the practice in between. The syllabus adds: “The process must be followed by all involved stakeholders.” (ISTQB CTFL v4.0.1, 2024) A developer who reports bugs over chat bypasses it.
Who does what in defect management?
Four roles carry the process: the reporter, the triage group that reviews and prioritizes, the developer who fixes, and the tester who confirms. Triage is a practitioner term from tools like Bugzilla and Azure DevOps. Microsoft's docs say: “Typically, the project owner runs the bug triage meetings.” (Microsoft Learn, 2026)
Reporter: tester, developer, business user or operations. At Mozilla, the QA team sets an initial severity for bugs it files or reproduces (Mozilla triage policy, 2026).
Triage group: product owner, test manager and development review new reports, mark duplicates and set priority. Mozilla requires a comment explaining the rationale for any severity change.
The table maps each step to an artifact and a typical role. The role mapping is a common pattern, not an ISTQB requirement.
Step | Artifact | Typical role |
|---|---|---|
Log | Defect report, status “open” | Tester, developer, business user |
Analyze and classify | Severity, duplicate link | Triage group |
Decide on a response | Priority, status “deferred” or “rejected” | Product owner or project lead |
Fix | Code change referencing the report | Development |
Confirmation test | Test result, status “awaiting confirmation testing” | Test team |
Regression test | Test run, test report | Test team, CI pipeline |
Close | Status “closed” | Test team or triage group |
What goes into a defect report?
A defect report (bug report) is “Documentation of the occurrence, nature, and status of a defect,” per the ISTQB glossary (ISTQB Glossary, 2026). The syllabus names three objectives: give those responsible enough information to resolve the issue, track the quality of the work product, and provide ideas to improve the development and test process.
The syllabus lists 11 content items, from a unique identifier, test environment and steps to reproduce to expected and actual results, severity, priority, status and references (ISTQB CTFL v4.0.1, 2024). Templates live in ISO/IEC/IEEE 29119-3, which calls them incident reports.
Steps to reproduce matter most. Mozilla's Bugzilla guidelines put it plainly: “Steps to reproduce are the most important part of any bug report.” (Mozilla Bug Writing Guidelines, 2026) The same guidelines ask for a summary under 60 characters that explains the problem and leaves out your suggested solution.
Research backs this up. A survey of Apache, Eclipse and Mozilla developers and users drew 466 responses and found “an information mismatch between what developers need and what users supply.” Developers rated steps to reproduce, stack traces and test cases most helpful, and users found those the hardest to provide (Zimmermann et al., IEEE TSE, 2010).
For a complete template with a field list and a filled-in banking example, see our guide on how to write a defect report.
How do defect status, severity and priority fit together?
Defect status shows where a report sits in the workflow, severity describes the degree of impact, and priority sets how urgently it gets fixed. The syllabus gives example statuses: open, deferred, duplicate, waiting to be fixed, awaiting confirmation testing, re-opened, closed and rejected (ISTQB CTFL v4.0.1, 2024).
Which states a team uses, and how Jira, Azure DevOps or Bugzilla name them, is covered in our article on the defect life cycle and its states. Microsoft's rule: “Don't reopen closed bugs for regressions.” Open a new bug and link it to the original with a Related link (Microsoft Learn, 2026).
Reopened bugs happen often enough to track. In four Eclipse projects, “6%-10% of total bugs will lead to reopening eventually” (Mi & Keung, EASE, 2016). A team that uses the “re-opened” status consistently can measure that share in its own project.
How do severity and priority differ?
Severity measures impact; priority measures urgency. The official glossary defines severity as “The degree of impact that a defect has on the development, testing or operation of a component or system.” (ISTQB Glossary, 2026) Priority reads, more generically, “The level of importance assigned to a task according to specific criteria.” (ISTQB Glossary, 2026) An older wording mentioning “business importance” still circulates on mirror sites.
The two values can diverge. Microsoft's example: a rarely used link that crashes the application gets Severity 2 (High) and Priority 3 (Microsoft Learn, 2026). Our comparison of defect severity and priority sets the Mozilla, Azure DevOps, Jira and CVSS scales side by side.
How do teams find the root cause of a defect?
Root cause analysis (RCA) is the process a team uses to find the root cause of a defect; the ISTQB glossary calls it “The process used to identify the root cause of a defect.” (ISTQB Glossary, 2026) Per the syllabus, RCA “is typically performed when a failure occurs or a defect is identified.”
The syllabus states the benefit with care: “It is believed that further similar failures or defects can be prevented or their frequency reduced by addressing the root cause” (ISTQB CTFL v4.0.1, 2024). An illustration: a test fails, triggered by a rounding error. The root cause may be an ambiguous requirement for the rounding rule.
Google's blameless postmortem rule carries over: find the contributing causes “without indicting any individual or team for bad or inappropriate behavior” (Google SRE Book, 2016).
Methods such as the 5 Whys and Ishikawa diagrams, applied to failed tests, are covered in our guide to root cause analysis for software defects.
Which metrics show whether defect management works?
The ISTQB syllabus lists defect metrics as the “number and priorities of defects found/fixed, defect density, defect detection percentage” (ISTQB CTFL v4.0.1, 2024). The glossary adds “escaped defect” for defects a test activity should have caught. Our test reporting article shows how to report them.
Figure 3: Median fix times from 171 TSP projects (Menzies et al. 2016)
How do you calculate defect detection percentage (DDP)?
The glossary defines DDP as “The number of defects found by a test level, divided by the number found by that test level and any other means afterwards.” (ISTQB Glossary, 2026) As a formula: DDP = defects found by the test level ÷ (defects found by the test level + defects found later) × 100.
Worked example (illustration, not measured data): system testing finds 90 defects, and production turns up 10 more. DDP = 90 ÷ (90 + 10) × 100 = 90%. The number settles only after an observation period, since late defects change it retroactively.
What is an escaped defect, and what does defect density measure?
An escaped defect is “A defect that is not detected by a test activity that is supposed to find it,” per the glossary (ISTQB Glossary, 2026). The common term “defect leakage” is industry usage and has no entry in the ISTQB glossary.
Defect density is the number of defects per unit size of a work product (ISTQB Glossary v3.4 via imbus, 2020). In practice, teams count per 1,000 lines of code (KLOC) or per function point. Comparing teams only makes sense with the same unit and the same counting rule.
Is the 1:10:100 rule true?
The 1:10:100 rule, which says a defect gets ten times more expensive with each phase, is a historical rule of thumb, not an established law. It traces back to Boehm (1981), who reported a 100-fold increase for larger projects. Menzies and colleagues tested it on 171 projects from 2006 to 2014 with 47,376 defect logs (Menzies et al., arXiv/EMSE, 2016).
Their result: “We found no evidence for the delayed issue effect.” Coding defects took a median of about 10 minutes to fix at code review and 13 minutes at system test. Requirements defects took about 13 minutes at requirements review and 33 at integration test. The data comes from the Team Software Process (TSP), a disciplined method one of the authors helped develop.
That doesn't prove late defects are never more expensive. The authors suggest the effect may be a historical relic that occurs only in certain kinds of projects. If you want to justify testing early, fix times from your own defect tracker make a stronger case than a factor of 100.
What applies to defect management in regulated financial firms?
DORA, Regulation (EU) 2022/2554, governs the management of ICT-related incidents at EU financial entities and has applied since 17 January 2025; it does not regulate bug tracking in test. A software defect becomes DORA-relevant once it causes an ICT-related incident. Article 17(2) requires financial entities to “record all ICT-related incidents and significant cyber threats” (EUR-Lex, DORA, 2022).
Figure 4: Defect management in test compared with ICT incident management under DORA
The same article requires that root causes are identified, documented and addressed. Article 17(3)(b) calls for procedures to classify incidents “according to their priority and severity and according to the criticality of the services impacted.” For major ICT-related incidents, Article 19(4)(c) requires a final report “when the root cause analysis has been completed” (EUR-Lex, DORA, 2022).
Swiss banks fall under FINMA Circular 2023/1, which covers the management of operational risks, including ICT risks, at banks. DORA reaches Swiss institutions only through EU business or EU subsidiaries. Further obligations for test processes in banks are covered in our article on test automation in regulated banks.
Attribute | Defect management in test | ICT incident management under DORA Art. 17 |
|---|---|---|
Subject | Anomalies and defects in test objects | ICT-related incidents and significant cyber threats |
Basis | ISTQB syllabus (industry standard, not law) | Regulation (EU) 2022/2554, applies since 17 Jan 2025 |
Classification | Severity and priority | Priority, severity, criticality of services impacted |
Causes | Root cause analysis as needed | Root causes identified, documented, addressed (Art. 17(2)) |
Closure | Confirmation test, status “closed” | For major incidents, final report after root cause analysis (Art. 19) |
When a production incident traces back to a defect, the incident ticket links to the defect report, and the confirmation test documents the fix. Google observed internally that roughly 70% of outages are caused by changes to a live system (Google SRE Book, 2016). Regression tests after every change target exactly that.
How do automated test runs feed defect management?
A failed automated test is an anomaly, not yet a confirmed product defect: the cause may sit in the product, the test script, the environment, the test data or flakiness. Google reported “about 1.5% of all test runs reporting a 'flaky' result” and that “Almost 16% of our tests have some level of flakiness” (Google Testing Blog, 2016).
The same post found that about 84% of observed transitions from pass to fail involved a flaky test. A team that files every red run as a bug fills its tracker with false alarms. So each failure gets sorted into one of five classes before anyone writes a defect report:
Product defect: the application behaves incorrectly. This one goes into a defect report.
Test defect: the script, locator or expected result is wrong.
Environment: a service is down, configuration is wrong, a certificate expired.
Test data: a record was consumed, changed or is missing.
Flaky: the test gives different results on unchanged code.
What does Autemos contribute, and what doesn't it do?
Autemos is AI-assisted software for functional test automation across web, mobile, API and desktop. Autemos is not a defect tracker. It doesn't replace Jira, Azure DevOps or Bugzilla, doesn't create bug tickets automatically, doesn't set severity or priority, and doesn't perform root cause analysis. It's built by selementrix GmbH in Baar (Zug), Switzerland.
What Autemos provides is evidence from the test run for the defect report:
The visual test workflow is the step sequence that led to the failure. Exported Playwright or Appium code reruns outside Autemos.
Results land in Autemos reports and Allure reports; logs and telemetry can go to Splunk or Dynatrace.
Test executions sync both ways with Jira/Xray via the Jira/Xray integration. Our article on Xray in Jira covers the mechanics.
Every self-healing action appears in the report and can be reviewed, rejected or rolled back. A changed locator shows up as a test-side change, not as a product defect.
Per-environment configuration (DEV, TEST, STAGING, PROD) helps record the environment where the failure occurred. The audit trail records who changed what; role-based access control sets who may change it.
A fixed defect can become a regression test: the AI chat drafts a test workflow from the Jira ticket, and the team approves every AI-generated step before it runs.
Frequently asked questions
What is the difference between defect management and incident management?
Defect management handles anomalies and defects in test objects; incident management handles disruptions in live operation. At an EU financial entity, a software defect becomes DORA-relevant once it causes an ICT-related incident in production.
Who should close a defect report?
The test team usually closes a report after a passed confirmation test; the triage group closes duplicates and rejected reports. ISTQB prescribes no specific role, but it requires all stakeholders to follow the same process. Set the rule in your tool's workflow so it applies to every report.
What is the difference between confirmation testing and regression testing?
Confirmation testing checks that one specific defect is fixed; regression testing checks that the change broke nothing else. Per the syllabus, regression testing “confirms that no adverse consequences have been caused by a change, including a fix that has already been confirmation tested” (ISTQB CTFL v4.0.1, 2024). Both come before the report is closed.
Does DORA require a bug tracking tool?
No, DORA prescribes neither a bug tracking tool nor a defect report format. The regulation requires financial entities to record ICT-related incidents, classify them by priority and severity, and identify, document and address root causes. When an incident traces back to a software defect, the linked defect report and confirmation test document cause and fix.
Does Autemos replace a defect tracker like Jira?
No, Autemos doesn't replace a defect tracker. It automates functional tests and provides reproducible test runs, Allure reports and two-way sync of test executions with Jira/Xray. You keep managing defect reports, severity, priority and status in Jira, Azure DevOps or Bugzilla.
Conclusion
Defect management in software testing consists of four activities from the ISTQB syllabus: log, analyze and classify, decide, close. Clear roles, a defect report with reproducible steps and separate values for severity and priority keep the process clean.
Measure the process with DDP, escaped defects and defect density, and justify early testing with fix times from your own tracker. In regulated financial firms, keep test defect reports apart from ICT incidents under DORA, and link the two wherever a defect caused an ICT-related incident.
Classify each red run before it becomes a bug. If you want to see how reproducible test runs and Jira/Xray sync fit into your defect management, talk to us.
