A black-box test derives test cases from the specification, and a white-box test derives them from the internal structure of the test object, such as its code. That's how the ISTQB Certified Tester Foundation Level syllabus separates the two approaches (ISTQB CTFL v4.0.1, 2024). Black-box techniques check whether the software behaves as required. White-box techniques show which statements and branches your tests execute.

Many comparison tables still list use case testing as a black-box technique, yet ISTQB removed it from the Foundation syllabus in version 4.0. In IT security, “black box” has a second meaning. And the research supports combining both approaches.

TL;DR: Black-box tests come from requirements and user stories; white-box tests come from code and architecture. ISTQB CTFL 4.0 names four black-box techniques (equivalence partitioning, boundary value analysis, decision tables, state transitions) and two white-box techniques (statement and branch testing). In studies since 1987, no technique consistently finds more defects, so combine specification-based functional tests with unit tests that measure code coverage.

Figure 1: Black-box and white-box tests by test basis

What is the difference between a black-box test and a white-box test?

The difference between black-box and white-box testing lies in the test basis: black-box techniques are specification-based, white-box techniques are structure-based. According to ISTQB, black-box testing “is specification-based and derives tests from documentation not related to the internal structure of the test object” (ISTQB CTFL v4.0.1, 2024). White-box testing uses code, architecture, work flows and data flows.

Timing follows from that. You can write specification-based test cases once a specification exists, and structure-based ones only after design or implementation. The syllabus notes: “if the implementation changes, but the required behavior stays the same, then the test cases are still useful” (ISTQB CTFL v4.0.1, 2024).

What is a black-box test?

A black-box test is a test technique that derives test cases from the specified behavior of a system without looking at its internal structure. The test basis is requirements, specifications and user stories. The tester knows the inputs and the expected results; the code plays no part in test design. The approach fits every test level.

Figure 2: Test techniques in the ISTQB CTFL 4.0 syllabus

ISTQB CTFL 4.0 lists four common black-box techniques (ISTQB CTFL v4.0.1, 2024):

• Equivalence partitioning: divides data into partitions the test object processes the same way. Per the syllabus, “one test for each partition is sufficient.”

• Boundary value analysis: tests the edges of ordered partitions, as 2-value or the more rigorous 3-value BVA.

• Decision table testing: covers business rules as combinations of conditions.

• State transition testing: covers states and transitions, such as an account that is active, blocked or closed.

The syllabus gives a short reason for the boundary focus: developers “are more likely to make errors with these boundary values.” Our guide to equivalence partitioning and boundary value analysis walks through the derivation step by step.

Specification-based testing needs domain knowledge and test design skill. Gaps in the requirements turn into gaps in the test cases. The functional testing guide covers how to check business behavior systematically.

What is a white-box test?

A white-box test is a structure-based test technique that derives test cases from the implementation: code, architecture, work flows or data flows (ISTQB CTFL v4.0.1, 2024). The Foundation syllabus covers two code-related techniques, statement testing and branch testing. Statement coverage is the share of statements executed; branch coverage is the share of branches taken, such as both outcomes of an if condition.

Its strength is the view of the entire implementation. ISTQB notes that this makes defects easier to find even when the specification is vague, outdated or incomplete.

If the software never implements a requirement, there's no code for a test to execute. The syllabus says white-box testing “may not detect the resulting defects of omission” (ISTQB CTFL v4.0.1, 2024).

White-box testing isn't limited to component testing; the syllabus mentions white-box techniques at higher test levels, such as API testing. The unit testing article shows how developers write structure-based tests, and the piece on test coverage explains why a high coverage figure says little about test quality on its own.

What changed in the ISTQB 4.0 syllabus?

With CTFL 4.0, ISTQB removed use case testing from the Foundation black-box techniques and replaced decision testing with branch testing (ISTQB CTFL v4.0.1, release notes, 2024). Many comparison pages and current AI answers still show the old state. The changes:

• Use case testing: removed from the Foundation syllabus, still part of the Advanced Test Analyst syllabus.

• Decision testing and coverage: replaced with branch testing and coverage.

• Integration testing: split into component integration testing and system integration testing, which gives five test levels.

• Grey-box: appears nowhere in CTFL v4.0.1 or the German Lehrplan 4.0.2. Grey-box isn't a category in the ISTQB Foundation syllabus.

The syllabus lists functional, non-functional, black-box and white-box testing as four separate test types that “can be applied to all test levels” (ISTQB CTFL v4.0.1, 2024). “Functional test = black-box test” is a simplification: functional describes what you check, black-box describes where the test cases come from.

The syllabus adds a third group, experience-based test techniques: error guessing, exploratory testing and checklist-based testing. According to ISTQB, they can detect defects that black-box and white-box techniques miss.

When should you use each technique?

Use specification-based tests once requirements exist and system behavior matters; use white-box tests when developers need to cover the code paths of a component. Most projects need both:

Figure 3: Which technique fits which situation

• Requirements are ready, code isn't: design specification-based test cases now.

• System and acceptance testing: black-box, since overall behavior against the specification is what counts. The system testing guide goes deeper.

• New or changed component: white-box unit tests, measured by statement and branch coverage.

• Interfaces: both, as the API testing guide shows.

• Code coverage required as evidence: white-box only. ISTQB states: “Performing only black-box testing does not provide a measure of actual code coverage.” (ISTQB CTFL v4.0.1, 2024)

How do you combine both in a project?

1. Derive test cases from requirements and user stories using equivalence partitions, boundary values and decision tables.

2. Have developers write unit tests in parallel and measure statement and branch coverage.

3. Review unexecuted code: is a test missing, or is the code dead?

4. Trace every requirement to at least one test case, so requirements that were never built stand out.

5. Add exploratory tests and automate stable functional tests as regression tests.

Where does Autemos fit?

Autemos automates functional tests on the black-box side. Teams build them from requirements, Jira tickets or user stories, with the AI Recorder or in visual test workflows, and run them against web, mobile, API and desktop. The team approves every AI-generated step before it runs. Web tests export as Playwright code in Java or TypeScript and keep running without Autemos.

Autemos doesn't measure code coverage and doesn't replace white-box unit tests; those stay with your developers. When requirements or functional test cases already sit in Jira, specification-based testing starts there.

What do black-box, white-box and grey-box mean in penetration testing?

In penetration testing, black-box, grey-box and white-box describe how much the tester knows about the target system; they share only the name with the ISTQB split by test basis. NIST defines black box testing as “A test methodology that assumes no knowledge of the internal structure and implementation detail of the assessment object” (NIST SP 800-53A Rev. 5, CSRC glossary).

Figure 4: Black-box in test design (ISTQB) and in penetration testing (NIST)

For gray box testing, NIST assumes “some knowledge” of the internal structure (NIST CSRC); for white box testing, “explicit and substantial knowledge” (NIST CSRC). The OWASP Web Security Testing Guide v4.2 says penetration testing is “commonly known as black-box testing or ethical hacking” (OWASP WSTG v4.2).

The German BSI guide to IS penetration tests separates blackbox tests with little information from whitebox tests with internal information, and tends to recommend whitebox tests (BSI IS penetration testing guide).

Autemos does no security or penetration testing. When a pentest provider talks about black-box, they mean what their team knows going in, not how test cases are designed.

Which technique finds more defects?

In experiments since 1987, neither specification-based nor structure-based testing consistently finds more defects; results depend on the program and the participants. In 1987, Basili and Selby compared code reading, functional testing and structural testing with professionals and students (Basili & Selby, IEEE TSE, 1987). Among professionals, code reading found the most faults and functional testing found more than structural testing, with results varying by program and group.

Juristo, Moreno and Vegas reviewed 25 years of testing technique experiments and found that knowledge about testing techniques was still immature (Juristo, Moreno & Vegas, Empirical Software Engineering, 2004).

A study by Vegas, Riofrío, Marcos and Juristo found a second problem: participants misjudged which technique works best. In the authors' words, “participants' perceptions are wrong and that this mismatch is costly in terms of quality” (Vegas et al., Empirical Software Engineering, 2020).

So combine specification-based tests against the requirements, white-box tests in the code and exploratory sessions.

Frequently asked questions

Is a black-box test the same as a functional test?

Specification-based testing isn't the same as functional testing; ISTQB lists them as separate test types. Functional describes what gets tested. Black-box describes where the test cases come from. In practice, teams often design functional tests from the specification.

Which black-box techniques does ISTQB CTFL 4.0 name?

ISTQB CTFL 4.0 names four black-box techniques: equivalence partitioning, boundary value analysis, decision table testing and state transition testing. Use case testing was dropped in version 4.0 and now sits only in the Advanced Test Analyst syllabus.

Is grey-box testing an ISTQB term?

Grey-box isn't a category in the ISTQB Foundation syllabus, in either CTFL v4.0.1 or the German 4.0.2. In security testing, NIST and OWASP use gray-box for tests with partial knowledge of the target.

Who performs black-box and white-box tests?

Testers, analysts and business users usually design specification-based tests; developers usually perform white-box tests. People who know the requirements test against the specification, and people who know the code cover statements and branches.

Does specification-based testing measure code coverage?

Specification-based testing doesn't produce a code coverage figure. It measures coverage against the specification: partitions, boundary values, decision table rules or state transitions. Statement and branch coverage come only from white-box tests, typically in developers' unit tests.

Conclusion

Specification-based (black-box) testing checks whether the software does what the specification requires and catches requirements that were never built; a white-box test shows which code stays untested. ISTQB CTFL 4.0 puts four techniques in the black-box group and two in the white-box group; grey-box and use case testing aren't in the Foundation syllabus.

Studies since 1987 don't favor one approach, so combine both and add exploratory testing. Autemos covers the black-box side: functional tests built from requirements and run on web, mobile, API and desktop. Code coverage and unit tests stay with your developers. If you'd like to see how that works in your projects, talk to our team.