Testing basics (the ISTQB part)
The theory every tester is expected to know. ISTQB is the international testing certificate; the interviewer asked about it first. Everything here is explained with car examples.
- Testing = checking the product to find problems before the customer does.
- A person makes an error → it becomes a defect in the code → you see a failure.
- You can't test everything, so you test the most risky things first.
- Testing has fixed steps: plan → design → run → report.
Think of it like a doctor's check-up. The doctor can't test every cell in your body, so they check the most important things first, in a fixed order.
See it as a picture: Error → defect → failure · Testing process
What testing is, and why we do it
Testing means checking whether software does what it should, and finding problems before customers do. It also gives the team information to decide: "Is this software good enough to release?"
- Testing finds defects so they can be fixed.
- Testing builds confidence in the quality.
- Testing gives information for decisions (release or not).
- Testing prevents defects when done early (reviewing requirements before code exists).
- In cars, testing also helps meet legal and safety standards (ISO 26262, UN regulations).
Error, defect, failure, root cause
ISTQB uses these four words very precisely. This is a favourite basic question.
- Error (mistake)A human does something wrong. The developer writes "≤ 5" instead of "< 5".
- Defect (bug, fault)The mistake stays in the code (or in a document).
- FailureWhen the code runs, you see the wrong behaviour: the snowflake appears at exactly 5.0 °C.
- Root causeThe deeper reason the error happened: the requirement was ambiguous, or the developer misread it.
"A person makes an error, which puts a defect in the code; when that code runs, we observe a failure. Testers observe failures and report them; developers find and fix the defect; and root cause analysis tries to stop the same error happening again."
The 7 testing principles (with car examples)
| # | Principle | Simple meaning | Car example |
|---|---|---|---|
| 1 | Testing shows the presence of defects, not their absence | Tests can prove bugs exist; they can never prove there are none | 1,000 passed tests don't mean the head unit has zero bugs |
| 2 | Exhaustive testing is impossible | You can't test every combination, so you choose smartly | Every temperature × every language × every phone = impossible. Use BVA and risk |
| 3 | Early testing saves time and money | A bug found in the requirement is cheap; found after SOP it is very expensive | Asking "is 5.0 °C included?" during review costs 5 minutes |
| 4 | Defects cluster together | Most bugs are in a few areas | If Bluetooth had 40 bugs last release, test it harder in regression |
| 5 | Tests wear out (pesticide paradox) | Running the same tests again finds no new bugs; update them | Add new test data and new scenarios to the regression suite |
| 6 | Testing is context dependent | Test differently for different products and risks | A brake warning is tested far more strictly than a wallpaper setting |
| 7 | Absence-of-defects fallacy | Bug-free but useless software is still a failure | A perfectly working menu that drivers can't find or use while driving |
"It gave me structure and a common language. For example, I use boundary value analysis to pick a few strong test values instead of many random ones, I use risk and 'defects cluster together' to decide what goes into regression, and when I talk about 'confirmation testing' or 'severity' with colleagues in other countries, we mean exactly the same thing."
Verification vs validation
| Verification | Validation | |
|---|---|---|
| Question | "Did we build the product right?" | "Did we build the right product?" |
| Checks against | The specification / requirements | The real user need |
| Example | Snowflake appears below 5 °C exactly as the requirement says | Drivers actually notice the snowflake and understand it warns of ice |
How software is developed: V-model and Agile
The V-model (very common in automotive)
Automotive projects also follow ASPICE (Automotive SPICE), a process standard based on this V shape. In simple words, ASPICE demands: every requirement has tests, every test traces back to a requirement, results are recorded, and work products are reviewed.
Agile and Scrum
Many teams now combine the V-model with Agile: small pieces delivered often. The most common Agile method is Scrum:
| Scrum word | Meaning |
|---|---|
| Sprint | A fixed work cycle, usually 2 weeks |
| Product backlog | The ordered list of all work still to do |
| Sprint planning | Meeting at the start: what will we do this sprint? |
| Daily stand-up | 15 minutes each day: yesterday, today, blockers |
| Sprint review | Show what was built at the end of the sprint |
| Retrospective | Discuss how to work better next sprint |
| User story | A small feature: "As a driver, I want … so that …", with acceptance criteria |
| Definition of Done | Checklist a story must meet to count as finished (e.g. tested, no open critical bugs) |
Test levels: who tests what, when
| Level | What is tested | Usually by | Car example |
|---|---|---|---|
| Component (unit) testing | One small piece of code alone | Developers | The function that compares temperature with 5 °C |
| Component integration testing | Pieces of code working together | Developers / integrators | Temperature module + icon module |
| System testing | The whole system against its requirements | Testers (you) | The complete cluster on a HIL bench |
| System integration testing | The system together with other systems | Integration testers | Head unit + cluster + phone + other ECUs |
| Acceptance testing | Is it acceptable for the customer / user? | Customer, proxy team | The OEM's acceptance tests before delivery |
Test types: what kind of testing
| Type | Meaning in one line | Example |
|---|---|---|
| Functional | What the system does | Pressing "next" plays the next song |
| Non-functional | How well it does it (speed, stability, usability, security) | Boot time < 10 s; no crash in 24 h |
| Black-box | Test from outside, based on requirements, without looking at code | All your HMI tests |
| White-box | Test based on the code structure | Developer checks every "if" branch is executed |
| Confirmation (retest) | Check a fixed bug is really fixed | Rerun TC-03 on the fix build |
| Regression | Check changes didn't break anything else | Rerun temperature tests after the fix |
| Smoke | Quick check that a new build is testable at all | Boots, main screens open, no crash |
| Sanity | Quick narrow check of a specific change | Just the fixed Bluetooth reconnect |
| Exploratory | Learn and test at the same time, guided by experience | 30 min session "break the media player" |
| Ad-hoc | Informal testing without planning | Random clicking to see what happens |
| Positive / negative | Valid inputs / invalid inputs and error conditions | Correct PIN / wrong PIN |
| End-to-end | A full user journey across systems | Get in car → phone connects → navigate home with music |
| Performance | Speed and timing | Camera within 2 s |
| Endurance / soak | Long run to find slow problems | 24 h music playback, memory leak |
| Usability | Easy to understand and use | Driver finds the setting within 3 taps |
| Localisation | Correct for each language and country | German texts fit, date formats |
| Compatibility | Works with different external devices | 20 phone models for Bluetooth |
Smoke: wide and shallow — "is the whole new build basically alive?" Done on every new build before other testing. Sanity: narrow and focused — "does this particular fix or change work?"
Static vs dynamic testing, and reviews
- Dynamic testing: running the software and checking the behaviour (most of your work).
- Static testing: checking without running anything — reviewing requirements, designs, test cases, code. Finds problems earliest and cheapest.
Review types, from informal to formal: informal review (a colleague has a look) → walkthrough (the author explains it to others) → technical review (experts discuss and decide) → inspection (formal, with roles, checklists and metrics).
Requirement: "The snowflake shall appear below 5 °C and disappear above 4 °C." Review findings: (1) contradiction between 4 and 5 °C; (2) no reaction time given; (3) no behaviour defined for a sensor failure; (4) °F not mentioned. You raise these as review comments or questions before testing. This is static testing.
The test life cycle (STLC), step by step
This is the answer to "What is your process from requirement to delivery?" Learn the order.
| Phase | What you do | Input | Output |
|---|---|---|---|
| 1. Requirement analysis | Read and review requirements; ask questions; check testability in your environment | Requirements, specs | Questions list, testability notes |
| 2. Test planning | Decide scope, approach, resources, benches, schedule, risks, entry/exit criteria | Requirements, project plan | Test plan |
| 3. Test analysis & design | Decide what to test (test conditions) and design test cases using techniques; link to requirements | Requirements, test plan | Test cases, traceability, test data |
| 4. Test implementation / environment setup | Prepare bench, flash build, simulation config, test data, automation; smoke test | Build, test cases | Ready environment, test suites |
| 5. Test execution | Run tests, record results, raise defects, retest, regression | Everything above | Results, defect tickets, logs |
| 6. Test monitoring & reporting | Track progress and report status regularly | Results | Status reports |
| 7. Test completion (closure) | Final report, lessons learned, archive | All results | Test summary report, release recommendation |
What goes into a test plan
- Scope: what will be tested and what will not (e.g. "HUD not in scope").
- Test approach / strategy: manual vs automated, test levels, techniques, risk-based focus.
- Test items: which software versions and hardware samples.
- Environment: benches, tools, phones, licences.
- Schedule and milestones, resources and roles.
- Entry and exit criteria.
- Risks and what to do about them.
- Deliverables: test cases, results, reports.
- Defect management: tools, severity definitions, workflow.
Entry and exit criteria
Entry criteria (when can testing start?)
Build delivered with release notes · smoke test passed · bench ready and correct build flashed · test cases reviewed · requirements baselined.
Exit criteria (when is testing finished?)
All planned tests executed · e.g. ≥ 95% pass · no open blocker/critical defects · majors have agreed workarounds · 100% coverage of safety requirements · test report delivered.
Writing a good test case
| Field | Example |
|---|---|
| ID | TC-TEMP-003 |
| Title | Snowflake not shown at exactly 5.0 °C (boundary) |
| Requirement | REQ-HMI-TEMP-012 |
| Priority | High |
| Preconditions | Build flashed; ignition ON; units °C; language English; outside temp 6.0 °C for 3 s |
| Steps | 1. In CANoe set Outside_Temp = 5.0 °C 2. Wait 2 s 3. Observe the cluster |
| Expected result | No snowflake shown; temperature shows "5.0 °C" |
| Actual result / status | Filled during execution: Pass / Fail + evidence |
Bad test case
"Check the snowflake works." — No values, no steps, no expected result. Two testers would test different things.
Good test case
Exact values, exact steps, one clear expected result, linked to a requirement. Anyone gets the same result.
Tips: one test case = one purpose; expected result must be measurable; write for someone who has never seen the feature; include the requirement ID.
Traceability (RTM)
Traceability = links between requirements, test cases, results and bugs. A Requirements Traceability Matrix (RTM) is a table showing these links. It answers: "Is every requirement tested?" and "Which tests must I rerun if this requirement changes?"
| Requirement | Test cases | Last result (build 24.3.1) | Defects |
|---|---|---|---|
| REQ-HMI-TEMP-012 (snowflake) | TC-01, TC-02, TC-03, TC-04, TC-05 | 3 pass, 2 fail | IVI-1433 |
| REQ-HMI-TEMP-013 (°F display) | TC-06, TC-07 | 2 pass | – |
| REQ-HMI-TEMP-014 (sensor fault) | – | not covered! | – |
The last row shows a coverage gap: a requirement with no test. Traceability tools (Polarion, DOORS, Jira Xray, Codebeamer) find these automatically.
Risk-based testing
Because you can't test everything (principle 2), you test the riskiest things first and most. Risk = likelihood that it breaks × impact if it breaks.
| Feature | Likelihood | Impact | Risk | Testing effort |
|---|---|---|---|---|
| Speed display | Low | Very high (legal, safety) | High | Always, thoroughly |
| Bluetooth reconnect (just rewritten) | High | High (customers notice) | High | Thoroughly |
| Wallpaper selection | Low | Low | Low | Quick check |
Test metrics (numbers you report)
- Execution progress: executed / planned (e.g. 180 / 200 = 90%).
- Pass rate: passed / executed.
- Defects by severity and status: e.g. 0 blocker, 2 critical open.
- Requirement coverage: requirements with at least one test / all requirements.
- Defect density: defects per feature or per size of code.
- Defect leakage: bugs found later (by the customer) that testing missed.
- Blocked tests and their reasons.
About the ISTQB Foundation certificate
- Name: ISTQB Certified Tester Foundation Level (CTFL). Current syllabus: version 4.0 (2023).
- Exam: 40 multiple-choice questions, 60 minutes, pass mark 65% (26 correct).
- Topics: fundamentals, testing in the development life cycle, static testing, test techniques, test management, tools.
- There is also an automotive extension: ISTQB Automotive Software Tester (CTFL-AuT) — nice to mention if you have or plan it.
"I haven't taken the exam yet, but I've studied the Foundation syllabus and I apply it: the test process, BVA and equivalence partitioning, risk-based prioritisation. I plan to take the exam by [month]."
Quick answers for the interview
What is the difference between a test plan and a test strategy?
A test strategy is the general approach for the organisation or project, like "risk-based, 60% automated, these test levels". A test plan is specific: for this project or release, what we test, how, with which people and benches, by when, and with which risks and exit criteria.
What is the difference between verification and validation?
Verification checks we built the product right, against the specification. Validation checks we built the right product for the user's real need.
What is the STLC?
The Software Testing Life Cycle: requirement analysis, test planning, test analysis and design, environment setup and implementation, execution with defect reporting, and test closure with the final report. Monitoring and reporting runs throughout.
What is the difference between smoke and sanity testing?
Smoke testing is a quick, broad check that a new build is stable enough to test at all. Sanity testing is a quick, narrow check that one specific fix or change works.
What are entry and exit criteria?
Conditions to start and finish a test activity. Entry: e.g. build with release notes delivered, smoke test passed, bench ready. Exit: e.g. all planned tests run, no open critical defects, required coverage reached, report delivered.