Retest & regression testing
The interviewer spent many questions on regression: strategy, which fixed bugs to add, and how to cut thousands of tests down to what fits in the time. This chapter gives you clear, structured answers.
- Retest = check that the fixed bug is really fixed.
- Regression = check that the fix did not break other things.
- You can't run 3,000 tests every day, so choose by risk: safety, what changed, what is used most, where bugs were before.
- Add serious fixed bugs to the regression suite, not tiny ones.
Think of it like a plumber fixes one leaking pipe. You check that pipe (retest) and then open all the other taps in the house (regression).
See it as a picture: Plumber story · Regression funnel
Retest vs regression, in plain words
A plumber fixes a leaking pipe in your kitchen. Retest = you check that the kitchen pipe doesn't leak anymore. Regression test = you check the bathroom and the washing machine too, because the plumber turned off the water and touched other pipes. Maybe something else broke.
| Retest (confirmation testing) | Regression testing | |
|---|---|---|
| Question | "Is this bug really fixed?" | "Did the change break anything else?" |
| What you run | The exact failed test / steps from the ticket | Other tests around and beyond the change |
| When | When a fix is delivered | After any change: fixes, new features, new builds, configuration changes |
| Result used for | Closing or reopening the ticket | Confidence to release |
| Order | First | Second |
"Retesting confirms that a specific defect is fixed by repeating the failed steps. Regression testing checks that the fix or other changes didn't break existing functionality. I always retest first, then run regression for the affected area."
Why regression is so important in cars
- Car software is huge and connected: a change in the audio code can break phone calls, navigation voice and warning chimes.
- New builds come very often (daily/weekly), so old features must be re-checked constantly.
- Some features are safety-relevant or legally required; they must never break.
- Bugs that were fixed can come back (e.g. a code merge undoes a fix).
A regression strategy: layers
You can't run everything on every build. So regression is organised in layers, from small and frequent to big and rare:
| Layer | Size (example) | When | Contents |
|---|---|---|---|
| Smoke | 20–50 tests, ~30 min | Every new build | Boots, main screens, key functions, no crash |
| Sanity / targeted | As needed | After a specific fix | The fixed area and its neighbours |
| Core regression | 300–800 tests, 1–2 days | Each release candidate or sprint end | Safety/legal functions, most-used features, recently changed areas, previously buggy areas |
| Full regression | Everything, 1–3 weeks | Major releases, before SOP | All test cases |
Automation helps a lot: stable, repetitive tests run automatically every night; people focus on new features, visual checks and exploratory testing.
"Risk-based and in layers. A smoke test on every new build; targeted regression around each fix; a core regression set on each release candidate with safety-relevant functions, frequently used features, recently changed and historically buggy areas; and full regression for major releases. Stable repetitive tests are automated and run nightly, while manual effort goes to new features and visual checks. I use the release notes and the developers' impact analysis to decide what to add for each build."
Impact analysis: what did the change touch?
Before choosing regression tests, find out what changed:
- Read the release notes: which tickets were fixed, which features added.
- Ask the developer: which modules/files changed? What could be affected?
- Use traceability: which requirements belong to the changed feature → which tests are linked to them.
- Think about dependencies: audio change → also calls, nav voice, chimes. Bluetooth change → also CarPlay, phonebook, media.
Choosing tests when time is short
The interviewer: "Regression time is limited and there are thousands of tests. What criteria would you use?" Use these criteria, in this order:
- Safety-critical and legal functionsAlways in: speed display, telltales and warnings, rear camera, seatbelt chime. (ISO 26262 / ASIL-relevant, UN regulations.)
- Changed areasEverything directly affected by this build's changes (from impact analysis).
- Recently fixed, high-severity bugsTheir tests confirm the fixes still hold. This is the "criticality of bugs" follow-up: critical and major bugs' tests stay; trivial ones can be dropped or sampled.
- Historically buggy areas"Defects cluster together": areas with many past bugs get more tests.
- Most-used customer featuresPhone, media, navigation: customers notice immediately.
- Integration pointsWhere systems meet: phone ↔ head unit, head unit ↔ cluster, CarPlay ↔ native.
- Cheap automated testsIf a test is automated, include it; it runs overnight at no extra people-cost.
Then remove: duplicate tests (same check twice), obsolete tests (removed features), and low-value cosmetic tests on unchanged screens.
| Priority | What | Rule |
|---|---|---|
| P1 — must run | Safety/legal + changed areas + critical bug fixes | Always, even if the release is tomorrow |
| P2 — should run | Most-used features, buggy areas, integration points | Run if time allows (usually yes) |
| P3 — could run | Stable, rarely used, cosmetic, unchanged | Automated overnight, or sampled, or next cycle |
"I prioritise by risk. First safety-critical and legally relevant functions, then everything touched by the change according to the release notes and developer impact analysis, then tests for recently fixed critical and major defects, then historically buggy areas and the most-used features. Automated tests I include anyway because they run overnight. I remove duplicates and obsolete tests, and whatever doesn't fit I report as not covered — residual risk — so the release decision is informed."
Should every fixed bug go into regression?
No. If every fixed bug added a test, the suite would grow forever and become too slow. You add a bug's test when it's likely to come back or would be bad if it did.
Add it when…
• it's safety-relevant or legal (speed, warnings, camera)
• it's severe and customer-visible
• the area changes often or the bug came back before
• it's a boundary/logic bug that a code change could easily reintroduce
• existing tests didn't catch it (a coverage gap)
Don't add it when…
• it was a one-off typo already covered by a language/screenshot check
• the feature was removed or completely redesigned
• it was caused by the test environment, not the software
• an existing regression test already covers the same thing
• it's trivial and in a rarely used, unchanged screen
"I would add it: the speed shown on the cluster was wrong after an ignition cycle. That's safety-relevant, and ignition-cycle code changes often, so I'd add that test to the core regression set. I would not add it: a spelling mistake in one German menu label. It's fixed, it's already covered by our screenshot check of all texts, and adding a separate test gives no extra value."
Safety-critical vs non-critical tests
- Safety-critical first, always, completely. They are never cut for time.
- Non-critical: by risk (changed? often used? buggy before?) and sampling.
- Safety-critical tests may also need stricter evidence (videos, traces) and review.
Scenario: regression must finish in one day
- Morning: smoke test the build (30 min). If it fails badly, stop and report immediately.
- Start the automated suite right away on a free bench (it runs while you work).
- Run P1 manually: safety/legal + changed areas + critical fixes.
- Then P2 as time allows. Use several benches/testers in parallel if possible.
- Triage automation failures as they come.
- Evening: report: what was run, results, open bugs, and what was not covered (residual risk).
Scenario: urgent release deadline
- Agree the reduced scope with the customer/proxy team in writing (which tests will be skipped).
- Parallelise: more benches, more testers, overnight automation.
- Focus on P1 + areas changed since the last fully tested build.
- Report the residual risk clearly: "Not tested: X, Y. Risk: low/medium because…"
- Never skip tests silently. The release decision belongs to the project, but it must be based on honest information.
Keeping the regression suite healthy
- Review it regularly: remove obsolete and duplicate tests.
- Refresh test data (principle: tests wear out / pesticide paradox).
- Add tests for new features and for important bugs that escaped.
- Track flaky automated tests and get them fixed.
- Keep traceability updated so impact analysis works.
Quick answers for the interview
What is the difference between retesting and regression testing?
Retesting repeats the failed test to confirm a specific defect is fixed. Regression testing re-runs other tests to make sure the change didn't break anything else. Retesting is planned per ticket; regression is selected by risk and impact.
How do you verify that a fix hasn't impacted other functionality?
I find out what the fix changed — release notes and the developer's impact analysis — then run regression tests for that area and its dependencies, plus the smoke test and the automated suite. If something in a neighbouring area fails, I check whether it's a new regression and report it with a link to the fix ticket.
How would you prioritise when regression time is limited?
By risk: safety and legal first, then changed areas, then fixes for critical defects, then buggy and frequently used areas, and I always include automated tests because they're cheap. I report what I couldn't cover as residual risk.