Critical Thinking rules in the AI-Enhanced Software Testing
A 2025 talk arguing that critical thinking has to stay human while AI drives up the volume of test output.
By Michał (Mike) Buczko, quality and delivery leader at Carrier/Viessmann.
This talk argues that critical thinking has to stay human while AI drives up the volume of test output. An AI assistant is a very fast generator of plausible checks. Whether those checks match what the product is supposed to do is a question the tool does not ask for you.
What the talk covers
- Testing versus checking: a check confirms what you already expect, and a test is an investigation that can surprise you. I first argued this at test:fest in 2015.
- The same distinction at a different scale: AI writes assertions that match the code in front of it, and the result looks like rigour.
- The critical-thinking question: does any of those assertions match what the product is supposed to do?
- How AI tool capabilities relate to the quality fundamentals that came before them. This is the part the Istanbul discussion kept returning to.
Who it is for
Testers, test leads and quality engineers who use AI assistants to write or maintain tests.
What you take away
- A check for your own suite: take ten tests your AI tooling wrote last month and name the requirement each one protects without looking at the implementation.
- A question for the "balance AI with quality practice" debate: which specific practice do you expect to need less of?
- Three fundamentals that decide whether AI output can be trusted: requirements thinking, review discipline and gate design.
Where I have given it
- test:fest 2025, Wrocław, Poland, 24 May 2025, the tenth edition. Recap: test:fest 2025.
- Istanbul Software Testing Conference 2025, Istanbul, Turkey, 22 May 2025. Recap: Istanbul Software Testing Conference 2025.
I wrote both recaps in 2026, after a year of building software with AI agents. One thing changed in that year. In Istanbul I talked about balancing AI with fundamentals. I now think balance is the wrong word: fundamentals are the input condition for AI capability, not a counterweight to it.
Video
Recording from the Istanbul Software Testing Conference 2025 (26 minutes):
Format and practical details
- Length: about 30 minutes; the Istanbul recording runs 26.
- Language: English in Istanbul.
- Room and tech: a projector or screen to connect to.
Related talks
- From Tester to AI Operator: Building the Career Path Before You Need It
- Is AI the future of Software Delivery: Can it replace a Feature team?
- Who Signs the Release When AI Writes the Code?
Book this talk
To book this talk, start at the Speaking page, book a 30-minute call or message me on LinkedIn.
