Course · ISTQB CT-AI v1.0
The four-day course for testing AI-based systems, and using AI in testing.ISTQB Certified Tester AI Testing (CT-AI), v1.0.
Official outline for the ISTQB Certified Tester AI Testing course as delivered by Rex Black Inc. Four days. Eleven chapters. Covers testing AI-based systems and using AI to support software testing.
- Duration
- 4 days
- Chapters
- 11
- Class time
- ~380–400 min/day
Key takeaways
Four things to remember.
01
Two directions, one course
Testing AI-based systems, and using AI to support testing. CT-AI covers both, which is why it is the fastest-growing ISTQB credential.
02
Experience-based, not lecture-only
Every chapter includes explanations, discussions, and exercises. Attendees implement and test a real ML model, not just read about one.
03
AI quality characteristics, front and center
Bias, ethics, transparency, explainability, safety, self-learning, non-determinism, not afterthoughts, but the backbone of Chapters 2 and 8.
04
Certification-ready
Sample exam, syllabus coverage, and glossary are included. Attendees leave ready to sit the ISTQB CT-AI exam.
Overview
The Certified Tester AI Testing course is for anyone involved in testing AI-based systems and/or AI for testing. This includes people in roles such as testers, test analysts, data analysts, test engineers, test consultants, test managers, user acceptance testers and software developers. The certification is also appropriate for anyone who wants a basic understanding of testing AI-based systems and/or AI for testing, such as project managers, quality managers, software development managers, business analysts, operations team members, IT directors and management consultants.
This course is experience-based, highly interactive with a mixture of explanation of concepts and exercises. It covers the ISTQB Certified Tester AI Testing Syllabus (CT-AI) v1.0. Solutions are provided for the exercises demonstrated and discussed in the class, along with sample questions, copies of the ISTQB Certified Tester AI Testing Syllabus and Glossary, and more.
01
What attendees will gain
- Understand the current state and expected trends of AI.
- Experience the implementation and testing of an ML model and recognize where testers can best influence its quality.
- Understand the challenges associated with testing AI-based systems, such as their self-learning capabilities, bias, ethics, complexity, non-determinism, transparency and explainability.
- Contribute to the test strategy for an AI-based system.
- Design and execute test cases for AI-based systems.
- Recognize the special requirements for the test infrastructure to support the testing of AI-based systems.
- Understand how AI can be used to support software testing.
02
Course materials
- Course Outline, general description plus learning objectives, materials, and approximate section timings.
- Noteset, approximately 500 PowerPoint slides covering the topics.
- Sample Exam, assess your readiness for the ISTQB CT-AI exam.
- Exercise Solutions, approximately 80 pages of demonstrated exercises.
- ISTQB CT-AI Syllabus, the official syllabus that forms the basis of the certification.
- ISTQB Glossary, the latest glossary of software testing terms from ISTQB.
03
Session plan (4 days)
The course runs for four days. Each day is about 380–400 minutes of class time, from 9:00 AM to 5:00 PM, including lunch and other breaks. Timings are approximate and depend on attendee interest and discussion.
- Introduction (30 minutes)
04
Chapter 1, Introduction to AI (105 minutes)
- Definition of AI and AI Effect
- Narrow, General and Super AI
- AI-Based and Conventional Systems
- AI Technologies
- AI Development Frameworks
- Hardware for AI-Based Systems
- AI as a Service (AIaaS)
- Pre-Trained Models
- Standards, Regulations and AI
05
Chapter 2, Quality Characteristics for AI-Based Systems (105 minutes)
- Flexibility and Adaptability
- Autonomy
- Evolution
- Bias
- Ethics
- Side Effects and Reward Hacking
- Transparency, Interpretability and Explainability
- Safety and AI
06
Chapter 3, Machine Learning: Overview (145 minutes)
- Forms of ML
- ML Workflow
- Selecting a Form of ML
- Factors Involved in ML Algorithm Selection
- Overfitting and Underfitting
07
Chapter 4, ML: Data (230 minutes)
- Data Preparation as Part of the ML Workflow
- Training, Validation and Test Datasets in the ML Workflow
- Dataset Quality Issues
- Data Quality and its Effect on the ML Model
- Data Labelling for Supervised Learning
08
Chapter 5, ML: Functional Performance Metrics (120 minutes)
- Confusion Matrix
- Additional ML Functional Performance Metrics for Classification, Regression and Clustering
- Limitations of ML Functional Performance Metrics
- Selecting ML Functional Performance Metrics
- Benchmark Suites for ML Performance
09
Chapter 6, ML: Neural Networks and Testing (65 minutes)
- Neural Networks
- Coverage Measures for Neural Networks
10
Chapter 7, Testing AI-Based Systems Overview (115 minutes)
- Specification of AI-Based Systems
- Test Levels for AI-Based Systems
- Test Data for Testing AI-Based Systems
- Testing for Automation Bias in AI-Based Systems
- Documenting an AI Component
- Testing for Concept Drift
- Selecting a Test Approach for an ML System
11
Chapter 8, Testing AI-Specific Quality Characteristics (150 minutes)
- Challenges Testing Self-Learning Systems
- Testing Autonomous AI-Based Systems
- Testing for Algorithmic, Sample and Inappropriate Bias
- Challenges Testing Probabilistic and Non-Deterministic AI-Based Systems
- Challenges Testing Complex AI-Based Systems
- Testing the Transparency, Interpretability and Explainability of AI-Based Systems
- Test Oracles for AI-Based Systems
- Test Objectives and Acceptance Criteria
12
Chapter 9, Methods and Techniques for Testing AI-Based Systems (245 minutes)
- Adversarial Attacks and Data Poisoning
- Pairwise Testing
- Back-to-Back Testing
- A/B Testing
- Metamorphic Testing (MT)
- Experience-Based Testing of AI-Based Systems
- Selecting Test Techniques for AI-Based Systems
13
Chapter 10, Test Environments for AI-Based Systems (30 minutes)
- Test Environments for AI-Based Systems
- Virtual Test Environments for Testing AI-Based Systems
14
Chapter 11, Using AI for Testing (195 minutes)
- AI Technologies for Testing
- Using AI to Analyze Reported Defects
- Using AI for Test Case Generation
- Using AI for the Optimization of Regression Test Suites
- Using AI for Defect Prediction
- Using AI for Testing User Interfaces
15
Wrap
Question and answer period.
Take it with you
Download the piece you just read.
The library is free. Tell us who you are so we can send you updated versions. One form; this browser remembers you after that.
In the library
Pair this with.
Process · The Master Framework
Four phases, applied in order, every project.
The top-level testing process used on every engagement. Four phases contain twelve sub-processes, each of which is documented in its own checklist in this library.
Read →Process · Quality Risk Analysis
Prioritize risk before you prioritize tests.
Testing without a prioritized quality risk analysis is testing by politics. This six-step process produces a risk register with explicit stakeholder consensus, the basis for every downstream testing decision.
Read →
Keep reading
Related reading
- Primer
Careers in Software Quality: Roles, Skills, and First Steps
What testers, test automation engineers, quality engineers, and test managers actually do, the skills each role uses, and practical first steps for students and career changers, including the ISTQB Foundation certification.
Read → - Primer
Errors, Defects, and Failures: What a Bug Really Is
Everyone says 'bug', but testers use three precise words: error, defect, and failure. Learn the difference, why it matters, and how severity and priority decide which problems get fixed first.
Read → - Primer
How AI Language Models Work, in Plain English
What a large language model actually does: learning from text, predicting the next token, context windows, why models sometimes state false things with confidence, and why some models cost a hundred times more than others. No math required.
Read → - Primer
How Software Gets Attacked: Common Vulnerabilities in Plain English
The most common ways attackers break into software, explained without jargon: trusting input, broken access control, weak sign-in, misconfiguration, and outdated components. Each one comes with the defense that stops it.
Read → - Primer
How to Report a Bug So It Actually Gets Fixed
Finding a bug is half the job. Learn how to write a bug report a developer can act on: a clear summary, exact steps to reproduce, expected versus actual results, and the habits professional testers use to make every report count.
Read → - Primer
Security Testing Basics: Finding Weaknesses Before Attackers Do
An introduction to security testing for new testers and developers: thinking in threats, designing abuse cases, testing access control, the role of automated scanners and penetration tests, and the rules that keep security testing legal and ethical.
Read →
Practices
Where this leads
- QA & testingSoftware testing
Software quality consulting since 1994: we test the releases that matter, coach your team on the method, and measure how its testing matures.
Book a call → - AI agentsAI agents
Voice and intake agents that run in production: they answer the phone, book meetings, and hand off to your team.
Book a call → - TrainingQA and AI training
ISTQB certification and hands-on courses for testers, engineers, and leaders, from one of the first ASTQB-accredited training providers in the United States.
Book a call →