ISTQB
Certified Tester
Testing with GenAI

Seminar contents

1. introduction to generative AI in software testing

• Basics of generative AI and Large Language Models (LLMs)

• Types of LLMs: Foundation, Instruction-Tuned, Reasoning

• Multimodal models (text, image)

• Possible uses of GenAI in software testing

• LLM-supported test applications vs. chatbots

2. prompt engineering in software testing

• Building structured prompts (role, context, instruction, input, constraints, output)

• Prompt techniques: prompt chaining, few-shot, meta-prompting

• Application to test analysis, test design, regression tests, test control

• Evaluation and optimization of prompts

• Metrics for quality assurance of GenAI results

3. risks and challenges in the use of Generative AI

• Hallucinations, errors in reasoning and distortions

• Data protection and security when testing with LLMs

• Energy consumption and environmental aspects

• Regulatory frameworks (e.g. EU AI Act, ISO/IEC 42001)

4. LLM-based test infrastructure and operationalization

• Architecture of LLM-supported test tools

• Retrieval augmented generation (RAG)

• LLM agents in the test process

• Fine-tuning and LLMOps: Introduction, operation and maintenance of LLMs

5. introduction and sustainable integration in test organizations

• Strategy and roadmap for GenAI in the test process

• Selection of suitable LLMs/SLMs

• Training and competence building in the team

• Changing role models (e.g. prompt engineer, hybrid tester)

• Dealing with "Shadow AI" and change management

Optional: practical exercises and demonstrations

• Creating and refining prompts

• Evaluation of GenAI-generated test cases

• Analysis of test reports with LLM support

• Simulated use of RAG and agents in the test process