Abstract
Designing A/B testing experiments is relatively easy if you have plenty of subjects or hardware to run LLMs. What if you have many more design knobs than just two conditions, or can only ask relatively few developers/users/LLMs to test your solutions? This technical briefing will help FSE attendees gain a hands-on understanding of designing combinatorial experiments in Software Engineering. The activity is grounded in the research group's significant experience and will draw on many examples. It includes interactive hands-on exercises with the learning objectives of designing, conducting, and analyzing one's own experimental study.
| Original language | English |
|---|---|
| Title of host publication | FSE Companion '26: Proceedings of the 34th ACM International Conference on the Foundations of Software Engineering |
| Editors | Shin Hwei Tan, Foutse Khomh |
| Publisher | Association for Computing Machinery, Inc |
| Pages | 66-67 |
| Number of pages | 2 |
| ISBN (Electronic) | 9798400726361 |
| DOIs | |
| Publication status | Published - 2026 |
| Event | ACM International Conference on the Foundations of Software Engineering, FSE 2026 - Montreal, Canada Duration: 5 Jul 2026 → 9 Jul 2026 |
Conference
| Conference | ACM International Conference on the Foundations of Software Engineering, FSE 2026 |
|---|---|
| Country/Territory | Canada |
| City | Montreal |
| Period | 5/07/26 → 9/07/26 |
Bibliographical note
Publisher Copyright:© 2026 Copyright held by the owner/author(s).
Keywords
- crossover experimental design
- design of experiments
- full factorial design
- orthogonal design
Fingerprint
Dive into the research topics of 'Technical Briefing on Beyond A/B Testing - Orthogonal and Experimental Designs'. Together they form a unique fingerprint.Cite this
- APA
- Author
- BIBTEX
- Harvard
- Standard
- RIS
- Vancouver