Planning for Precision when Designing Difference Tests

Dr. John Ennis
Vice President of Research Operations
The Institute for Perception

Once a consumer relevant difference is identified, it is still necessary to determine the business relevant details of optimal difference test design. To ensure that important differences are detected when they exist, the design of sensory difference tests has traditionally focused on achieving high test power. While there are historical reasons for this focus, the business role of difference testing within modern sensory programs necessitates a shift in focus from the “yes”/”no” answers of null hypothesis testing to a measurement-based approach in which the size of the sensory difference is estimated and the precision of the estimate is computed. With this new focus, it is possible to design a difference test with a consumer relevant difference in mind, and to quantify the risk associated with various business decisions. We will discuss how to design a sensory difference test around a specific business goal, using the concepts of precision of measurement and consumer relevant difference, and will highlight the advantages over power and statistical significance.