Abstract:

 

Difference testing methods such as the Triangle, Duo-Trio, Tetrad, and 2-Alternative Forced Choice (2–AFC) generate choice counts or numbers of correct responses that are often analyzed statistically as binomial variables. This analysis approach provides the basis for hypothesis tests for difference testing methods and for commonly available tables for that purpose. Since any difference can be shown to be significant with a sufficiently large sample size, there has recently been increasing interest in using difference tests to measure the size of sensory differences using Thurstonian theory. Concurrent with this shift in perspective away from hypothesis testing and towards effect size estimation has been a desire to quantify the precision of these measurements. Along with these developments, the importance of determining the size of consumer-relevant differences has also become apparent. For all these reasons, in this report we will consider the use of confidence intervals to help with the interpretation of sensory differences obtained through Thurstonian scaling and to make use of the precision of these estimates.

 

A value in converting choice outcomes to Thurstonian scaled estimates of d, a standard measure of sensory difference, is that it allows a comparison of methodologies on a common basis and has provided empirically supported predictions about the relative power of different methods. This insight was recently used to support a switch from the Triangle test to the Tetrad test and earlier provided a very satisfying explanation for the large difference in power between the 3-AFC and the Triangle test 7 . The experimental estimate of d, called d', can be obtained easily from many methods and it is of interest to provide a way of calculating confidence intervals for d and comparing the results to consumer-relevant specifications.

​

​

​