The Institute for Perception

Journal Article: A Comparison of three commonly used methods for treating no preference votes

ifpressdelta — Tue, 08 May 2012 18:47:12 +0000

Abstract:
The treatment of no preference votes continues to be an issue in sensory science, especially as the proper treatment of these votes has recently gained importance in advertising claims support. There are currently three main methods in common use: dropping the no preference votes, splitting the votes equally and splitting the votes proportionally according to the results among those who expressed a preference.The analyses then proceed as if the data were binomially distributed. In this paper, we compare these methodswith respect to power and type I error.We showthat proportional splitting returns more false alarms than expected and hence should not be used.We then discuss the meaningful interpretation of statistical significance in the presence of large numbers of no preference votes before providing general recommendations and indicating a promising direction of future research in this area.

This is paper # 99 when requesting reprints. To request a paper, please click here.

This article appears as:
Ennis, J. M. and Ennis, D. M. (2012). A comparison of three commonly used methods for treating no preference votes. Journal of Sensory Studies, 27, 123-129.

Journal Article: Justifying Count-Based Comparisons

ifpressdelta — Mon, 07 May 2012 21:12:45 +0000

Abstract:
Count-based comparisons such as “7 out of 10” or “2 to 1” are often used to quantify superior product performance. Because of experimental variability, statistics are needed to ensure confidence in such comparisons. Even so, count-based comparisons commonly appear without any statistical treatment of the data. In this article, we discuss statistics to support these comparisons. Specifically, we identify two different types of count-based comparisons: count-based proportional comparisons and count-based ratio comparisons.We then provide statistics to justify these comparisons before providing significance tables for practitioners.

This is paper # 98 when requesting reprints. To request a paper, please click here.

This article appears as:
Ennis, J.M. and Ennis, D. M. (2012). Justifying Count-Based Comparisons. Journal of Sensory Studies, 27, 130-136.

Technical Report: Reducing Costs with Tetrad Testing

ifpressdelta — Mon, 26 Mar 2012 17:39:24 +0000

ABSTRACT
The Triangle test is one of the most popular discrimination methodologies used in consumer products companies. Beginning in 1941, Joseph E. Seagram and Sons were the first to employ it – since then it has been used in a variety of applications including product discrimination testing and panelist selection. An advantage of the Triangle test is that it does not require specification of the nature of the difference. Yet the Triangle test requires large sample sizes to be effective and, recently, the Tetrad test has received interest due to its potential to provide increased power without specification of an attribute. This greater power means that for the same sample size, an existing difference is less likely to be missed. Alternatively, greater power means that smaller sample sizes can be used to achieve the same performance as the Triangle - the sample sizes required by the Tetrad test are theoretically only one third that required by the Triangle test. Importantly, like the Triangle test, the Tetrad test does not require specification of a sensory attribute. The purpose of this technical report is to illustrate how sensory scientists can determine whether or not Tetrad testing is worth considering as a standard testing methodology within their discrimination testing programs.

This technical report appears as:
Ennis, J. M. and Rousseau, B. (2012). Interpreting Applicability Scores. IFPress, 15(1) 4-5.

Download the entire technical report here:
Reducing Costs with Tetrad Testing

This technical report also appears in our book, Short Stories in Sensory and Consumer Science.

Journal Article: A group level validation of the supercombinatorality property: Finding high-quality ingredient combinations using pairwise information (2012)

ifpressdelta — Tue, 07 Feb 2012 18:27:05 +0000

Abstract:
This study tested the principle of supercombinatorality, i.e. that food combinations (of more than two items) that are fully compatible on a pairwise basis are more compatible than combinations that are not fully compatible pairwise. Previous work has shown this to hold for salad ingredient combinations predicted for individuals, but this has not yet been tested for groups. This study extended the previous findings to group data, and in a different product system, namely pizza toppings. Purchase intent responses to pairs of 25 different pizza toppings were collected and used to predict pizzas (with 1–6 toppings) that would appeal to the entire group. Results showed purchase interest to be higher for the predicted pizzas than for non-predicted pizzas supporting the supercombinatorality principle. The study demonstrates that food product developers can use consumer-driven data and a graph theoretic approach to screen large numbers of potential food combinations in order to predict potentially successful combinations and to do so in a highly cost-efficient manner.

This is paper # 97 when requesting reprints. To request a paper, please click here.

This article appears as:
Nestrud, M. A., Ennis, J. M., and Lawless, H. T. (2012). A group level validation of the supercombinatorality property: Finding high-quality ingredient combinations using pairwise information. Food Quality and Preference , 25, 23-28.

Technical Report: Interpreting Applicability Scores

ifpressdelta — Wed, 11 Jan 2012 19:38:58 +0000

ABSTRACT
“Check-all-that-apply” (CATA) lists are a popular tool in product tests1,2,3. In a typical test, consumers respond to a series of statements and mark those statements that apply to the product of interest. An advantage of CATA testing is that it provides an opportunity to obtain information from consumers that would be difficult in some cases to extract using either a rating or 2-AFC format. A related method, explored by Loh and Ennis4 in 1982, is called applicability scoring. In applicability scoring, consumers mark statements that are applicable but also mark statements that are not applicable. In a CATA list, an unmarked item may imply that the consumer does not think that the item applies, but could also mean that the consumer merely missed that item – applicability scoring avoids this ambiguity. In this report the topic of how to analyze and interpret applicability scores will be discussed. This report will provide guidance on the analysis of applicability counts to test a null hypothesis of no difference and will also discuss the scaling of applicability data using a Thurstonian model. One application of particular interest will be the comparative evaluation of two products on liking.

This technical report appears as:
Ennis, D. M. and Ennis, J.M. (2011). Interpreting Applicability Scores. IFPress, 14(4) 3-4.

Download the entire technical report here:
Interpreting Applicability Scores

This technical report also appears in our book, Short Stories in Sensory and Consumer Science.

Journal Article: Internal preference mapping and the issue of satiety (2012)

ifpressdelta — Mon, 09 Jan 2012 14:35:07 +0000

Abstract:
Internal preference mapping (IPM) and Landscape Segmentation Analysis (LSA) are two techniques broadly used to unfold consumers’ overall product liking ratings and create spatial maps that will provide further insights on consumers’ preferences. IPM is based on a vector model while LSA involves an ideal point model. Through a simulation and the analysis of 27 market research data sets, it is shown that IPM consistently creates a hedonic dimension that prevents the identification of satiety prone attributes (intensities higher or lower than a optimal level being disliked by the consumers) on that dimension. As a result, subsequent steps taken upon generating an IPM map such as the investigation of drivers of liking, population segmentation and the estimation of optimal product profiles have also a strong likelihood of resulting in distorted results, the level of distortion being dependent on the actual configuration of the underlying structure that IPM tried to uncover. It is also shown that a technique based on ideal points such as LSA does not exhibit this systematic artifact when unfolding liking data. Consequently, sensory scientists and market researchers should use caution when interpreting and using results issued from an internal preference mapping analysis.

This is paper # 96 when requesting reprints. To request a paper, please click here.

This article appears as:
Rousseau, B., Ennis, D. M., and Rossi, F. (2012). Internal preference mapping and the issue of satiety. Food Quality and Preference, 24(1), 67-74.

Journal Article: Accounting for no difference/preference responses or ties in choice experiments (2012)

ifpressdelta — Wed, 04 Jan 2012 21:25:29 +0000

Abstract:
The analysis of choice data in which no difference/preference responses, or ties, occur is considered in this paper. A key issue addressed in the paper is the need for ‘‘identicality norms’’ for difference and preference tests. These norms reflect the researcher’s expectation for the number of ties that would have occurred in the experiment had the products tested been putatively identical. Without these norms, the issue of how to account for ties can never be fully resolved. After this idea is developed, some methods from the statistics literature to account for ties are reviewed and the Thurstonian 2-AC (2-Alternative Choice) model is discussed. Common practices of equal or proportional redistribution of ties are noted to be either conservative or liberal, respectively, when the binomial distribution is used to evaluate results. In particular, the exact probability function for the equal allocation method is given as a particular type of mixing distribution, known as a convolution, of binomial probability functions. Regardless of which statistical method is used to test tied data, however, none of the current methods of analysis can account for segmentation or the effect of heterogeneity in individual assessors. To study the possible effect of heterogeneity, the data could first be tested against an identicality norm. Thus, this research clarifies the assumptions that are made when conducting tests on paired comparison data with ties and provides guidance on the choice of analytic method.

This is paper # 95 when requesting reprints. To request a paper, please click here.

This article appears as:
Ennis, D. M. and Ennis, J. M. (2012). Accounting for no difference/preference responses or ties in choice experiments. Food Quality and Preference, 23(1), 13-17.

Journal Article: eTURF: A competitive TURF algorithm for large datasets (2012)

ifpressdelta — Tue, 03 Jan 2012 21:31:30 +0000

Abstract:
Although Total Unduplicated Reach and Frequency (TURF) analysis has repeatedly demonstrated value in a variety of market research applications, TURF analyses on large problem sizes have historically either been unacceptably slow or approximate in their solutions. To resolve this dilemma, we begin by identifying and explaining the principle of non-synergy that is present in all situations for which TURF analyses apply. We use this principle to provide a competitive algorithm, called eTURF, for determining exact solutions to large TURF problems in reasonable time and we provide an illustrative example to demonstrate how eTURF can obtain considerable speed improvements. We then discuss how eTURF is useful not only in terms of providing optimal solutions but also in providing a metric by which to gauge the quality of heuristic TURF solutions.

This is paper # 94 when requesting reprints. To request a paper, please click here.

This article appears as: Ennis, J. M., Fayle, C. M., and Ennis, D. M. (2012). eTURF: A competitive TURF algorithm for large datasets. Food Quality and Preference, 23(1), 44-48.

Journal Article: The power of sensory discrimination methods revisited (2011)

ifpressdelta — Sat, 31 Dec 2011 21:38:24 +0000

Abstract:
“The power of sensory discrimination methods” (PSDM) was published in this journal in 1993.PSDM clarified the need for power considerations in the interpretation of testing results while providing a series of sample size tables. Despite the fact that the data considered in PSDM were binomially distributed, a normal approximation was used that both overestimated power and underestimated sample sizes. Although exact power functions have been examined in the sensory literature, the unusual behavior of these functions has not been embraced; the fact that increasing sample size can decrease power has not yet been incorporated into stable sample size recommendations. In this paper,we provide sample size recommendations with the property that any larger sample sizes also have the desired level of power. These recommendations are given in the form of tables updating those found in PSDM. In addition, a relatively new discrimination testing method known as the tetrad test has grown in popularity recently and this test now needs to be examined from a power perspective.We show that the tetrad test is remarkably powerful for an unspecified test and in some cases only requires one third the sample size as that required by the triangle test.

This is paper # 93 when requesting reprints. To request a paper, please click here.

This article appears as: Ennis, J. M. and Jesionka, V. (2011). The power of sensory discrimination methods revisited. Journal of Sensory Studies, 26(5), 371-382.

Journal Article: Validating a graph theoretic screening approach to food item combinations (2011)

ifpressdelta — Thu, 29 Dec 2011 21:41:58 +0000

Abstract:
Tools from the mathematical field of graph theory potentially allow the consumer scientist to efficiently analyze large numbers of combinations of food items, such as components on a salad. In this study, we tested the validity of such an approach.We began by asking subjects whether or not pairs of ingredientswould be appropriate to combine on a salad.Next, using graph theoretic methods, we predicted which combinations of 3–8 components should go together, and perhaps more importantly, which combinations should not. Subjects were then asked whether or not particular combinations were appropriate to combine on a salad. A paired Wilcoxon test between the predicted and nonpredicted combinations was significant for all combination sizes.

This is paper # 92 when requesting reprints. To request a paper, please click here.

This article appears as:
Nestrud, M. A., Ennis, J. M., Fayle, C. M., Ennis, D. M., and Lawless, H. T. (2011). Validating a graph theoretic screening approach to food item combinations. Journal of Sensory Studies, 26(5), 331-338.

Journal Article: Confidence Bounds for Multiplicative Comparisons (2011)

ifpressdelta — Tue, 20 Sep 2011 14:39:18 +0000

Abstract:
Statements that are inherently multiplicative have historically been justified using ratios of random variables. Although recent work on ratios has extended the classical theory to produce confidence bounds conditioned on a positive denominator, this current article offers a novel perspective that eliminates the need for such a condition. Although seemingly trivial, this new perspective leads to improved lower confidence bounds to support multiplicative statements. This perspective is also more satisfying as it allows comparisons that are inherently multiplicative in nature to be properly analyzed as such.

This is paper # 91 when requesting reprints. To request a paper, please click here.

This article appears as:
Ennis, J.M. and Ennis, D.M. (2011). Confidence Bounds for Multiplicative Comparisons. Communications in Statistics: Theory and Methods, 40(17), 3049-3054.

Talk: Workshop on Combinatorial Tools in Sensory and Consumer Science

ifpressdelta — Thu, 15 Sep 2011 18:07:27 +0000

Presented at the 2011 Pangborn Symposium in Toronto, Canada.

Download the entire presentation here:
Workshop on Combinatorial Tools in Sensory and Consumer Science

Poster: Too Much or Too Little: Why Ignoring Satiety Can Result in Misleading Mapping Results

ifpressdelta — Thu, 15 Sep 2011 17:47:49 +0000

Presented at the 2011 Pangborn Symposium in Toronto, Canada.

Download the poster here:
Too Much or Too Little: Why Ignoring Satiety Can Result in Misleading Mapping Results

Poster: Establishing Identicality Norms for No Preference Data

ifpressdelta — Thu, 15 Sep 2011 17:41:49 +0000

Presented at the 2011 Pangborn Symposium in Toronto, Canada.

Download the poster here:
Establishing Identicality Norms for No Preference Data

Poster: Searching for a Single Grain of Sand: How to Find Optimal Combinations of Features or Components

ifpressdelta — Thu, 15 Sep 2011 17:38:56 +0000

Presented at the 2011 Pangborn Symposium in Toronto, Canada.

Download the poster here:
Searching for a Single Grain of Sand: How to Find Optimal Combinations of Features or Components