Decision tree algorithms essentially create a flowchart to guides the model predictions. These flowcharts are often easy to understand by humans and can handle highly correlated predictor variables, but they can also be inaccurate and unstable - when the data change even a little, trees can change drastically. Random forests were created to address these stability issues. Random forest methods create many trees, each from a subset of the predictor variables, and combine their results to create a final prediction which is more stable and usually more accurate - but at the expense of interpretability (Kuhn, 2013, Ch. 8 and 14). Of interest to consumer scientists is the fact that decision trees are excellent for modeling consumer segmentation, allowing us to see which factors are most differentiating and where those factors induce splits between segments.

​

Another method which works by dividing the predictor space is support vector machines (SVMs). SVMs are an extension of linear discriminant analysis, and work by finding planes in the space which best divide the data into categories (James, 2013, Ch. 9). An improvement over regression-based approaches, SVM algorithms automatically transform the space of predictor values to allow for non-linear relationships. Moreover, SVMs can be used for continuous or categorical predictions, although they work best for categorical outcomes. While SVMs can be powerful for prediction, models produced by SVMs are “black box” and nearly impossible to interpret. Hence they can propagate latent bias in the training data without such bias being conspicuous. Such concerns notwithstanding, support vector machines are a good choice when you are unsure of the relationship between predictors and outcome, especially if you have a large number of predictors.

​

One final example of machine learning algorithms are those built on networks. Inspired by neurobiology, artificial neural networks (ANNs), which include deep learning and convolutional neural networks (CNNs), employ a network of nodes where each node implements a rule about how it transforms input into output. The outputs of individual nodes are input to other nodes until a final output gives the prediction (Géron, 2017, Ch. 10). These techniques work best when there is a large amount of data, but recent advances such as adversarial neural networks provide alternative methods of model training. Like support vector machines, ANNs are difficult to interpret but can make predictions from problems that are traditionally difficult for computers to solve, such as classifying images, playing “go” at superhuman levels, or even creating novel portraiture.