A.  Approach and Limitations

We focus on a specific set of measures and study their empirical relationships in order to create an empirical basis that can aid the development of comprehensive theories of decision-making. Our approach is complementary to others in the literature. One standard approach is to focus on specific theoretical links (e.g., Chakraborty, Halevy, and Saito 2020, which links time and risk preferences). Others start with a specific theoretical mechanism and then study—theoretically and, in some cases, also empirically—which behaviors it can generate. This is the case of popular modern research programs on rational inattention (Stevens 2020; Woodford 2020; Khaw, Li, and Woodford 2021; Frydman and Jin 2022), salience (Bordalo, Gennaioli, and Shleifer 2012, 2013; Li and Camerer 2022), limited strategic thinking (Camerer, Ho, and Chong 2004; Crawford, Costa-Gomes, and Iriberri 2013; Farhi and Werning 2019; García-Schmidt and Woodford 2019), incomplete preferences (Masatlioglu and Ok 2014; Cerreia-Vioglio, Dillenberger, and Ortoleva 2015), preference imprecision (Butler and Loomes 2007, 2011), and cognitive uncertainty (Enke and Graeber 2019). While these important programs are complementary to our approach, our investigation is primarily empirical and does not seek to relate a large number of behaviors to a single underlying mechanism. Rather, it aims to construct a basis for theoretical modeling by exploring the empirical structure of our data, which will turn out to be more complex than what can be explained by a single mechanism.

Examining all links between econographics, as we do—including many not studied by theories that seek to link different behaviors—has a number of benefits, especially when compared with the standard approach of theorizing about a particular connection between two behaviors and testing it. First, as correlations are not transitive, identifying underlying structures requires observing all links (or lack thereof) between behaviors.² Second, once that structure is identified, its components can be examined to see whether they are associated with existing constructs—such as demographics or cognitive ability. Third, by measuring all behaviors simultaneously in a representative sample, we ensure that the patterns we identify are not due to shifting participant populations between studies.³

The empirical structure we uncover could have taken one of three forms, with different implications. First, there may be no discernible structure, suggesting the continued usefulness of examining behaviors in isolation. Second, there may be a structure that matches well with some existing theoretical approach. Third, it could be that a structure exists but does not match any extant theory. Our results are somewhere between the second and third possibilities: extant theories match some of the correlations we observe, but no theory explains the patterns in a specific domain, let alone the overall patterns we observe. This suggests the need for further theoretical exploration, for which we hope we give an empirical basis.

There are inherent constraints on the number and type of behaviors we could elicit on an incentivized survey. This meant that we had to leave out many interesting phenomena. While we would like to have been able to capture many other behaviors, making choices was unavoidable. Some measures were infeasible in our context: for example, measuring time preferences using a real-effort task (Augenblick, Niederle, and Sprenger 2015; Cohen et al. 2020) or preferences for competition (Niederle and Vesterlund 2007); such measures, while interesting, would have taken as much time as approximately a half-dozen of our other elicitations combined. Other measures—such as inattention, character, and some noncognitive skills—lack clear gold-standard elicitations. There are standard measures of cognitive skills (IQ and cognitive reflection) and personality (see, for early examples in economics, Almlund et al. 2011; Becker et al. 2012). We choose to focus on cognitive skills because of the possibility of their central role in behavioral economics (see, e.g., Benjamin, Brown, and Shapiro 2013; Stango and Zinman, forthcoming). Future studies can and should focus on alternative sets of behaviors. Indeed, as we discuss in section VII, some contemporaneous studies already are.

B.  Analysis and Results

Describing the 378 correlations between 21 econographic variables, as well as between cognitive and demographic variables, is a daunting task. However, we are aided by the fact that many of the econographics fall into clusters of variables—featuring high intracluster correlations and low intercluster correlations. To summarize them, we make use of principal components analysis (PCA), a statistical technique that produces components—linear combinations of variables—that explain as much variation in the underlying behaviors as possible, as discussed in section III.⁴

In our analysis, discussed at length in section IV, we are aided by a fact that was not ex ante obvious: risk and social preferences are largely—but, as we will see below, not completely—independent. Thus, we first study social and risk preferences separately, before combining them with measures of time preferences and overconfidence.

Our results show that there is significant scope for representing our eight measures of social preferences in a more parsimonious way. The behaviors we measure break down into three clusters, as summarized in table 1. In particular, altruism, trust, and two different types of reciprocity form a cluster. Pro-social punishment and antisocial punishment constitute a second cluster. The third cluster is formed by two different types of inequality aversion—dislike of having more than another person and a dislike of having less. As we discuss below, these results are not in line with any of the existing theories of social preferences.

Risk preferences show a structure that is less parsimonious than standard theory. Our nine measures form three clusters, as summarized in table 1. Two separate ones contain different measures of risk attitudes toward ordinary lotteries. One cluster is formed by the willingness to accept (WTA) for a lottery ticket, together with risk aversion for lotteries with gains, losses, and gains and losses (see sec. II.B) as measured by certainty equivalents. The other cluster is formed by the willingness to pay (WTP) for a lottery ticket, along with risk aversion as measured by lottery equivalents. While the members of each cluster are highly related to each other, they are largely independent across clusters, suggesting the presence of two separate and independent forms of risk attitude toward lotteries.⁵ These two components have different relationships with other features (e.g., age), as shown in section V. The third cluster consists of aversion to compound lotteries and ambiguity aversion. These behaviors are highly correlated, consistent with previous studies (Halevy 2007; Dean and Ortoleva 2019; Gillen, Snowberg, and Yariv 2019). Our richer data allow us to document that they are largely unrelated to other aspects of risk preferences.

Analyzing risk and social preferences together—along with overconfidence and time preferences—results in six components, as summarized in table 1. Two social components and two risk components are largely unchanged. The three overconfidence measures form a new component.⁶ Patience loads most heavily (and negatively) on the punishment component of social preferences. However, one social component—inequality aversion—and one risk component—related to WTP—combine with each other, showing the possibility of further parsimony in the representation. Of course, the generic idea of a connection between risk and inequality aversion has been suggested before. For example, from behind the “veil of ignorance” more inequality creates more risk (Carlsson, Daruvala, and Johansson-Stenman 2005). Our results indicate, on the one hand, that this has some empirical support (unlike other possible connections that have been suggested). On the other hand, they show how this holds for only one of the two components of risk attitudes toward ordinary lotteries (in particular, the one related to WTP), not the other (the one related to WTA).

We also examine the relationships between the components we identify and cognitive abilities and demographics, in section V. Four of the six components are correlated with cognitive abilities: higher cognitive ability is associated with a lower propensity to punish and greater patience, lower overconfidence, and higher generosity. Moreover, we document a relationship between cognitive abilities and risk preferences, but with only one of the two components linked with risk aversion—the one joined with inequality aversion. Connecting with demographics, the strongest links are with education and income; moreover, these variables do not seem to be simply proxying for cognitive ability but have their own individual effect.

C.  Relation to Theory and Literature

Finally, we draw out the implications of our data for existing theories, in section VI, some of which were alluded to above. Broadly speaking, we show that while there is some overlap between our data and common theories of social and risk preferences, no theory explains the overall patterns well, even within a specific domain. For social preferences, the three outcome-based models we consider—altruistic preferences, social welfare, and inequality aversion—predict that many of the measures that make up the Generosity component (described in table 1) should be positively correlated.⁷ However, these theories also predict a relationship with inequality aversion, which we do not observe. For risk preferences, models delineating risk and uncertainty do not match our data. Moreover, although reference points are clearly important in explaining the split between risk-aversion measures associated with WTA and those with WTP, common theories of reference dependence make additional predictions that are not consistent with our data.

Our study is uniquely suited to our aim of understanding the empirical basis for more parsimonious behavioral models across the risk, social, time, and confidence domains. There is a significant literature, summarized in section VII, that examines the correlation between two or three behavioral measures and/or cognitive ability. These more limited sets of relationships are not able to recover all of the nuanced structure we document here. There are a small number of studies that do measure more behaviors (Burks et al. 2009; Dohmen et al. 2018; Falk et al. 2018; Dean and Ortoleva 2019; Stango and Zinman, forthcoming). As discussed in section VII, these studies differ from ours in terms of behaviors examined, incentivization, representativeness, and/or how they deal with measurement error.

A.  Social Preferences

There are many examples where people’s actions take into account the preferences and beliefs of others, even in nonstrategic settings. The motivating factors behind these acts are often given the broad term of social preferences.

Altruism is defined as giving to strangers while expecting nothing in return. In experimental economics, it is usually measured using the dictator game (Forsythe et al. 1994; Falk et al. 2013), in which one participant decides unilaterally how to split money between themselves and another person. Following this literature, we measure Altruism as the amount given to another person in the dictator game.

Trust and reciprocity are intertwined. To understand why, imagine that a stranger asks for money for a sure-thing investment. In order to provide money in such a venture, you must trust that he will give some of the proceeds back to you. The act of giving money back is reciprocation, which may depend on how much money you gave to the stranger. We measure these concepts through a standard trust game: one participant (the sender) decides how much of an endowment to send to a second (the receiver). This amount is doubled by the experimenter, and then the receiver decides how much to send back—which is also doubled (Berg, Dickhaut, and McCabe 1995). We measure Trust as the amount sent by the participant when they are in the role of the sender. The original sender will also take the role of receiver in a different interaction: Reciprocity: Low corresponds to the amount sent back when receiving the lowest amount, and Reciprocity: High corresponds to the amount sent back when receiving the maximum amount.¹⁰

People are willing to punish others for what they perceive as bad behavior—even when that behavior does not directly affect them—and punishment is costly. To measure this, we allow participants to observe a trust game in which the sender gives all the money they have and the receiver returns nothing. We give each participant a stock of points to punish the receiver—that is, to pay a cost to reduce the points the receiver gets: the amount used is Prosocial Punishment. Prior studies document that a significant minority of people—the percent possibly depending on culture—also punish the sender (Herrmann, Thöni, and Gächter 2008). Thus, we also give a separate stock of points that can be used to punish the person who sent all their money. The amount used to punish the sender is Antisocial Punishment.

Many people seem uncomfortable with having a different amount (greater or less) than others, a phenomenon known as inequality aversion (Fehr and Schmidt 1999; Charness and Rabin 2002; Kerschbamer 2015). Dislike Having Less is how much a person is willing to forgo in order to ensure that they will not have less than another person. Dislike Having More is how much a person is willing to forgo in order to ensure that they will not have more than another person.

B.  Measures of Risk Attitudes

To measure attitudes toward risk and uncertainty, we elicit the valuation of various prospects. All lotteries involve only two possible payoffs, and most assign 50% probability to each.

Following the standard approach, we identify the behavioral manifestation of risk aversion as valuing a lottery at less than its expected value. Extensive research shows that the patterns of valuation depend on whether a lottery contains positive payoffs, negative payoffs, or both a positive and a negative payoff.¹¹ Thus, we include three measures of risk aversion: Risk Aversion: Gains elicits a participant’s certainty equivalent for a lottery containing nonnegative payoffs, Risk Aversion: Losses elicits a participant’s certainty equivalent for a lottery with nonpositive payoffs, and Risk Aversion: Gain/Loss elicits a participant’s certainty equivalent for a lottery with one positive and one negative payoff (Cohen, Jaffray, and Said 1987; Holt and Laury 2002). The difference between the expected value of the lottery and a participant’s value is used in the analysis, so larger numbers indicate more risk aversion.

The endowment effect is the phenomenon that, on average, people value a good more highly if they possess, or are endowed with, it. In our implementation, WTP is the amount a participant is willing to pay for a lottery ticket, and WTA is the amount the participant is willing to accept for the same ticket when she or he is endowed with it. The difference between WTA and WTP is the Endowment Effect (Kahneman, Knetsch, and Thaler 1990). We discuss this as a risk preference because the object being bought and sold is a lottery ticket, but more importantly, because of the patterns revealed in the analysis of section IV.C. However, as the pattern of correlations in our data suggests that WTA and WTP are also fundamental behaviors, we primarily examine these rather than the endowment effect.

Risk attitudes often change when one of the available options offers certainty, as demonstrated through the common-ratio effect (Allais 1953). Under expected utility, when the winning probabilities of two lotteries are scaled down by a common factor, a person’s ranking over those lotteries should not change. However, this is often not the case. To capture this effect, we ask the participant to make two choices, one that measures risk aversion with a certain alternative and another in which both options are risky. In Risk Aversion: CR Certain (where CR = Common Ratio), we elicit the amount b such that the participant is indifferent between a certain amount a and a lottery paying b with probability α (and zero otherwise): that is, a lottery equivalent of a sure amount. In Risk Aversion: CR Lottery, we elicit the amount c such that the participant is indifferent between a lottery paying a with probability $1/x$ (and zero otherwise) and c with probability $\alpha /x$ (and zero otherwise). Under expected utility, $b=c$; the Common Ratio measure is then $b-c$ (Dean and Ortoleva 2019). In keeping with our treatment of the endowment effect, we enter the constituent measures in most analyses.¹²

Ambiguity aversion is a preference (or beliefs that lead to a preference) for prospects with known probabilities over those with unknown probabilities. To measure it, we use an ambiguous urn filled with balls of two different colors: one color gives the participant a positive payoff, and the other gives them zero. Participants do not know the proportions of the different colors of balls in the urn but are allowed to choose which color gives a positive payoff. They are then asked for their certainty equivalent for a draw from this urn. If participants have a prior over the composition of the urn, they must believe that a draw from the urn has a winning probability of at least 50%, yet many participants prefer a 50/50 lottery with known odds. The difference between the certainty equivalents for draws from the risky urn—with a known composition of 50% of each color—and the ambiguous urn is Ambiguity Aversion.¹³ Similarly, a draw from a risky urn is usually more highly valued than one in which the number of balls is unknown but drawn from a uniform distribution—that is, a compound lottery. The difference between the certainty equivalents for draws from the risky urn and a compound urn is Compound-Lottery Aversion (Halevy 2007).

C.  Overconfidence

Overconfidence can be divided into three types. Overestimation refers to a person’s estimate of her performance on a task (vs. her actual performance). Overplacement refers to her perceived performance relative to other participants (vs. her real relative performance). In order to measure these phenomena, we ask participants to complete two tasks: on one, we ask them to estimate their performance, and on the other, we ask them to estimate their performance relative to others taking the survey. The difference between these subjective estimations and actual performance, in absolute or percentile terms, gives us Overestimation or Overplacement, respectively (Moore and Healy 2008).

Overprecision refers to a belief that one’s information is more precise than it actually is. We ask participants to estimate a number (such as the year the telephone was invented) and then tell us how close they think they were to the correct answer. To difference out Overprecision from justified precision, we regress how close the participant thought they were on a fourth-order polynomial of their accuracy (Ortoleva and Snowberg 2015a, 2015b).¹⁴

D.  Patience (Time Preferences)

A payoff sometime in the future is generally seen as less valuable than a payoff of the same size today. The value today of a fixed future payoff is Patience (Andersen et al. 2008).¹⁵

E.  Choice of Measures

The choice of measures to include was driven largely by a desire to focus on “the basics” and by time constraints inherent in the study design. This general logic still left room for many subjective choices. In this subsection, we try to lay out how this logic was applied and how subjective considerations ultimately led to the set of measures we focused on.

Most of our measures are simple deviations from the standard neoclassical model of a selfish expected-utility maximizer. Behavior under that model is relatively simple: choices involving risk and uncertainty are driven by the curvature of the utility function. Adding time adds a discount rate. Choices involving other individuals are no different than choices involving the market: all result in maximizing the welfare of the individual, with no regard for the welfare of others. Finally, complex choices are processed no differently than simple ones. In the domain of risk, deviations from this model allow for distortions in the way these choices are perceived, including reference points and violations of independence. In the domain of social choice, deviations allow for caring about the welfare of others. Deviations in the domain of cognitive limitations include biased cognitive processing, how those biases affect strategic interactions, and so on.

We focused mainly on preferences in the risk and social domains. These areas have generated enormous interest both within and beyond behavioral economics. Arguably, they constitute the areas where behavioral economic models have been most extensively applied in other fields (e.g., finance and development economics). Each area has generated multiple empirical regularities that are inconsistent with the standard model of economic decision-making, leading to model proliferation. As different modeling approaches suggest different empirical relations between these regularities, it is particularly useful to establish the correlational structure between these behaviors to guide future modeling work.

Within each domain, the measures we included were guided by a combination of theory and importance to the literature. In the case of risk, we first focused on the two classic violations of independence: common-ratio and ambiguity-aversion measures. Both have been central to generating huge literatures on non–expected utility theory. Moreover, both have been explained as manifestations of probability weighting (e.g., Quiggin 1982; Segal 1987), as has aversion to compound lotteries (Segal 1990). Probability weighting has also been advanced as a possible explanation for small-stakes risk aversion.

We then focus on the second major behavioral force in choice under risk: reference dependence. Since the seminal work of Kahneman and Tversky (1979), the ideas of reference-dependent risk attitudes and loss aversion have been central to understanding risky choice. The key behavioral patterns associated with reference dependence are differing risk attitudes in the gain-and-loss domain and an increase in risk aversion for lotteries that include both gains and losses, thus motivating our three risk-aversion measures. At the same time, the idea of loss aversion has been used to explain the endowment effect (see, e.g., Kahneman, Knetsch, and Thaler 1990; Kőszegi and Rabin 2007). This motivated us to also include a measure of the endowment effect in our study.

We would argue that our nine measures of risk attitude span the behaviors that have been most influential in guiding the development of models of risky choice and are the most obvious manifestations of the three key theoretical constructs in the literature—ambiguity attitudes, nonlinear probability weighting, and reference dependence.

In the case of social preferences, we selected a set of measures that both reflect standard experiments and connect to most widespread theories of social preferences in a parsimonious way. In virtually all models of social preferences, people account for the other’s well-being. This is most directly captured by our measures of altruism and trust, which are both associated with classical measures in their own right. Common approaches, most prominently that of Fehr and Schmidt (1999), add the role of inequality aversion, with a specific distinction between preferences over advantageous and disadvantageous inequality. Other theories point toward the importance of reciprocity and different types of punishment. Importantly, different theories make different predictions about the distribution and correlation of these behaviors.

We made less of an attempt to cover behaviors thought to be driven by biases or mistakes, largely because of time constraints on the survey. Moreover, many of these judgment biases are carefully considered in Stango and Zinman (forthcoming), which can be seen as a companion to our more intensive focus on preferences. However, we did have space for a couple of such behaviors and chose to focus on overconfidence (Moore and Healy 2008) and level-K thinking (Nagel 1995). Overconfidence was motivated by a longer-term interest in the relationship between economic and political behavior, and prior results suggest that overconfidence is an important link between the two (Ortoleva and Snowberg 2015a, 2015b). Level-K thinking has long been a subject of interest for one of our coauthors, although our elicitations did not seem to produce reliable results; see figure E.1.

Naturally, there are other areas arguably of equal interest to behavioral economists that we have ruled out on various grounds. While there are many interesting behavioral phenomena and traits in strategic settings—for example, quantal response and competitiveness—the types of questions needed to measure these behaviors comprehensively would be too time-consuming for our survey. For similar reasons, we were able to measure only time preferences using money, rather than, say, real-effort tasks. Given the well-known issues with this approach, we restricted ourselves to a single time-preference measure. In the interests of parsimony, we measured reference dependence within the context of risk preferences, rather than separately, and did not measure violations of the independence of irrelevant alternatives, such as the compromise or asymmetric dominance effect. Overall, time constraints in the survey forced us to make choices and to leave out many measures that would be interesting to explore in future work.

A.  Survey Implementation

Administering an incentivized survey to a representative population presents challenges not normally dealt with in lab environments. To surmount these challenges, we partnered with YouGov, a worldwide leader in online surveys serving the public, businesses, and governments.¹⁶ Our study was given to a representative sample of 1,000 US adults between March 30 and April 14, 2016. We consulted extensively with YouGov on our study design to utilize its expertise in survey design and implementation.

Constructing a representative sample is difficult, given variation in response rates. In order to do so, most modern surveys weight on demographics. YouGov supplements this with its own panel of participants. It continually recruits new people, especially from difficult-to-reach and low-socioeconomic-status groups. To generate a representative sample, it randomly draws people from various Census Bureau products and matches them on observables to members of its panel. Oversampling and differential response rates lead to the over- and underrepresentation of certain populations. Thus, YouGov provides sample weights to recover estimates that would be obtained from a fully representative sample. According to Pew Research, YouGov’s sampling and weighting procedure yields more representative samples than traditional probability (i.e., random) sampling methods, including Pew’s own probability sample (Kennedy et al. 2016; YouGov is Sample I). We use these weights throughout the paper.¹⁷

Incentivized questions pose additional challenges: stakes and whether the experimenter is seen as credible in making future payments or running randomizations as specified. Two randomly selected questions were chosen for payment.¹⁸ To enhance the credibility of our study, we took advantage of YouGov’s relationship with its panel and restricted the sample to those who had already been paid (in cash or prizes) for their participation in surveys.

All outcomes to incentivized questions were expressed in points. This is an internal YouGov currency used to pay participants. It can be converted to US dollars or prizes, using the approximate rate of $0.001 per point.¹⁹ The average payment to participants was around 9,000 points (or $9). The survey took participants between 45 minutes and an hour. This compensation level is quite high for an internet survey and represents a rate of pay approximately three times the average for similar surveys.

B.  Measurement Error

Measurement error causes a downward bias in correlations. This would make it more difficult to pick out the clusters of interrelated measures. Moreover, this downward bias would also make it difficult to know which behaviors are actually not related. To circumvent this issue, we use the ORIV technique of Gillen, Snowberg, and Yariv (2019), which uses the instrumental variables approach to errors in variables to produce efficient, consistent estimates. At the heart of this method is duplicate observations of variables—multiple elicitations that are similar but not exactly the same—that are likely to have orthogonal measurement error. This technique takes duplicate measures and estimates a stacked regression in which each measure of one behavior is used as both an independent variable and an instrument. This is done twice, once for each measure of the other behavior, effectively averaging across all four specifications in the stacked regression model.²⁰

Our setting requires us to deal with an additional issue: constructed variables will often have correlated measurement error due to the nature of their construction. For example, the Compound-Lottery Aversion and Ambiguity Aversion measures are both constructed by taking some behavior (the certainty equivalent for a compound or ambiguous urn) and subtracting off the same quantity measured with error (certainty equivalent of a risky urn). This leads to correlation in the measurement error of Ambiguity Aversion and Compound-Lottery Aversion. To avoid spurious correlations, we make use of the fact that we have two observations for each measure and modify the ORIV procedure so that the measurement error in the instrument is uncorrelated with the measurement error in the left-hand-side variable.²¹ We use this formulation for all sets of variables constructed from two elicitations: Compound-Lottery Aversion and Ambiguity Aversion, and Overplacement and Overprecision.

C.  Multiple-Hypothesis Testing

This paper displays a large number of correlations and standard errors. There are no theoretical predictions for most of the correlations we examine. Thus, we omit any description of statistical testing or significance from our tables. However, if one were interested in null-hypothesis statistical testing, the appropriate critical value for significance at the 5% level is between 1.96 for a single-hypothesis test and 3.82 using a Bonferroni correction (Dunn 1958, 1961) for all 378 correlations underlying sections IV or V. List, Shaikh, and Xu (2016) provide an alternative approach that is generally less restrictive than a Bonferroni correction but still provides valid inference.

D.  PCA

We examine correlation matrices directly and use PCA to summarize them. As PCA is an analysis of a correlation matrix and our correlation matrices are corrected for measurement error, so too are the results of PCA. The aim of PCA is to extract the m components most useful for explaining $n>m$ variables. Components are linear combinations of the variables. The first component is constructed to capture the highest possible fraction of variance in the data (subject to the constraint that the linear weights sum to 1), the second to capture the highest fraction of the remaining variance, conditional on being orthogonal to the first component, and so on.²²

Once components are identified, the key question is, How many are necessary to provide a good description of the underlying data? Heuristically, we want to retain components only when the marginal explanatory power is high. In order to determine the number of components to retain, we use an approach that captures this intuition: parallel analysis. Parallel analysis creates many random data sets with the same numbers of observations and variables as the original data. The average eigenvalues of the resulting correlation matrices are then computed. Components are kept as long as their associated eigenvalues are greater in the actual data than the average in the randomly generated data.²³

The retained components help in understanding the relationship between the original variables in the data set. The correlation between a component and a variable is called the variable’s loading on that component. Variables that load heavily on the same component are highly related. In order to facilitate interpretation, retained components can be rotated relative to the data. Following standard practice, we rotate the resulting components using the varimax rotation (Furr 2017, 92). This rotates the basis identified from the retained components to maximize the variance of the squared loadings, easing interpretation of the components. However, as we will see below, the patterns in the components largely line up with apparent patterns in the correlation matrices, making interpretation straightforward.

In section V, we analyze whether the components we identify correlate with cognitive and demographic measures. As we have two elicitations of most variables, to compute the components we multiply the weights from the PCA by the average of the two elicitations.

A.  Summary Statistics

The summary statistics in Table 2 show that behavior in our data is consistent with standard findings in the laboratory. In addition to summary statistics, we also show the percentage of participants whose responses are in the “expected” direction.²⁵ Surveying the information in this column, the majority of participants are risk averse or risk neutral over gains, risk loving over losses, and inequality averse; exhibit an endowment effect and the common-ratio effect; are ambiguity averse; have a negative reaction to compound lotteries; and are overconfident. (A majority of participants are not all of these things simultaneously.) Note that although patience in this table is represented as a discount rate, in the correlation analysis we code the variable as discussed in section II.D. Either coding gives the same (directional) interpretation to correlations. However, the coding in section II.D is linear in a participant’s answer, allowing for the measurement error correction discussed in section III.B.

Our data exhibit fairly standard levels of noise. As discussed in Gillen, Snowberg, and Yariv (2019) and Snowberg and Yariv (2021), the correlation of duplicate measures—subtracted from 1—gives the level of noise in a particular elicitation. Gillen, Snowberg, and Yariv (2019) report correlations of around 0.65 between duplicate measures, using data from Caltech undergraduates. In most cases, the correlations we observe—in the final column of Table 2—are somewhat higher. This implies that our data are less noisy than similar data obtained from Caltech undergraduates. The exceptions are the overconfidence measures, which are noisier than the rest.²⁶ When there is no correlation listed, we have only one elicitation of that behavior.

B.  Links between Social Preferences

There is ample opportunity to create a more parsimonious representation of the social preferences we measure: altruism, trust, anti- and prosocial punishment, and distributional preferences. These measures fall into three clusters shown in correlation table 3: one formed by the two measures of reciprocity, and altruism and trust; a second formed by the willingness to punish pro- and antisocial behavior; and a third formed by our two measures of inequality aversion. These clusters are characterized by high within-cluster correlations and low correlations between measures in different clusters. To make these clusters visually apparent, we present the correlation matrix in the form of a “heat map,” where the shade of red indicates the magnitude of the correlation.²⁷

The first correlation in the first cluster—0.86 between the two reciprocity measures—is a useful example for interpretation. There are distinctions between those who are more and less reciprocal when a partner is more or less generous, resulting in a less than perfect correlation. Yet the predominant behavioral distinction, reflected in the very high correlation, is how reciprocal someone is in all conditions. This overarching behavior is (empirically) related to both trust and altruism, although more closely to the former. We note that although some readers may have anticipated some of these correlations, the fact that the literature distinguishes between, say, different forms of reciprocity, indicates that these anticipations are not universally shared.

The second cluster contains the two punishment measures.²⁸ Like other clusters, they are highly correlated with each other but with few other measures. However, these measures are characterized by both an extensive and an intensive margin. The extensive margin is whether or not someone punishes and the intensive margin is how much punishment a person metes out. This leads to obvious questions about how these two margins contribute to the overall relationship. The correlations on both margins are roughly equal. About two-thirds of participants engage in prosocial punishment, whereas only one-third engage in antisocial punishment. However, almost everyone who engages in antisocial punishment also engages in prosocial punishment. Of those who engage in both types of punishment, there is a correlation of ∼0.4 in the amount they choose to spend on punishment of both parties. Both the extensive and intensive margins are poorly related to other measures of social preferences.

The third cluster contains the distributional preferences measures. It features the weakest intracluster correlation. Moreover, both measures in this cluster—Dislike Having More and Dislike Having Less—are moderately correlated with other econographics. These moderate correlations extend to some measures in the risk domain, leading to this component combining with risk preferences in the analysis of all 21 econographics in section IV.D.

Before turning to the PCA, it is worth noting that there is very little in the literature that indicates which specific correlations we should, and should not, find. This could lead to a lot of plausible storytelling. For example, one might believe that the dislike of having more (one form of inequality aversion) entirely motivates altruism and that a dislike of having less (another form) motivates punishment. Yet these patterns are not present in our data. Alternatively, one might expect that the predominant feature of distributional preferences is the presence or absence of a preference for equality, which would generate the observed correlation between Dislike Having More and Dislike Having Less. Whatever one’s priors (or lack thereof), these results should be informative. We discuss what we believe are the biggest takeaways for theory in section VI.

The PCA of these correlations shows the same patterns: results appear in Table 4. Three components are suggested for inclusion under parallel analysis—see figure C.1 for the scree plot. Together, these three components explain 68% of the variation in the eight measures of social preferences we explore here.

These three components have fairly obvious interpretations. The first is generosity in behaviors that directly influence the well-being of another person. The second is a general affinity for punishment. The third and final component captures both types of inequality aversion. The components are thus named accordingly: Generosity, Punishment, and Inequality Aversion.

Overall, these results show that there is a clear structure to the connections between different social preferences. These measures tend to group in clusters with high within-cluster correlations and low across-cluster correlations, leading to the latent structure shown by the PCA.

C.  Links between Risk Attitudes

In this subsection, we show that models of decision-making under risk and uncertainty may be too parsimonious. As shown by the clear clusters in table 5, Ambiguity Aversion and Compound-Lottery Aversion group together, blurring the line between risk and uncertainty. Moreover, we find two clusters of risk attitudes: one related to WTA and the other to WTP.

The easiest cluster to interpret contains Ambiguity Aversion and Compound-Lottery Aversion.²⁹ This high correlation is consistent with extant empirical work (Halevy 2007; Dean and Ortoleva 2019; Gillen, Snowberg, and Yariv 2019) and theoretical observation (Segal 1990; Dean and Ortoleva 2019). Note that both of these measures are essentially unrelated to measures of risk attitudes toward ordinary lotteries, suggesting either a delineation between risk and uncertainty, with compound lotteries grouped with ambiguity aversion, or the lack of a clean line between risk and ambiguity.

The two remaining clusters both contain risk attitudes and suggest two separate aspects of risk preferences. The first contains Risk Aversion: Gains, Risk Aversion: Losses, Risk Aversion: Gain/Loss, and WTA, and the second includes Risk Aversion: CR Certainty, Risk Aversion: CR Lottery, and WTP. Crucially, there are low correlations between the measures in different clusters (with the exception of the relationship between Risk Aversion: Losses and WTP—a correlation of 0.3, meaning that risk aversion using WTP is negatively correlated with risk aversion over losses). This suggests that there are two separate aspects of risk aversion that are largely unrelated to each other or to preferences under uncertainty. Note that these two aspects are both within a fairly narrow domain: the valuation of risky lotteries. Thus, unlike psychology research, which has shown differences in risk preferences across outcome domains, these patterns suggest differences in preferences within the domain of money lotteries—the most studied domain in choice theory and experimental and behavioral economics.

These two separate aspects of risk preference align with WTA and WTP, respectively, giving this division substantive and theoretical importance. We find that certainty equivalents fall into one cluster while lottery equivalents fall into another. This is in line with prior research finding that different fixed elements in an MPL lead to different degrees of risk aversion. Here, as in the literature, this is consistent with the fixed element of the MPL acting as a reference point (Sprenger 2015). Our results add, first, that these groups are largely uncorrelated with each other and, second, that they naturally align with WTA and WTP. WTA—where the lottery is explicitly the reference point—naturally aligns with certainty equivalents: measures where the lottery is fixed. Similarly, WTP aligns with measures with a fixed monetary amount. At the same time, as we discuss in section VI and more in depth in Chapman et al. (2022b), our data are difficult to reconcile with current theoretical explanations. The most visible signature of this difficulty in table 5 is the lack of an obvious component of loss aversion positively related to WTA and negatively related to Risk Aversion: Gain/Loss.

Before proceeding, it is worth discussing the two measures in the correlation table that we do not include in the clusters or in the following PCA: the endowment effect and the common-ratio effect. Both measures are the difference of two other measures we already include: for example, the endowment effect is the difference between WTA and WTP. Thus, including all three measures in a PCA makes little sense. As described above, our own recent research compiles extensive evidence that WTA and WTP should be treated as separate behavioral measures (Chapman et al. 2022b). Thus, we include the component measures (WTA and WTP) rather than the endowment effect. We take the same approach with the common-ratio measures: entering the constituent measures into the correlation and PCA.³⁰

Once again, the PCA in Table 6 confirms the visual patterns in the correlation table. The first three components explain 66% of the variation in the nine measures of risk preferences considered here. The first and the second clearly capture different aspects of risk attitudes. Following the discussion above, we use the names Risk Aversion: WTA and Risk Aversion: WTP to describe them. The third encompasses more complex lotteries, which may induce uncertainty: hence, we use the name Uncertainty.

D.  Putting It All Together

Analyzing all 21 econographics together leads to six components. The structure of components from the risk and social domains is preserved; however, one social and one risk component combine. The sixth component is comprised of the three overconfidence measures. Time preferences load on several components, most heavily on the Punishment component.

Analyzing risk and social preferences, together with overconfidence and time preferences, is straightforward because there are few important relationships between risk and social preferences. However, they are not completely unrelated. One of the social preference components (Inequality Aversion) combines with a risk-preference component (Risk Aversion: WTP). Otherwise, the structure in the previous two subsections is largely unaltered.

The first and second components in Table 7 are similar to the first components in the social and risk analyses, respectively. Thus, we retain the same names. Patience loads moderately on both components: more generous and less risk-averse people are more patient.

The third component is of particular interest. It combines the measures included in the second risk component, Risk Aversion: WTP, with those of the third social component, Inequality Aversion. This suggests that some aspects of social and risk preferences are distinct, while others are more related. Moreover, this relationship follows a clear pattern building upon the previously identified components: Inequality Aversion combines with Risk Aversion: WTP. We note that both of these components deal with aversion to spreads—in possible payoffs or distributional assignments. Thus, we conjecture that a similar form of caution may lead participants to both dislike entering conditions of risk and generically dislike unequal allocations. Note, however, that this pertains only to entering a situation of risk (WTP for a lottery), rather than leaving one (WTA). In line with its constituent parts, we call this component Inequality Aversion/WTP.

The fourth component loads heavily on all three overconfidence measures; naturally, we call it Overconfidence. Given existing work on the conceptual distinctions between different types of overconfidence, the fact that they would load on a single component may not have been ex ante obvious (Moore and Healy 2008).

The fifth is the social component Punishment, to which (a bit of) time preferences have been added. People who score highly on this component enjoy both pro- and antisocial punishment and are impatient. Thus, we dub this new component Impulsivity.

The sixth component is essentially the same as the third risk component. Thus, we continue to call this component Uncertainty.

There are a number of ways to examine the robustness of our results. We discuss two here—changing the number of components and using other methods to extract latent variables—and another—adding and removing variables—in section VIII.

The Inequality Aversion/WTA component is robust to adding or subtracting components. Reducing the number of components to five, as shown in table E.3, causes the Uncertainty component to combine with both the Overconfidence and Impulsivity components but otherwise leaves the structure of the components qualitatively unchanged. Perhaps more interesting is what happens when we add components. With either seven or eight components—the latter recommended by the Kaiser criterion—the Overconfidence component splits into two, as shown in tables E.4 and E.5. Both of these overconfidence components contain overprecision, but one contains overestimation, and the other, overplacement. Adding an eighth component further splits the Generosity component into two, with the reciprocity measures on one component and the altruism and trust measures on the other. The rest of the components are largely unchanged by the inclusion of additional components.

Using other ways of computing the correlation matrix or other latent dimension recovery techniques has very little effect on our results, as shown in tables E.6–E.9. In particular, our results are robust to using unweighted measures or the average of measures (rather than ORIV) to compute the correlation matrix. Moreover, one can use Spearman rank-order correlations, which are robust to nonlinear relationships and outliers, to compute the correlation matrix, and this also results in very little change to the structure shown in Table 7. Finally, one can use factor analysis (on the average of measures), rather than PCA. This, too, produces qualitatively similar results.

Overall, the underlying structure of our data can be summarized with four points. First, six interpretable and separate components explain 60% of the variance of our 21 variables. Second, the social and risk components do not change from the earlier analyses in tables 4 and 6, but two components—Risk Aversion: WTP and Inequality Aversion—collapse into one. Third, time preferences are captured by several of the existing components, possibly signaling that they are related to many elements, although none particularly strongly. Fourth, overconfidence measures are largely separate and are captured by a single component.

A.  Econographics and Cognitive Measures

There are several important relationships between our six econographic components and the two measures of cognition on our survey: a six-question IQ battery and the three-question cognitive-reflection test (CRT; Frederick 2005). These two measures have an ORIV correlation of >0.8.³¹

Our IQ measure consists of three questions from each of two classes of questions from the International Cognitive Ability Resource, a public domain intelligence measure (Condon and Revelle 2014). We chose three matrix-reasoning questions, similar to Raven’s Progressive Matrices. In these questions, participants determined which of a set of possibilities correctly completed a graphic pattern. We also constructed a second battery of three questions based on three-dimensional rotations: a drawing of a cube was shown, and participants had to identify which of a set of six other drawings of a cube were compatible. These questions were chosen to be of progressively greater difficulty in order to try to capture a variation in fluid intelligence in the general population.³²

There are several important relationships between our six econographic components and IQ or the CRT, shown in Table 8. Note that, unlike results in prior sections that were primarily concerned with magnitudes, here positive and negative correlations have very different interpretations. Consistent with the high correlation between the IQ and CRT measures, the general pattern of correlations is largely the same: higher cognitive ability is positively correlated with Generosity and negatively correlated with Inequality Aversion/WTP, Overconfidence, and Impulsivity.³³

These results show that, in general, the underlying behavioral components that we have identified are not simple proxies for intelligence. Moreover, there is a more nuanced relationship between our measures of intelligence and the extent to which a participant is “behavioral”—that is, behaves differently than a selfish, risk-neutral expected-utility maximizer. While it is the case that lower measured intelligence is related to higher Impulsivity and Inequality Aversion, it is also related to lower Generosity. Moreover, risk and uncertainty attitudes are largely unrelated to intelligence, with much of the positive relationship between Inequality Aversion/WTP, Overconfidence, and intelligence being driven by distributional preferences.³⁴

B.  Econographics and Demographics

There are interesting relationships in our data between our components and standard economic variables—such as income and education—or demographics—such as age and gender. Table 9 shows these correlations in panel A and the same correlations, controlling for our six-question IQ battery, in panel B.³⁵

Greater education and income are associated with higher Generosity and lower Impulsivity, as was the case with higher cognitive ability. This may be unsurprising, as education and income are often associated with higher cognitive ability. However, as panel B of Table 9 shows, the relationships with education and income remain even after IQ is controlled for. There is no association between income or education and Overconfidence or Inequality Aversion/WTP; however, once we control for cognitive abilities, we obtain that higher education is (slightly) correlated with higher Overconfidence. That is, even though more education increases participants’ own ability and knowledge, it seems to increase their perceptions of these skills even more. This suggests, if anything, that education is not a “cure” for nonstandard preferences.³⁶

Older people are more generous, and men are more overconfident, in accordance with the literature.³⁷ These results only become stronger once we control for cognitive abilities. Additionally, older people exhibit a higher Inequality Aversion/WTP score, while men have a slightly lower one. As with cognitive abilities, we find that neither demographic is related to the other risk component, Risk Aversion: WTA, adding nuance on the relation between these aspects and risk preferences. Finally, frequency of church attendance is positively associated with Generosity. It is also positively associated with Overconfidence, although not statistically significantly at conventional levels.

A.  Social Preferences

Theories of social preferences can capture some of the moments in our data, but no single theory can explain all the patterns we observe. Here we consider four common theories of social preferences: simple models of altruistic preferences, inequality aversion (Fehr and Schmidt 1999), and reciprocity (Rabin 1993) and a model incorporating the latter two features plus social welfare (Charness and Rabin 2002).³⁸

A simple model of altruistic preferences—in which utility is increasing in one’s own monetary payoff and that of others—predicts a positive correlation between our measures of Altruism, Trust, Reciprocity: High, and Reciprocity: Low, in line with the Generosity component (see tables 3, 4, and 7). This simple model also predicts that more altruistic people would have to be paid more to move to an uneven split in which the other person got less but paid less to move to a split in which the other person got more. That is, it also predicts that measures in the Generosity component should be positively correlated with Dislike Having More and negatively related to Dislike Having Less. There is some evidence for such a pattern in our data; however, it does not reflect the most parsimonious reading, as expressed by PCA. Moreover, a model of altruistic preferences would also predict a negative correlation between Dislike Having More and Dislike Having Less—the opposite of what we find in our data. The altruistic preferences model also predicts that people would accept a reduction in allocation to move away from an even split to one in which another person got more. In our data, the majority of non-indifferent participants require an increase in income (79% of non-indifferent participants in question 1 and 74% in question 2) to move away from the even split, the opposite of the theoretical prediction. Finally, a purely altruistic person would never engage in costly punishment.

Social welfare preferences modify the altruistic preferences model by assuming that the utility a decision-maker gets from the consumption of others grows faster when others are poorer than the decision-maker (Charness and Rabin 2002). Thus, altruistic behavior is governed by two parameters, one that dictates behavior when a person has more than others and one when they have less. Like the altruism model, this model also predicts a positive correlation between Altruism, Trust, Reciprocity: High, and Dislike Having More, as these are all situations in which the decision-maker will (choose to) have more than the other person. In contrast, Reciprocity: Low would be governed by a separate parameter, which would also determine Dislike Having Less and lead to a negative correlation between these two measures. This is not the primary pattern we see in the data. As with altruistic preferences, a person of this type would never engage in costly punishment.

Perhaps the most common characterization of social preferences is Fehr and Schmidt’s model of inequality aversion. This adds two parameters to the standard model: one codes how much the decision-maker dislikes having more than other people and the other how much they dislike having less. As with social welfare preferences, the first of these parameters governs behavior in Altruism, Trust, Reciprocity: High, and Dislike Having More, while Dislike Having Less is governed by the second parameter. The model of Bolton and Ockenfels (2000) would imply similar correlations. Again, this does not match the most parsimonious reading of our data. Moreover, a model of inequality aversion would predict that no money should be returned in the Reciprocity: Low measure, but that is not what happens empirically. Inequality-averse participants might engage in Prosocial Punishment (as this would reduce inequality) but not Antisocial Punishment (as this would increase inequality). This is inconsistent with the strong positive correlation we find between these two measures. The single punishment factor is also inconsistent with evidence from cross-cultural studies on public goods games with punishment, such as Gächter, Herrmann, and Thöni (2010), in which cultures tend to bifurcate between prosocial and antisocial punishment.

Other important theories of social preferences are not tested in our survey. Intention-based models, such as Rabin (1993), apply to only one of our choice environments—the trust game—that has a second player whose actions affect both players’ payoffs. Ideal tests of these models must collect actions, beliefs about others’ actions, and second-order beliefs (e.g., Charness and Dufwenberg 2006), which are hard to measure, especially given the constraints of our survey (Schotter and Trevino 2014). While we do not have ideal measures of intention, to test reciprocity, a simple comparison of our items can plausibly lead to a negative correlation between Reciprocity: High (in which participants have been treated fairly) and Reciprocity: Low (in which they have been treated unfairly). This is in contrast to what we find in our data.³⁹

B.  Risk Preferences

Our results exhibit inconsistencies with standard models of preferences under risk and uncertainty. These models typically separate risk and uncertainty, with the distinction being whether probabilities of outcomes are known or unknown (Arrow 1951; Gilboa and Marinacci 2016). The curvature of the utility function, and possibly aspects of non–expected utility, should affect behavior in both domains, while beliefs and ambiguity aversion should affect only choice under uncertainty. In our data, the classic delineation between risk and uncertainty fails to hold: attitudes toward compound lotteries, where probabilities are known, are related to ambiguity aversion, where they are not. This empirical relationship has been shown before, with different possible explanations (Halevy 2007; Dean and Ortoleva 2019; Gillen, Snowberg, and Yariv 2019). Segal (1990) proposes that ambiguous prospects are treated like compound lotteries, in which case probability weighting could explain both phenomena. This theory would further imply a link between Ambiguity/Compound-Lottery Aversion and the Common Ratio effect. However, we do not observe this relationship in our data. An alternative is to argue that compound lotteries are complex objects that people perceive as ambiguous (Dean and Ortoleva 2019). In either case, these results point to a different structure, where compound lotteries are associated with prospects with unknown probabilities.

The split into two separate types of risk preferences for objective, noncompound risk is also difficult to reconcile with models that treat risk aversion as a unitary phenomenon—for example, driven by the curvature of a utility function. As discussed at length in Chapman et al. (2022b), the specific clusters we find can be interpreted using the framework of Sprenger (2015), in which question structure serves as an implicit frame and determines reference points. In particular, Sprenger shows that certainty and probability equivalents yield different levels of risk aversion. He explains this finding by appealing to the different reference points these frames induce. In line with this hypothesis, the two clusters in our data are related to the two explicit frames we administer in the WTA and WTP tasks: certainty equivalents are correlated with WTA, while probability equivalents are correlated with WTP.

A natural question is whether this pattern of correlations is in line with existing theories of reference dependence. Cumulative prospect theory and the Kőszegi-Rabin model predict specific relationships between WTA, WTP, and the Endowment Effect and the three measures Risk Aversion: Gains, Risk Aversion: Losses, and Risk Aversion: Gain/Loss. While deriving these specific predictions and testing them is beyond the scope of this paper, Chapman et al. (2022b) shows that the predicted relationships fail to hold and in some cases are in the opposite direction of predictions from reference-dependence models. Thus, while the reference point induced in these questions is of clear first-order importance, existing theories of reference dependence cannot account for the overall pattern of correlations observed in our data.