I.  Introduction

Many markets are characterized by a substantial asymmetry between an incumbent provider and competing firms in that consumers know the contract with their current provider, but have to pay a search cost to be informed of alternative contracts. Once they observe other contracts, consumers have to pay a transaction cost to switch to alternative providers. This is the case, for instance, in markets such as electricity or gas, where liberalizations have taken place but the former incumbent still serves a large fraction of consumers. The incumbent can use this asymmetry to price discriminate between consumers with high and low search costs.

This paper studies how the optimal pricing policies of the incumbent and entrants depend on consumer search behavior. Our empirical analysis focuses on local retail electricity markets in Germany: each local market has an incumbent from the pre-liberalization era and many retailers that have entered the market since.¹ Consumers may search for tariffs at an online platform and decide whether to switch to a cheaper tariff offered by the incumbent—a form of price discrimination by the incumbent between searching and nonsearching consumers—or to an even cheaper rate offered by an entrant retailer. We show that differences in the fraction of searching consumers across local markets explain quite a large part of the observed heterogeneity in pricing behavior: in markets where consumers search more, there is more price discrimination by the incumbent and overall price dispersion is also larger. Our theoretical model shows that the empirical findings are consistent with the strategic incentives of market participants, but that other pricing patterns are also possible, and it also performs a welfare analysis.

By describing the model, the main features of the market become clear. Consumers observe the baseline price of the incumbent at no cost. Having observed this price, consumers decide whether or not to search for alternative tariffs. Search is costly and allows the consumers to observe all other prices in the market by consulting an online price-comparison platform. As consumers are heterogeneous in their search costs, some consumers search the platform, while others do not. At the platform, consumers choose between buying from the lowest-price entrant or staying with the incumbent at the incumbent’s online discount price. As the transaction costs of switching suppliers differ across consumers, some consumers who search the platform will stay with the incumbent, even if the incumbent’s discount price is not the lowest price on the platform. This way, the incumbent can price discriminate between consumers with high search costs (those who do not search) and lower search costs (those who search) and prevent searching consumers (those with high transaction costs) from switching to a retail competitor. We show that by varying the search cost distribution, this simple model can accommodate a rich pattern of pricing behaviors, including the one we find in our empirical analysis, where price dispersion and price discrimination increase with the fraction of consumers who search online, and where the incumbent raises its baseline price to consumers who do not search. In a welfare analysis, we show that banning price discrimination benefits high search cost consumers but makes low search cost consumers worse off.

The empirical part of our analysis uses a unique dataset on retail electricity prices and consumer search intensity at online platforms at the German zip code level for the period 2011–14. The German retail electricity market was liberalized at the end of the previous millennium, when former local monopolies were replaced by local retail competition. Since then, local incumbent suppliers have competed with new entrants. All consumers are by default served by the incumbent at a baseline tariff, which is the most expensive tariff in a local market, but have the freedom to search for cheaper offers. Even though in recent years most consumers use online platforms to search for cheaper rates,² in 2015 76% of all households were still served by the incumbent—with 33% remaining at the expensive baseline tariff, while 43% have switched to a cheaper incumbent tariff—and only 24% have switched to an entrant (BNetzA 2015). Hence, some two decades after liberalization, the incumbent still prices well above costs, strategically price discriminating between different types of consumer groups, thereby having successfully prevented many consumers from switching to entrants.

A key feature of our data is that we can measure the consumer search intensity per zip code and year. In particular, we have data on the actual number of households’ search queries at online price comparison platforms, and given that most of the search for lower prices is via these platforms, we interpret these data as a direct measure of search intensity at the local level. With some notable, recent exceptions (such as De Los Santos, Hortaçsu, and Wildenbeest 2012; Blake, Nosko, and Tadelis 2016; Coey, Larsen, and Platt 2020), other empirical studies on consumer search markets often have to rely on indirect measures of consumer search activity.³

In terms of prices, we observe the incumbent’s baseline tariff and the incumbent’s cheaper online price, as posted at the platform. We also observe the lowest online price, offered by an entrant retailer. Using these data, we empirically show that incumbents increase their baseline rates when consumers search more. Moreover, the incumbent increases the extent of price discrimination and lowers its online tariff significantly when consumers search more at platforms. We also find that entrants reduce their tariffs with more consumer search. We estimate that a 1 standard deviation increase in within-zip-code search intensity explains nearly 50% of the observed price discrimination. Hence, one key takeaway message of our analysis is that, confronted with competitors entering the market, an incumbent can increase profits by price discriminating between consumers with different search costs. As consumer search intensity may also be a function of price (e.g., Lewis 2008; Tappata 2009; Lewis and Marvel 2011; Byrne and De Roos 2017; Cabral and Gilbukh 2020; Heim 2021), and because retailers’ pricing strategies depend on consumers’ search efforts, endogeneity may be a concern in the empirical analysis. We thus employ an instrumental variable to address the potential endogeneity of search intensity. In particular, we take the search intensity for heating gas tariffs as an instrument for electricity search. As the same households or households with similar features search for electricity and heating gas tariffs, these two search intensities are correlated, but search for heating gas tariffs does not directly cause electricity prices.

Many markets where market liberalizations have taken place (including electricity markets in several states of the United States, Canada, and other EU member states) share important features with the German electricity market. In all these markets, new firms have entered, incumbents may engage in price discrimination, and there is an important asymmetry, as consumers know the base price of the incumbent but have to incur a search cost to learn prices set by entrants. Other liberalized sectors such as natural gas, telecommunications, health insurance, railways, postal services, and airlines share similar features. A key dividing line between these examples is whether or not consumers have an ongoing relation with their suppliers. Thus, markets such as electricity, telecommunications, and health insurance markets have the feature that consumers are naturally informed about their current supplier and will automatically continue their contract as long as they do not search for and switch to alternatives. The role of incumbency effects is also of importance to sectors beyond the liberalization context, such as retail banking, where (online) searching consumers may get much better deals than loyal consumers.⁴

Our study contributes to different strands of literature. There is a large and varied theoretical literature on how consumer search affects price dispersion in homogeneous goods markets (see, e.g., Stahl 1989; Janssen and Moraga-González 2004). Several empirical studies focus on price dispersion and search intensity (see, e.g., Sorensen 2000; De Los Santos, Hortaçsu, and Wildenbeest 2012). Tang, Smith, and Montgomery (2010) find that an increase in shopbot use reduces average prices and price dispersion in online book retailing. Lach and Moraga-González (2017) show that competition may be more beneficial for consumers who are better informed. Pennersdorfer et al. (2020) find an inverted-U-shaped relation between price dispersion and the share of informed consumers (as proxied by the share of commuters) in the Austrian gasoline retail market. This literature does not, however, deal with incumbency effects or the possibility of price discrimination.

A growing literature explicitly deals with search in electricity markets, but most of these papers mainly focus on how consumers search without considering the implications for price setting. Giulietti, Waterson, and Wildenbeest (2014) analyze the retail electricity market in the United Kingdom and find that roughly half the households had relatively high search costs. Hortaçsu, Madanizadeh, and Puller (2017) analyze switching in the Texas retail electricity market and find that even though households rarely switch to alternative retailers, they do switch more after experiencing a “bill shock.” Moreover, they also find that households attach a brand advantage to the incumbent. Both papers do not observe the actual search behavior of consumers, however. Dressler and Weiergraeber (2019) use a structural demand model of the Belgian electricity market focusing on switching costs and limited awareness. In contrast, Byrne, Martin, and Nah (2022) use a field experiment to study how heterogeneous search frictions are used by electricity firms in Australia to differentiate between consumers by combining posted prices and sequential bargaining with individual households. Their setup is different since private negotiations do not play a role in Germany.⁵

Another related literature argues that entry may lead to higher incumbency prices and/or profits (see, e.g., Perloff, Suslow, and Seguin 1995; Ishibashi and Matsushima 2009). In all these models, because of either horizontal or vertical product differentiation, after entry the incumbent will focus on a more targeted group of consumers who are less price sensitive. Using a similar logic, Doganoglu (2010) shows that small switching costs may lead to lower prices relative to a situation without switching costs. Even though the mechanism of our theoretical model also relies on the incumbent targeting a specific group of consumers, our focus is different as we take entry as given and analyze the incumbent’s price discrimination strategy and how it depends on search and switching behavior.

There is a small literature dealing with price discrimination and incumbency. For the UK retail electricity market, Davies, Price, and Wilson (2014) present evidence suggesting that firms deliberately differentiated their tariff structures, resulting in market segmentation according to consumers’ usage. For the US airline industry, Goolsbee and Syverson (2008) indicate that incumbents respond to the threat of entry by substantially reducing average fares on the directly threatened routes, but that they do not cut prices on routes to nearby airports in the same market. This bears some relationship to our result that the incumbent price discriminates between searching consumers who may choose an alternative option and nonsearching consumers who do not. Allen, Clark, and Houde (2019) study the Canadian mortgage market, in which firms and consumers individually bargain about contracts, and estimate that search frictions cause an incumbency advantage, which generates significant consumer welfare losses. A difference between their setup and ours is that in Allen, Clark, and Houde (2019) prices are negotiated and each customer gets a different price offer depending on their search costs. In our setup, the incumbent sets two relevant prices, resulting in price discrimination between high and low search cost consumers. This way, our model predicts that high search cost consumers can be worse off the higher the search intensity in a market, while in their setup, the price of a customer with high search cost is not affected by the share of customers with low search cost.

At a theoretical level, the idea that a firm would like to price discriminate against consumers with higher search cost is not new. Salop (1977), for example, studies a monopoly setting and his argument critically depends on the assumption that the monopolist is committed to charging prices according to a price distribution, while consumers can somehow react to changes in the price distribution (assuming they observe the distribution, but not the prices) by adopting a different search strategy. Cabral (2016) analyzes conditions for which switching costs may lead to higher or lower equilibrium prices in markets in which sellers discriminate between locked-in and not locked-in consumers. Cabral and Gilbukh (2020) also model firms engaging in price discrimination between active and passive searchers. Unless they pay a search cost, consumers buy from the high price of a firm. The focus of Cabral and Gilbukh (2020) is, however, very different from ours in that they study symmetric firms facing cost shocks, whereas we focus on how asymmetric pricing is affected by the presence of more searching consumers. Armstrong and Vickers (2019) analyze the welfare effects of price discrimination in the presence of captive consumers who only buy from the incumbent while others choose freely among alternative offers. While Armstrong and Vickers (2019) do not analyze search behavior of consumers, their main result is that the welfare effects of price discrimination depend on the degree of symmetry between firms. With symmetric firms, discrimination against captive customers harms consumers overall because it does not affect profits but widens the variation of profit across consumers (profit varies with consumer surplus and consumers are risk averse). Fabra and Reguant (2020) model price discrimination in a market in which sellers compete for buyers, who differ in their search costs and in size, essentially determining their willingness to search. While sellers do not observe buyers’ search costs, they form beliefs about them based on observed buyer size. This is different from our setup, where the incumbent sets a cheaper tariff to those consumers who search at a platform, whereas all consumers have the same buyer size (3.5 MWh of electricity per year).

The rest of the paper is structured as follows. Section II describes the German retail electricity market in more detail. Section III provides a theoretical model to guide the empirical approach and findings. Section IV describes the empirical identification strategy and section V discusses the data. Section VI presents the econometric results and section VII discusses their robustness. Section VIII concludes.

II.  Institutional Details

In 1999, Germany’s electricity liberalization brought about the end of local monopolies by allowing entry to local retail markets. While electricity generation continued to be in the hands of a few firms, it was believed that increased retail competition and freedom of consumer choice would result in large economic benefits for consumers. Prior to market liberalization, the local incumbent served all customers in its distribution grid area at a regulated tariff. Since liberalization, the incumbents have been legally obliged to supply electricity at a default baseline tariff to all households that do not proactively choose another supplier. Moreover, a household that moves to another zip code is automatically supplied by the local incumbent at its baseline tariff.⁶ However, the incumbents’ baseline tariffs are no longer regulated and households are free to switch to alternative tariffs that are offered by one of the many new entrants or by their local incumbent. Consumers can switch away from the incumbent baseline tariff at any time with 2 weeks’ notice. Consumers who switch generally take a 1 year contract with their new supplier, which is automatically renewed if the consumer does not cancel the contract in time.⁷

Entry in the retail market involves low entry costs and risks. This is also witnessed by the large number of active retailers: there are on average 133 electricity retailers per zip code, with a range of 55 to 192. In contrast to incumbent electricity providers, which are typically vertically integrated (possessing power plants to generate electricity and retailing electricity to end consumers), entrant retailers are typically small, nonintegrated resellers/arbitrageurs, buying electricity at the wholesale market and selling it at a margin to final consumers.

Another important market characteristic is that retailers competing in a zip code have almost identical costs: some cost components, such as grid charges and concession fees, differ over time and across zip codes, but are equal for all retailers in a zip code. Other cost components, such as the surcharge for renewable energy subsidies, only change over time but do not have local variation. Costs for purchasing wholesale electricity are also almost identical across retailers since wholesale electricity prices are determined centrally at the European Energy Exchange (EEX).⁸ Some other costs, such as administrative or advertisement costs, may differ across retailers but account only for a minor part of the (variation in) retail costs. Thus, while costs are similar for all retailers within a local market, they vary substantially across local markets. Many incumbents operate only at a very local level and 46% of the incumbents only have a single zip code in their incumbency area. These small incumbents are mostly municipal utilities. The incumbency areas of incumbents with more than one zip code cover five zip codes at the median and 32 at the mean. Hence, as the costs differ between zip codes, incumbents serving more than one zip code area face different costs within their incumbency area, and on average they set 3.5 different prices in their incumbency areas. Incumbents operating in more than one price zone set prices that differ, on average, by 10.4 euros per year for a typical household with an annual electricity consumption of 3.5 MWh.⁹ Thus, retailers set local prices that vary in most cases at the zip code level.

In recent years, most households, which consider changing their supplier, visit an online price comparison platform. Despite this fairly recent trend of searching via online platforms, in 2011 80% of the switchers had already searched online for alternative providers (A. T. Kearney 2012). The switching rate has been growing in recent years (see fig. 1), as online price comparison platforms have significantly reduced the costs of searching for cheaper providers (something that is also acknowledged in other markets; e.g., Bar-Isaac, Caruana, and Cuat 2012). A comparison portal requires a consumer to enter all relevant details (zip code, expected yearly electricity consumption, whether the contract is for private or commercial use). Then, there are several options to choose from, such as whether to only consider “green” electricity, whether prices are guaranteed throughout the year, and whether the listed tariffs should include one-off bonuses. The platform then lists the “personalized” prices of all providers that are active in the indicated zip code, ranked from lowest to highest. For each tariff, the platform also provides information on how much consumers can save over the year compared to the incumbent’s baseline price. Thus, the search process costs some time and effort, but for all consumers who are familiar with online shopping, the search costs are relatively small compared to the potential savings of switching from the incumbent’s baseline tariff to the overall cheapest tariff, which are, on average, almost 200 euros per year for a standard two-person household with 3,500 kWh consumption (as shown in the sample statistics presented in Table 1 in the data section).

Not only have search costs declined over time; switching costs have also been significantly reduced, because switching is now an automated process and conducted entirely by the new provider, which automatically arranges all switching activities for new customers, such as unsubscribing from the old supplier and registration, at no additional cost.¹⁰

There is a tiered pricing system in Germany (two-part tariffs with a fixed and a variable component). The consumption profiles depend on how much consumers heat, whether they use air conditioning, how much time they watch TV, and so forth. For their tariff choice, household consumers thus typically consider their average annual electricity consumption (e.g., as stated in their last year’s electricity invoice).

Finally, as there are no retailer-specific differences regarding the quality of supply, retail electricity can be considered a fairly homogeneous product, which helps us to rule out product differentiation as a possible explanation for price dispersion. If an entrant fails to deliver, the incumbent provider has the legal obligation to deliver electricity at the baseline tariff without interruption. Not all consumers may be aware of this safety net, however. Hence, even though theoretically it should not matter for the end consumer which retailer delivers the electricity, it still may matter in practice.

As prices other than the incumbent baseline tariff can only be observed by consumers who proactively search, an incumbent is able to have an online tariff that is lower than the baseline tariff. The incumbent’s online tariff is larger than the cheapest overall tariff set by an entrant. Figure 2 shows that there are considerable price differences between the incumbent’s baseline tariff $P_H^I$ (price incumbent high), the incumbent’s lower online tariff $P_L^I$ (price incumbent low), and the overall cheapest entrant tariff P^E (price entrant). As consumers who switch away from the incumbent most likely choose the cheapest tariff available, we focus on the cheapest entrant price.¹¹ As a result, we observe three forms of price dispersion: (i) overall price dispersion ($P_H^I-P^E$), which is the difference between the incumbent’s baseline tariff and the overall cheapest tariff; (ii) price discrimination by the incumbent ($P_H^I-P_L^I$), measured by the difference between the incumbent’s baseline tariff and the incumbent’s cheaper online tariff; and (iii) online price dispersion measured by the difference between the incumbent’s cheaper online tariff and the cheapest entrant tariff ($P_L^I-P^E$).¹²

Figure 2 also depicts the (approximated) costs of retailers (see sec. V for more details). We see that costs and prices have increased over time (mostly due to increased taxes and levies to finance the integration of renewables). Evidently, even nearly two decades after the retail liberalization in the industry, the incumbent baseline tariff remains well above costs. Moreover, the figure emphasizes that incumbents price discriminate with the cheaper incumbent online price, which is still well above costs. By contrast, the cheapest tariffs set by entrants are very close to costs.

III.  A Simple Search Theoretical Model

A.  Model

In this section, we consider a simple model that describes the main features of the market and show how the incentives of electricity providers and consumers interact to produce the patterns of price discrimination and price dispersion we find across different local markets. We also perform a welfare analysis. The model features apply to any liberalized market in which an incumbent firm competes with entrants for a homogeneous product and the incumbent is able to price discriminate between searching and loyal consumers.

The model describes how we think of the market interaction between incumbent and entrants and closely follows the institutional details described above. All consumers observe the regular (baseline) price $P_H^I$ of the incumbent at no additional costs and can consult an online price comparison website at a search cost s that differs across consumers. The search cost distribution function in a zip code area is denoted by F(s; z), where we use z to represent exogenous parameters that determine the shape of the search cost distribution in a zip code area. By varying z, we determine how pricing patterns across different local markets depend on exogenous factors affecting the search cost distribution. In the empirical part of the paper, z is an instrument that is exogenous to search and that does not directly affect pricing strategies.¹³ The search cost reflects the time it takes consumers to get familiar with the tariff comparison platform and to enter the required personal information on the price comparison website. At the website, consumers will see potentially many prices, but (in line with the data we have) we are only interested in two of them: the price P^E of the overall cheapest firm (usually an entrant) and the cheapest (online) price $P_L^I$ of the incumbent.

Apart from their search cost, consumers also pay a transaction cost if they want to switch away from the incumbent. These costs also differ between individuals and refer to all the objective and psychological costs consumers face if they switch. As explained in section II, the objective switching costs are small, but consumers may perceive the incumbent as more trustworthy. To keep the analysis simple, we assume that these transaction costs are proportional to the search cost; that is, the transaction cost of a consumer with search cost s is denoted by θs.¹⁴ Thus, once a consumer with search cost s is online and observes both prices P^E and $P_L^I$, the consumer will continue to buy from the incumbent if $P_L^I-\theta s<P^E$.

We make two further simplifying assumptions. First, in real markets, the following dynamic aspect may play a role: once some consumers have switched to entrants, they gain some incumbency effect as these consumers will have to search at a later moment if they want to switch away from their provider. Thus, over time entrants and incumbents may become more symmetric to each other. In the theoretical model, we have abstracted from these considerations as individual entrants in local German electricity markets typically have a very small market share.¹⁵ Second, our main model looks at the behavior of one entrant that does not compete with other entrants. We use this as a shorthand approximation for the small incumbency effect entrants may have. It can be shown that qualitatively similar effects continue to hold if entrants engage à la homogeneous Bertrand competition with each other (see n. 20).

The sequence of actions is as follows. In the first stage, the incumbent and entrant choose prices $P_H^I$, $P_L^I$, and P^E simultaneously.¹⁶ At the beginning of the second stage, consumers only observe $P_H^I$ and decide whether or not to search based on their expectation regarding online prices. If they do not search, they buy from the incumbent at $P_H^I$. If they do search, they observe the online prices and buy where it is best for them, taking the transaction cost into account. We use perfect Bayesian equilibrium with passive beliefs as our solution concept. Thus, we look for an equilibrium in which consumers have correct beliefs about the online prices and in which, if consumers observe an unexpected price $P_H^I$ (different from the equilibrium level), they will continue to believe that $P_L^I$ and P^E are at their equilibrium levels.

A natural candidate for an equilibrium is where low search cost consumers with $s<{\widehat{s}}_2$ search online and all other consumers stay with the baseline price of the incumbent. Moreover, of the consumers who search online, the ones with a transaction cost $\theta s<\theta {\widehat{s}}_1$, with ${\widehat{s}}_1<{\widehat{s}}_2$, buy from the entrant, while other online consumers, namely those with ${\widehat{s}}_1<s<{\widehat{s}}_2$, buy from the incumbent at its online price. In such an equilibrium, the cutoff values for search costs are ${\widehat{s}}_1=\left(P_L^I-P^E\right)/\theta$ and ${\widehat{s}}_2=\left(P_H^I-P_L^{I^e}\right)$.¹⁷

Assuming, without loss of generality, that the firms have no supply cost, the equilibrium prices we derive can be interpreted as firms’ margins. Thus, the respective profits of the entrant and incumbent are as follows:

$${\pi }_E=F({\widehat{s}}_1;z)P^E=F\left(\frac{P_L^I-P^E}{\theta };z\right)P^E$$

and

$$\begin{array}{rl}{\pi }_I& =\left[F({\widehat{s}}_2;z)-F({\widehat{s}}_1;z)\right]P_L^I+[1-F({\widehat{s}}_2;z)]P_H^I\\ & =\left[F(P_H^I-P_L^{I^e};z)-F\left(\frac{P_L^I-P^E}{\theta };z\right)\right]P_L^I+[1-F(P_H^I-P_L^{I^e};z)]P_H^I.\end{array}$$

This yields the following first-order conditions (FOCs; evaluated at the equilibrium at which $P_L^{I^e}=P_L^I$) for the entrant and the incumbent, respectively:

$$\begin{array}{}\text{(1)}& F\left(\frac{P_L^I-P^E}{\theta };z\right)-f\left(\frac{P_L^I-P^E}{\theta };z\right)\frac{P^E}{\theta }=0,\text{(2)}& F(P_H^I-P_L^I;z)-F\left(\frac{P_L^I-P^E}{\theta };z\right)-f\left(\frac{P_L^I-P^E}{\theta };z\right)\frac{P_L^I}{\theta }=0,\end{array}$$

and

$$\begin{array}{}\text{(3)}& -f(P_H^I-P_L^I;z)(P_H^I-P_L^I)+[1-F(P_H^I-P_L^I;z)]=0,\end{array}$$

where f(⋅) is the density function that is associated with F(⋅). Note that the fraction of actively searching consumers is given by $F(P_H^I-P_L^I;z)$.

For a given z, these three FOCs determine the equilibrium values of $P_H^{I^{*}}$, $P_L^{I^{*}}$, and $P^{E^{*}}$, and the corresponding levels of price discrimination and price dispersion. To explain our observations, we have to see how these equilibrium price levels change with variations in z. It is clear that a rich set of patterns is possible, and in the proposition below we focus on the conditions that guarantee that the model generates the patterns we find empirically, namely, that price discrimination increases and online price dispersion decreases with the fraction of people in a zip code searching online.

Proposition 1. 

The effects of exogenous changes in the search cost distribution, reflected in changes in z, are as follows. More consumers search and price discrimination increases if, and only if, the inverse hazard rate evaluated at the equilibrium values $[1-F(P_H^{I^{*}}-P_L^{I^{*}};z)]/f(P_H^{I^{*}}-P_L^{I^{*}};z)$ is increasing in z. The cheapest online price P^E and online price dispersion are positively related to $P_L^I$ if the density functions are nonincreasing; that is, $\partial f\left((P_L^I-P^E)/\theta ;z\right)/\partial (P_L^I-P^E)\le 0$. Finally, online price dispersion and price discrimination are linked by $1-f(P_H^I-P_L^I;z)(P_H^I-P_L^I)=f\left((P_L^I-P^E)/\theta ;z\right)\left[(P_L^I+P^E)/\theta \right]$.

The economic intuition behind the result on price discrimination is as follows: for a given value of $P_L^I$ the incumbent faces a trade-off in its decision whether or not to increase $P_H^I$. Raising $P_H^I$ increases the profits over all consumers $1-F(P_H^{I^{*}}-P_L^{I^{*}};z)$ who stay at the baseline tariff, but a fraction proportional to the density $f(P_H^{I^{*}}-P_L^{I^{*}};z)$ will decide to search. At the margin, those that decide to search will eventually buy at the incumbent’s online price $P_L^{I^{*}}$ as the marginal consumer has a higher search and transaction cost. The incumbent will lose $P_H^{I^{*}}-P_L^{I^{*}}$ per (marginal) consumer who searches. If, evaluated at the equilibrium values, the inverse hazard rate is increasing in z,¹⁸ relatively more consumers will stay at the baseline tariff if z increases, making price discrimination more profitable. Also, in equilibrium, the fraction $F(P_H^{I^{*}}-p_L^{I^{*}})$ of consumers search, which is directly related to the price discrimination strategy of the incumbent. To understand online price dispersion, if $P_L^I$ increases, then there is a larger potential demand for the entrant and, under “normal” demand conditions, it should increase its price, but not to the full extent (thereby also increasing sales).

Combining the effects, consider the special case in which the relevant densities remain constant (or are not much affected) and the change in the search cost distribution is such that the incumbent price discriminates more and more consumers search; then the last equality in proposition 1 implies that the sum of online prices must decrease. As the second result implies that online prices and online price dispersion change in the same direction, it must be that they decrease.

The effects outlined in the proposition and the above intuitive explanation rely on the shape of the search cost distribution as the outcome of price discrimination depends on how many consumers continue to stay with the incumbent’s baseline price and how many will search and switch to the entrants’ and incumbent’s online prices. To verify in the data whether this condition holds one needs to know the search cost distribution or access to quantity data on how many consumers buy at which tariff, information that we unfortunately do not have.

The proposition leaves the effect on the incumbent’s baseline price undetermined. In appendix C, we analyze the case of a piecewise linear search cost distribution¹⁹ to show that the baseline price may well increase:

$$F(s)=\{\begin{array}{cl}zs& \text{\hspace{1em}for }s<{\tilde{s}}_1,\\ \alpha +\beta s& \text{\hspace{1em}for }{\tilde{s}}_1\le s<{\tilde{s}}_2,\\ s& \text{\hspace{1em}for }1\ge s\ge {\tilde{s}}_2,\end{array}$$

where, to have a proper piecewise linear distribution function, $\alpha =[(z-1){\tilde{s}}_1{\tilde{s}}_2]/({\tilde{s}}_2-{\tilde{s}}_1)$, $\beta =({\tilde{s}}_2-z{\tilde{s}}_1)/({\tilde{s}}_2-{\tilde{s}}_1)$, ${\tilde{s}}_2>{\tilde{s}}_1$, and $z>0$. If $z=1$, we have the uniform distribution.

Figure 3 depicts how the different prices change as a function of z when ${\tilde{s}}_2=3/5$, ${\tilde{s}}_1=1/5$, and $\theta =2/5$. Detailed derivations are given in the online appendix. As $\partial F(P_H^{I^{*}}-P_L^{I^{*}})/\partial z$ is a constant positive number, this figure can also be interpreted as how prices are linked to the fraction of searchers. One can see that the incumbent’s baseline price is increasing in the fraction of searchers, whereas the other two prices are decreasing, resulting in more price discrimination and overall price dispersion, while online price dispersion is decreasing. This is also what we find in our empirical analysis (see sec. VI).²⁰

IV.  Identifying the Effect of Consumer Search on Pricing Strategies

To examine the causal effect of consumer search intensity on pricing strategies, we first explain our identification strategy and then describe our data and results.

V.  Data

We use panel data at the German zip code level for the period 2011–14.²⁶ As consumers typically have annual contracts, we aggregate all data to the annual level. Table 1 provides summary statistics of the variables in our regressions. Table B2 additionally reports the between and within standard deviations of our key variables, indicating that we have sufficient temporal and spatial variation. Figures F2–F8 provide heat maps of our main variables, search intensity and tariffs, visualizing their between and within variation.

Tariffs.—ene’t, a German software and data provider for the electricity industry, provided monthly data on retail electricity tariffs and cost components (except for $P_L^I$, which is already structured annually). In the estimations, we use gross prices (including 19% VAT), which are the relevant prices for end consumers that are also displayed on the online platforms. We focus on a typical household with an annual consumption level of 3,500 kWh. This is the default consumption level suggested by all major price comparison platforms.²⁷ The summary statistics in Table 1 show that, on average, a household pays around 1,007 euros per year for the incumbent’s baseline tariff. The incumbent’s online tariff is around 8% lower at 931 euros, while the overall cheapest entrant tariff is around 808 euros, which is 20% cheaper than the incumbent default tariff. Figure 4 shows the local variation of how much a household can save by switching from the incumbent’s baseline tariff to the cheapest entrant across Germany in 2012.

Consumer search intensity.—ene’t also provided the data on individual consumer search queries for electricity retail tariffs at several online price comparison sites, which enables us to construct a direct measure of consumer search intensity for each zip code and year. The database covers detailed information on all search queries conducted at several well-known online price comparison platforms including Toptarif.de, Stromtipp.de, Energie-verbraucherportal.de, and mut-zum-wechseln.de, of which Toptarif.de is by far the largest platform.²⁸ For each query, we observe a timestamp, the entered zip code for which the offered electricity tariffs are requested, the (expected) yearly consumption entered into the interface, whether the search is performed by a household or an industrial customer, and consumer preferences (e.g., only “green” certified tariffs). In addition, we are also able to track the search history: each platform user obtains a unique search session ID (created by ene’t), indicating the order of the queries from the same user.²⁹ Figure 5 provides a screenshot of the interface of a typical tariff comparison platform. For each tariff the platform shows how much a consumer can save compared to the incumbent’s baseline tariff.

In sum, we have information on 35,855,071 search queries from 17,302,530 search sessions of which 96.7% (i.e., 16,778,214 sessions) are conducted by households and the remaining 3.3% (i.e., 524,316 sessions) by industrial customers. As many searchers conduct several search queries within a search session (e.g., comparing prices for different consumption levels), we focus on the number of search sessions per year and zip code (rather than on the absolute number of search queries). Since our focus is on household consumers, we disregard search by industrial consumers. Furthermore, we exclude 551,256 search sessions that exclusively consider eco-label (i.e., “green”) certified tariffs.³⁰ Those searches are most likely not predominantly price driven and, on average, €152 more expensive than the cheapest tariff.

We construct our measure of search intensity as the number of search sessions within a zip code per year divided by the number of households:³¹ ${\mu }_{it}=({\mathrm{Search Sessions}}_{it})/({\mathrm{Households}}_{it})$. At the mean, 9.1% of households within a zip code search for retail tariffs at one of our sample comparison platforms, whereas there is substantial variation ranging from 0% to 34.7%.³² Several factors may cause variation in local search costs. Clearly, an important driver of search intensity is the distribution of search costs, which depend for instance on population characteristics such as income or age (Nishida and Remer 2018). Another factor is the local development of the broadband internet infrastructure that makes internet usage and online shopping more convenient. Similarly, local advertisements for price comparison platforms, word-of-mouth communication, or discussions about electricity prices and costs in the media may also incentivize consumer search. Of course, retail tariffs also affect search intensity.

Instrument.—Analogously to the construction of our measure for search intensity for electricity tariffs, we construct our measure of search intensity for heating gas tariffs using data on individual search queries for gas tariffs from price comparison websites. Here, we have information on 8,522,591 search queries in total.

Control variables.—We compute a variable reflecting retailers’ net costs (excluding VAT) in order to control for spatial and time-variant cost differences. Detailed data on cost components are primarily obtained from ene’t and include, for example, grid charges, concession fees, renewable energy surcharges (“EEG Umlage”), CHP (combined heat and power) surcharges (“KWK Umlage”), and electricity taxes. Grid charges are paid by the electricity provider to the respective system operator and, thus, vary across grid areas (i.e., clusters of zip codes) and time as they are adjusted annually. The concession fee has to be paid by the system operator to the respective municipality for the right to install and operate electricity cables on public roads. Hence, the concession fees vary at the municipality level and also over time. The remaining cost components only vary over time but not spatially. Moreover, we also add the 1 year ahead future prices of electricity at the EEX spot market to our cost variable to proxy for the costs of wholesale electricity, as this 1 year ahead price presents the standard purchasing strategy for retailers.³³

Other control variables refer to structural household characteristics, which we obtained from Acxiom, a commercial data service provider. These variables are the available income per household, the average household size, and the number of households per zip code–year pair.

VI.  Results

Before we present the regression results, we provide some descriptions showing the relationship between consumer search and prices. Every year the German Federal Network Agency (Bundesnetzagentur) announces the adjustment of the renewable energy surcharge (“EEG Umlage”) in mid-October. The EEG Umlage constitutes a major component of a consumer’s electricity bill (e.g., 20%–22% of the electricity bill in 2014) and electricity retailers have to inform their customers shortly after that—until November 20—about price changes (BNetzA 2015, 207). The left panel in figure 6 shows the aggregate weekly search sessions on the online price comparison sites we observe. The vertical solid lines indicates the week of November 20. It is evident that consumers search more in November immediately after they get informed about price changes. To cross-validate the representativeness of our date we contrast these data with Google Trends data for the word “Stromwechsel” (change of electricity supplier). The Google Trends data are shown in the right panel of figure 6 and exhibit very similar search patterns. The significant bumps in consumer search intensity around November 20 are clearly an indication of the endogenous relation between price and search and thus emphasize the importance of applying an IV strategy for causal identification.

In Table 2, we present the results of our IV estimations for the three retail prices of interest, $P_H^I$, $P_L^I$, and P^E. As we use a log-log specification the coefficients can be interpreted as elasticities.³⁴ The instrument is sufficiently strongly correlated with the endogenous variable, as shown by the high values of the first-stage effective F-test, suggested by Olea and Pflueger (2013). Results from the first-stage estimation are reported in Table B3. Also, the Durbin-Wu-Hausman test for endogeneity (Davidson and MacKinnon 1993) suggests that the consumer search intensity μ should indeed be treated as endogenous, because the null hypothesis of consumer search being an exogenous regressor is clearly rejected.

The OLS estimates are provided in Tables B4 and B5. Even though the sign and the significance are similar, the magnitudes of the OLS estimates are much lower, suggesting that neglecting endogeneity leads to a substantial underestimation of the impact of consumer search on prices.

Coming to the results, column 1 of Table 2 provides evidence that the incumbent reacts to a higher search intensity by increasing its baseline tariff. For a change in consumer search intensity by 10%, the incumbent raises its tariff by approximately 0.4%. Column 2 shows that the incumbent reacts to more search activity in its zip code by reducing its online tariff considerably. For a 10% increase in search activity, the incumbent decreases its cheapest tariff by 1.7%. Moreover, column 3 reveals that the overall cheapest tariff in the market provided by an entrant supplier also decreases with more consumer search, but its effect is less pronounced than for the incumbents’ online tariffs. For every 10% increase in search intensity in a zip code the overall cheapest tariff in the market decreases by approximately 0.4%. Thus, the incumbent’s online tariff reacts more strongly to consumer search than the overall cheapest tariff.

The empirical effects can be explained along the lines of proposition 1. With more low search cost consumers in a region, there is more competition online yielding lower online prices. To prevent too many consumers from switching to the entrant, the incumbent has to decrease its online price more aggressively than entrants do: the incumbent would lose a larger markup when losing a customer, as the incumbent’s online price is still higher than the overall cheapest price offered by an entrant. At the same time, if there is still a considerable fraction of consumers with high enough search costs, the incumbent has an incentive to increase the margin on its baseline tariff as it will not lose too many consumers by doing so. Hence, the incumbency advantage can be exploited by price discriminating between consumers with higher search cost and consumers who search online but have a higher transaction cost.

A back-of-the-envelope calculation shows the reasonableness and economic importance of our estimates. Our estimates from Table 2 imply that the incumbent increases its base tariff by 7.5 euros if search intensity in a zip code increases by 1 within-zip-code standard deviation (which is 5.1 percentage points), taking as starting points the mean values of prices and search intensity (i.e., 1,007 euros and 9.6%, respectively). Moreover, the incumbent decreases its online tariff due to the increased search activity in the zip code by 30.5 euros (mean value is 931 euros). The cheapest entrant decreases its tariff by a further 5.9 euros (mean value is 808 euros). Thus, we would expect from our estimates that price discrimination increases by 38 euros on average (which is 49.7% calculated from the mean value of price discrimination of 76.5 euros) due to a 1 standard deviation increase in search intensity within a zip code. Thus, increased search activity appears to be a substantial part of the explanation of why incumbents price discriminate in liberalized markets.

With regard to the control variables, it may be noteworthy that our estimate of the cost pass-through to the end-user retail tariffs is much higher in the competitive segments of the electricity retail market. For the incumbents’ baseline tariffs, we estimate a pass-through of only around 23%, whereas 38% of cost increases are passed on to consumers for the incumbents’ online tariffs and 52% for the cheapest entrants’ tariffs. These pass-through patterns are in line with Duso and Szücs (2017), who investigate pass-through in the German electricity retail markets and also find that incumbents pass-through costs to a lesser extent.

Table 3 presents estimates of the impact of consumer search on the three price dispersion measures. Column 1 focuses on overall price dispersion, measured as the incumbent’s baseline tariff ($P_H^I$) minus the overall cheapest tariff (P^E). Evidently, price dispersion goes up if more consumers search, since the incumbent slightly increases its baseline tariff and at the same time the overall cheapest price declines with search. For every 10% increase in search intensity, the extent of price dispersion goes up by 3.7%, suggesting that consumers’ gain from searching increases with the share of searching consumers.

Incumbents react to increased price pressure from consumer search via price discrimination, as they offer a cheaper tariff for searching consumers, which is still above the overall cheapest tariff in the market, and a high incumbent baseline tariff for consumers who do not search. Price discrimination becomes more pronounced with increasing search intensity. An increase in the share of searching consumers by 10% widens the gap between the incumbent’s baseline tariff and its cheaper tariff by 24.8%. The extent of price discrimination unambiguously increases if a larger share of consumers search, predominantly because the incumbent decreases its cheapest tariff significantly as a reaction to consumer search to aggressively prevent existing customers from switching to competitors. This can be explained in line with proposition 1 of our theoretical model: more searching consumers imply more price discrimination if there are relatively sufficiently many consumers left with relatively high search cost who “always” buy at the baseline price of the incumbent.

We also see that online price dispersion, measured as the difference between the incumbent’s cheapest tariff and the overall cheapest tariff in the market, narrows considerably with search intensity. The more consumers search in a market, the more the incumbent is forced to set the online price closer to the overall cheapest price. For a 10% increase in search intensity, the online price dispersion narrows by 17%.

Overall, we find that the high search cost consumers who stay with the incumbent’s baseline tariff get “milked” when there are more searching consumers in a local market. In contrast, those who are willing to search either get a lower incumbent tariff or switch to the entrant. The incumbent reacts to more consumer search with price discrimination by slightly increasing its baseline tariff while at the same time significantly reducing its cheaper online tariff. Entrants react to more search with somewhat lower prices. Intensified consumer search thus increases overall price dispersion and price discrimination, and it leads to fiercer price competition (i.e., an alignment of incumbent and entrant prices) in the competitive online segment.

VII.  Robustness

Our results are robust to various alternative specifications, such as using alternative instruments, level-level estimation, allowing for a nonlinear relationship between search and tariffs, and adding or removing control variables. We present and discuss these specifications below and report the results in appendix B and appendix E.

Hausman-type instruments.—As an alternative IV we apply “Hausman-type instruments” in the spirit of Hausman (1996; see also Hausman, Leonard, and Zona 1994; Nevo 2000; Berry and Haile 2016). In doing this, we take the average of our instrument—the search intensity for gas tariffs—in the surrounding zip codes as the instrument for electricity search intensity in the focal zip code. Surrounding zip codes are identified through the nature of the German zip code system: zip codes in Germany have five digits and are ordered geographically in that zip code 12345 is next to zip code 12346. Thus, we use the average search intensity for heating gas tariffs in the other zip codes with the same first four digits in our Hausman-type IV. As a second condition we only use information from those surrounding zip codes if their prices differ from that in the focal zip code. Thus, if a zip code is only surrounded by zip codes within the same price zone they are dropped from the estimation sample. The idea behind these Hausman-type instruments is that variation in search costs in surrounding zip codes is correlated with search costs in the focal zip code (introducing correlation of heating gas searches across several neighboring zip codes and electricity search in the focal zip code) while the variation in gas searches in surrounding zip codes is not directly related to electricity prices in the focal zip code. The correlation between our original instrument and the Hausman-type instrument is high with a correlation coefficient of .51. This high correlation is also reflected in the high first-stage F-test of the excluded instrument. We find that the results stay robust to these alternative instruments, as shown in Tables B6 and B7.

Alternative clustering of standard errors.—Many incumbents operate only locally and 46% of the incumbents only have a single zip code in their incumbency area. These small incumbents are mostly municipal utilities. However, larger incumbents often have several zip codes in their incumbency area and charge locally differing baseline tariffs. The different price zones of the larger incumbents are not necessarily at the zip code level as we discussed in section II. Hence, as a robustness check, we cluster standard errors at the price zone level instead of at the zip code level. Tables G1 and G2 show that the results are robust.

Control variables.—We also estimate models in which we either drop all covariates or include many more. The control variables we include (in logarithms) are the share of unemployed, the degree of urbanization, the share of households with a household head younger than 40, and between 40 and 60, the share of self-employed, shares of households that moved in or out of the zip code, and shares of households with low or medium social status (based on an index taking into account home ownership, number of cars, and education). Tables G3–G6 show that the results are robust.

Level-level instead of log-log.—Instead of a log-log relationship, our results are also robust level-level specifications, as shown in Tables G7 and G8.

Nonlinear relationship.—We also relax the constant-elasticity assumption and allow for a nonlinear relationship between search and prices, by adding a μ² in equation (4) and instrument for μ² with the square of the search intensity for gas tariffs. The results remain robust and using the method by Lind and Mehlum (2010) we find that there is no U-shaped (or inverse-U-shaped) relationship within the range of the data. The results are reported in Tables G9 and G10.

VIII.  Conclusion

In markets in which consumers have an ongoing relation with their provider, they know the price they pay. To get informed about alternative price offers (by other firms, or other tariffs of the same firm), consumers have to pay a search cost. Firms can effectively use this asymmetry to price discriminate between consumers with different search costs. This is especially true for incumbent firms with a large customer base.

Our empirical analysis of local German retail electricity markets shows that search is an important factor in explaining pricing patterns. In particular, differences in the fraction of searching consumers across local markets explain a large part of the observed heterogeneity in pricing behavior: when consumers search more, the incumbent price discriminates more (with higher baseline and lower online tariffs) and the entrant charges lower prices. This strategy implies that few consumers actually switch, with the incumbent appropriating an important share of market revenue.

Our theoretical model shows that the incumbent’s incentive to increase the baseline tariff arises if a lower price would not keep many consumers from searching and catering to high search cost consumers allows the incumbent to siphon off larger rents. Once a consumer has shown a willingness to search (e.g., by conducting a price comparison on an online platform), the incumbent has a strong incentive to prevent consumers from switching to an entrant by setting low online prices. In this way, the incumbent can simultaneously appropriate surplus from high search cost consumers and prevent searching consumers from switching to an entrant.

From a policy perspective, one may wonder whether this type of price discrimination should be banned. It is clear, however, that such a ban has different implications for different types of consumers. Low search cost consumers will be worse off as price discrimination is associated with very competitive behavior in the online segment of the market. High search cost consumers typically would benefit from banning price discrimination as it would allow them to benefit from the fact that the incumbent will charge a lower overall price than the price it charges them when it can target its prices. Whether or not consumers benefit on average depends on the search cost distribution.

Future research should reveal whether similar pricing patterns are found in other markets with similar characteristics. Our theoretical model suggests our results should be relevant in any market in which firms can price discriminate between consumers with different search costs. After having acquired a customer base themselves, entrants may also follow a similar strategy of price discrimination and increase their prices for their existing clients, while simultaneously setting a more competitive price to attract new customers. German electricity markets are special in that entrants are very small: it is likely that quantitatively there would be almost no effect if they engaged in price discrimination. This may clearly be different in other (e.g., telecommunication) markets in which entrants have been able to gain market share. Depending on the available data, such research could also take a more structural approach. We have shown that some of our results depend on the shape of the search cost distribution and progress may partially depend on whether data are available to estimate the search cost distribution, for example by using market share data of the different firms and tariffs.