The Conflict over Long-tail Search

The short version
Search advertisers receive conflicting advice about which queries they should target. Practitioners often advise focusing on the long-tail, those specific queries assumed to identify consumers further into the buying cycle. Alternatively, academic research advises either targeting queries with historically high click-through rates; or, the most-used queries. However, if you want to maximize click-through query usage is the wrong thing to focus on; what really matters is having your ad recognized as the most relevant part of the search results. The likelihood of such recognition is maximized when you target queries that are specific enough to match few other products besides your own, thus minimizing competitive interference. However, consumers do seem to reward seeing lots of competitors when they are comparing their options.

The whole story
Marketing practitioners and the academic literature contradict each other in their advice related to the choice of keywords and the consumer queries search advertisers should target. Academic literature related to this subject is nascent, yet confidently recommends advertisers select keywords that have prompted high click-through rates in the past; (Rutz et al. 2012) and, to track and use the currently most-used keywords. (Skiera et al. 2010)  While these two recommendations seem distinct, they both draw advertisers to target the queries most-used by consumers creating a quintessential “red ocean” (vis a vis Kim and Mauborgne 2004) of competition for the attention of the same searchers.

On the other hand, search engine marketers (e.g., Fishkin 2009, Hill 2012) widely advise that advertisers not exclusively compete for the opportunity to show ads in response to the most-used queries, but rather to also target the long-tail queries that are the most specific in the consumer intent they communicate. As Figure 1 depicts, these queries are individually low in their use among searchers, yet are collectively the largest category of search query. These practitioners often explain that there are three categories of search query, categories based on their frequency of use; and, that these categories are related to the three stages of the buying cycle beginning with (1) information gathering. The most-used and most general of search queries (sometimes called the fat-head) are believed to be used by those just beginning to contemplate a purchase. For example, a person starting to plan replacing tires on their car might query “tires” (point A in Figure 1) or the lesser-used English variant “tyres” at point B. As this person starts to consider the range of alternatives open to them they enter the next stage of (2) shopping. These queries are less often used and more specific; they are believed to be used by those who are assembling a consideration set and evaluating options. For example, after evaluating the results of a “tires” search the consumer will make many follow-up searches, perhaps being more specific about the type of car using “Honda tires” (point C) or “Honda Fit tires” (point D) until they finally settle on a specific product thus entering the final stage of (3) purchase. These queries are the least-used and the most specific, the long-tail of the query spectrum. They are believed to be the most desirable queries to target because the searcher is assumed to have decided what they want and is about to make a purchase. For example, a consumer who has decided to buy a specific size of tire might search “185/55R16” (point E) to find the lowest price right before they make a purchase. Clearly the practitioner perspective has merit as evidenced by the general increase in cost-per-click (CPC), the price that advertisers pay the search engine if their ad gets clicked, as queries get more specific. Conventional economic thinking might lead us to predict that the most frequent queries, those most targeted by advertisers, would have the highest CPC. However, it seems the opposite is true; if CPC reflects the economic value of queries then those in the long-tail are apparently more valuable, presumably because a sales conversion is more likely to occur. It would be wrong though to say that competition has no apparent effect on CPC. In Figure 2 the queries to the right of the reference line are so specific that they are only targeted by one advertiser. In Figure 1 the CPC among these queries was observed to decrease, likely because there was little price-raising competition to show ads in response to them.


Figure 1. Long-tail search and the buying cycle. Search queries are a long-tail phenomenon; some are widely used but most are less so. Some example queries with usage volume and cost-per-click (CPC) at the time of this writing: (A) “tires”, 9.1M, $1.80; (B) “tyres”, 4.1M, $1.90, (C) “Honda tires”, 49.5K, $2.73; (D) “Honda Fit tires”, 2.4K, $3.59; and, (E) “185/55R16”, 480, $2.02. As queries get more specific the usage volume tends to decrease while CPC increases, implying the rare queries are more valuable because they signal purchase propensity. The dashed vertical reference line also appears in Figure 2 to aid the reader in connecting them.


Figure 2. Competitive intensity and query usage. The black dashed line is a locally weighted regression of the plotted points. As queries get more specific, usage among consumers tends to decrease, as does the number of advertisers targeting the query. At the reference line search queries become so specific they only attract one advertiser.

Why does this difference in understanding between practitioners and academics exist? Clearly there is data and logical argument that supports both sides. The problem is that the academic side is focused on a metric rather than the dynamics of the context. Query usage is the wrong thing to focus on, what really matters is having your ad recognized as the most relevant part of the search results. Figure 3A shows the likelihood of such recognition to be maximized when competitive interference is minimized; and that likely occurs when you target queries that match few other products besides your own, queries most likely to be in the long-tail. Note though that a different dynamic appears to be present among fat-head queries. Figure 3B clearly shows consumers reward the presence of more competition among advertisers in the fat-head. Only practitioners have offered a cogent explanation for this: a fat-head query indicates a consumer is comparing alternatives and thus wants to see lots of options. So how should the advertiser respond? Target both types of query but do so with content that helps the consumer perform the task at hand. Ads that target the fat-tail should be all about showing how your value offering compares to your competition. Have the ad lead to a landing page that also does that job. Long-tail queries should be all about making it easy for the consumer to convert, same with the landing page. Good luck and happy marketing.


Figure 3. Advertiser competition and the long-tail. Part A shows that the highly specific queries in the long tail result in more clicks when there are fewer competitors targeting them. We can surmise that these ads are for products more authentically related to the query and are thus more useful to the consumer. Part B highlights that consumers reward seeing more competitors in the fat-head.



Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s