Amazon promotes its products and services through many advertising channels, and of course we want to know how much bang we get for our buck. We are thus trying to measure the effectiveness of our ads through a combination of data modeling and most importing experimentation.
In online survivors, where the majority of ads live today, researchers can randomize ads and make use of tracking technologies as cookies to measure behavior. This approach provides high quality data and statistically significant results. In contrast, more conventional ads found outdoors, in shopping malls and airports and on transportation, saved for fores-of-home (OOH) ads, date provides the same opportunity for personalized behavioral measurements and therefore the garden borrowed borrowed well to experiment.
To close this gap, our team at Amazon has developed and successfully implated an experience design that focuses on metro and switch lines. Based on knowledge from Amazon marketing teams and marketing agencies engaged by Amazon – including several specialized in OOH advertising – this is the first experience of its kind.
First we randomized ads across Switch lines in a particular city (not São Paulo: The figure below is a hypothetical example). Secondly, we feel an email survey for the residents and ask what lines they regularly use. From the study we calculated the fraction of residents who had the opportunity to see ads in different neighborhoods. We then used this data to directly measure the effectiveness of an ad campaign.
This radically simple approaches that are striking cleaving results: During the race of an oven-wek ad campaign, the neighborhoods with high ad-exposure experienced a measurable increase in sales of low exposure neighborhoods.
Our experience design borrows concepts from a technique called GeoExperimmentation, an approach often used for online ads when individual use tracking or personalized ads are not available. Researchers implementing geoxperimmentation take a given country and isolate geographical units – states, regions or metropolitan areas – where they randomly use an ad campaign or not.
Geographical devices are typically chosen to be wide enough for advertising within a giving unit almost exclusively by people living there; In this way, the campaign’s influence is concentrated with the unit. To estimate the campaign’s influence, researchers then compare sales numbers from different geographical devices over time, with and without ads. Geoexperiments are effective because within a given country there is usually a large enough number of receiving geographical units for the experience to have sufficient statistical power.
There was a work assumption that OOH advertising would be a good candidate for a real implementation of geo-experience, given that Howe geographically concentrated OOH ad campaigns are. The typical geoxperiment requires dozens of geographical units to ensure sufficient statistical power, but OOH ads are generally deployed in only a handful of larger cities or in some cases a single city in a country.
Using more granulal geographical units-for example, urban neighborhoods or districts-Hells are not an option as there would be no way to isolate or identify users who may be able to see the ads. This is, for example, the challenge that a billboard is facing in a district that residents of other districts can drive by.
However, we realized that these challenges can be if you choose your OOH advertising around wisely. Targeting ads against metro and commuter train lines lets us isolate and identify the users who have the opportunity to see them. Our study asks that it looks at the metro and commuts rail lines that the respondent uses for commuting and occasionally traveling, how often they are and what part of the city they live in; This gives the fraction of residents exhibited for our ads in every neighborhood. At the same time, the random ad investment creates random variation in ad -exhibition in the city’s neighborhoods, giving us the number of geographical units we need for statistical power.
The idea is, therefore, to compare changes in sales in the treated and control regions, adjusted for historical sales trends. Many techniques used for analyzing geoexperiments also apply here, including synthetic control and the difference-iniférence methods.
With this first experimental design ever for OOH advertising, we think we have opened the door to the causal measurement of OOH ad-efficiency. To be sure, this implementation is specific to metro and commuter track ads that restricts it to cities that support that kind of public transport. However, we believe that experiences from such an experiment can be extrapolated to other types of OOH advertisement around it and can be combined with other, non-experience-based goals of ad-performance, such as consciousness Superys or multimedia models.