Social Norms Campaign Strategy - Final Report

Evaluating a Social Norms Media Campaign

A social norms media intervention works in three stages:

  1. The target group is exposed to high levels of the social norms campaign message;

  2. Repeated exposure to this message works to correct the target audience's misperceptions about its peers' attitudes and behavior;

  3. These new, more accurate perceptions of peers help shift individual behavior in the direction of actual norms.

Thus, there are three key evaluation questions in assessing the progress and success of a social norms campaign:

  1. Did the campaign provide social norms message exposure?

  2. Did the campaign impact perceptions of peer norms?

  3. Did the campaign impact personal attitudes and reported behaviors?

This campaign successfully provided social norms message exposure, impacted the perception of peer norms, and changed personal attitudes and reported behaviors. Each of these evaluation questions will be addressed in turn.

Social Norms Campaign Message Exposure

The MOST of Us Prevent Drinking and Driving campaign was designed to strategically test the effectiveness of paid media by separating the State into control and intervention areas. Data regarding campaign message recall collected in the Time 2, Time 3, and Time 4 surveys shows unequivocally that campaign recognition in the intervention counties was significantly higher than in the rest of the State.

The data also reveals that respondents from counties neighboring the intervention area exhibited a much higher recall of the campaign message than their counterparts in the more distant eastern part of the State. The media message seems to have migrated eastward from the intervention area. This result makes sense, since radio and television messages cannot be completely contained within the target county lines. Some media will inevitably "drift" to neighboring counties on the broadcast airwaves. Moreover, travel between the intervention and non-intervention counties could cause non-intervention area residents to be exposed to the intensive social norms campaign. This "diffusion" of the campaign message through mobility would likely be greatest for residents in neighboring counties as compared to the more distant counties where travel back and forth would be less common.

To adjust for this "diffusion effect," MSNP defined the formal control group as the 26 counties in the eastern half of the State. The remaining 15 counties that were adjacent or in close geographical proximity to the experimental target area were designated as "buffer counties" (see figure 4). It was in these buffer counties that the greatest unintended campaign exposure could be expected.

Figure 4
Intervention, Buffer and Control Counties
figure 4 - click for long description

Unprompted Recall

In each of the four surveys, respondents were asked if they remembered seeing or hearing any alcohol prevention campaign advertisements (posters, radio or television commercials, brochures) and if so, what was the main message they remembered. If they volunteered having heard a message about "most of us," "the majority," or "4 out of 5" associated with drinking moderately, not driving after drinking, or using a designated driver, they were identified as recalling a social norms message as the main message.

At the outset of the campaign (Time 1) statewide campaign recognition was just under 52 percent. By the end of the campaign period (Time 3), unprompted recognition of the social norms message as the predominant media message about drinking (compared to recalling other or no message) had risen to 74 percent in the intervention counties. Three months later (Time 4) this level had fallen only slightly, to 71 percent. In the control counties, unprompted campaign recognition initially fell off sharply, to 29 percent at Time 2, but then steadily grew to 37 percent at Time 3 and 43 percent at Time 4. The difference in Time 1 to Time 4 change between the intervention and control counties was 24.8 percent (p<=.001).

In the buffer counties, the percentage of respondents who recalled a social norms message as the main message remained in between those found in the experimental and control counties for the Time 2 to Time 4 surveys, as would be expected. (See figure 5)

Figure 5
Percent Recalling Social Norms Message About Drinking as the Main Message From Media (compared to recalling other or no message)
figure 5 - click for long description

Prompted Recall

Respondents who did not volunteer having heard a social norms message as the main message were asked whether they recalled seeing or hearing any alcohol prevention campaign advertisements mentioning "most of us," "the majority," "4 out of 5," "80 percent not drinking and driving" or "80 percent using a designated driver." Between Time 2 and Time 3, prompted recall grew from 82 percent to 94 percent in the intervention area and from 55 percent to 66 percent in the control area. Prompted recall continued to increase after the paid campaign period ended, to 95 percent in the intervention area and 71 percent in the control area by Time 4. Prompted recall of a social norms message in the buffer counties remained in between the levels in the experimental and control counties, as would be expected. (see figure 6)

Figure 6
Percent Recalling Any Social Norms Message About Drinking in Last 12 Months (Including Prompting with Examples if No Voluntary Recall)
figure 6 - click for long description

These results clearly show that devoting resources to the high-intensity paid media placed in premium space in the experimental counties created a more powerful, memorable message that was recalled more often - even months after the campaign's conclusion - in comparison with the control counties. This was true even though the control counties likely received some diffusion of these messages in addition to PSAs, newspaper ads, and promotional items during the campaign time period.

Campaign Impact on Perceived Norms of Peers

Having shown that intense doses of the campaign message in the intervention counties correlated with message recall, the data was examined to see if misperceptions about Montana young adult norms were effectively reduced in the target audience over the 15-month period. Changes observed in perceptions of the pervasiveness of impaired driving in the intervention counties might be attributable to seasonal variation in impaired driving patterns (or perceptions of these patterns) or to changes in statewide legal deterrent policies and practices. It is also possible that an apparent lack of change in the intervention area could be the result of these seasonal or statewide influences suppressing or canceling out the positive effects of the intervention. Any changes observed in the western intervention counties over time must therefore be considered in relation to what changes took place simultaneously in the eastern control counties, where social norms media exposure occurred least.

As indicated by figure 7 (below), the high-intensity media intervention in the western counties proved to be a much more powerful tool for correcting misperceptions of the Montana norm than the low-intensity free media and other types of messages that were dosed to the eastern control counties. At Time 1, young adults statewide were suffering under the same misperception: 92 percent erroneously believed that in the past month the average Montanan their age had driven within one hour of consuming two or more drinks. By Time 4, the number of young adults in the intervention counties who believed that in the past month their peers had driven while potentially impaired dropped to 87 percent, while the prevalence of this misperception in the control counties increased to 94 percent. The difference in Time 1 to Time 4 change between the intervention and control counties was -7.5 percent (p<=.05).

Figure 7
Percent Thinking the Average Montanan Their Age Has Driven within One Hour of Consuming Two or More Drinks in the Past Month
figure 7 - click for long description

Changes were also found regarding the perception of peer use of designated drivers (see figure 8). At Time 1, 30 percent of intervention county respondents and 25 percent of control county respondents believed that the majority of Montanans their age almost always have a designated driver with them when they consume alcohol and will be riding in a car later. By Time 4, the intervention county respondents had a much more accurate picture of the normative environment, with 39 percent responding that they believed a majority of their peers used designated drivers. During this same period, misperceptions in the control county remained virtually unchanged. For this measure, the difference in November 2001 to June 2003 change between the intervention and control counties achieved marginal statistical significance (p=.056).

Figure 8
Percent Perceiving Majority of Same Age Montanans Almost Always Have a Designated Non-Drinking Driver with Them When Drinking and Will Be Riding in a Car Later
figure 8 - click for long description

In short, at the end of the campaign period the target population in the intervention counties was more accurately seeing the normative peer environment than was its counterpart in the control counties. By intensively exposing the intervention counties to messages about the existing healthy norms in their communities, this campaign successfully brought their perceptions of the behavior of their peers closer to the actual norm.

Campaign Impact on Personal Attitudes and Reported Behaviors

Upon finding that misperceptions of peer norms were reduced by the intervention, the final piece of the social norms model could be tested. Specifically, did changed perceptions translate into changed behaviors?

The data showed that respondents' reported changes in behavior very closely followed their changes in perception. Between Time 1 and Time 4, the percentage of young adults in the intervention counties who reported driving within an hour of consuming two or more drinks in the previous month dropped slightly from 23 percent to 21 percent (see figure 9). This is significant when compared to the change in the control counties, where the percentage who reported similar behavior grew from 17 percent to 29 percent between these November 2001 and June 2003 measurements. The increases in problematic behavior in the control area may have reflected some general increase in problem behavior in the State, but it is more likely that an increase in this behavior may simply tend to occur in the warmer months of the year. The western counties, with the protection of more accurate perceptions of peer norms, did not appear to suffer this deterioration. With reported driving after drinking decreasing in the western counties by 2 percent and increasing in the eastern counties by 12 percent, there was an overall relative decrease of 14 percent in the west compared to the east ( p<=.05).

Figure 9
Percent Driving After Having Two or More Drinks Within the Hour in Past Month
figure 9 - click for long description

The percentage of intervention county residents who reported they always (100-percent of the time) use a designated driver if they plan to drink increased from 42 percent to 46 percent from November 2001 to June 2003 (see figure 10). In contrast, consistent use of designated drivers in the control counties dropped from 42 percent to 32 percent during the comparable period. The notable drop in the use of designated drivers in the control counties is more likely due to the greater tendency to drink and drive during the warmer, drier summer months than to an actual shift of that magnitude in the rates of impaired driving. With either interpretation, however, what is key for this study is the relative result in the experimental counties. There was a statistically significant 15-percent difference in the overall change for the intervention counties compared to the control counties (p<=.05). The counties exposed to the intensive intervention avoided the downturn observed in the control counties.

Figure 10
Percent Reporting They Always ("100 percent of the time") Make Sure They Have a Designated Non-Drinking Driver Before They Consume Alcohol If They Will Be Riding in a Car Later
figure 10 - click for long description

As an additional measure of campaign impact on personal attitudes and behaviors regarding impaired driving, support for reducing the allowable blood alcohol concentration for drivers from .10 BAC (the legal limit in Montana at the time of this study) to the more conservative .08 national standard was examined. The State has since enacted an .08 per se law. Even though no specific campaign messages targeted the BAC issue, the data reveals that young adult support for a stricter BAC corresponded with improved accuracy of perceived impaired driving norms. At the November 2001 baseline survey, 64 percent of the intervention county respondents and 71 percent of the control county respondents supported such a change (see figure 11). By the end of the media intervention, support for this change in the intervention counties had increased 7 percent while support in the control counties had eroded by 9 percent, producing a relative difference of 16 percent over the course of the intervention (p<=.01).

Figure 11
Percent Supporting Changing BAC Legal Limit for Driving to .08
figure 11 - click for long description