Abstract
In this time of information overload, successfully engaging farmers with compelling outreach materials is a major challenge for conservation programs and related research projects. One potential approach is targeting information to the recipient, e.g., local rather than regional soil and water conditions, when sending messages to farmers. Targeted information may increase engagement by making materials stand out as more relevant and useful; conversely, it may decrease engagement by making farmers wary of the program and how it is using the information. We tested the effect of targeted information on farmer engagement using a large, randomized controlled trial in Iowa. In partnership with Iowa State University, we sent 2,996 farmers a single mailing with information about erosion at the local watershed (targeted) or state (control) level and measured their responses to a two-minute survey. We found that targeted information increased relative response rates by 20%, from 13.8% to 16.4%. This level of increase is meaningful for practitioners, as well as statistically significant. Our findings show that targeted information can be an important tool for practitioners and researchers seeking to better connect with farmers who are inundated with marketing mail.
Introduction
The success of conservation programs, and related research, depends on engaging farmers and landowners (Reddy et al. 2020). Engaging US farmers is especially difficult because 75 billion pieces of marketing mail are competing for their attention each year (USPS 2019). In-person methods of outreach have high costs that limit their scalability for a large audience. Low-cost and scalable contact methods like mailings, emails, and phone calls, therefore, are important tools. Making these contact methods more effective is critically important to reducing costs and increasing participation rates for conservation programs and research.
Increasing engagement has long been of interest to those doing outreach. Popular strategies include repeated survey mailings, reminders, and incentives (Dillman 2011), which are costly but may achieve higher response rates even among farmers and landowners (Glas et al. 2019; Weigel et al. 2020). Advances in data and analytics make new, low-cost strategies possible to replace or augment more costly methods. Programs and researchers can customize their messages using databases compiled by private marketing companies, like FarmMarketiD, that track information including age, gender, types of crops or livestock, field location, and acres farmed. These data could be employed on their own or combined with sources of environmental data to identify farmers with certain soil types or in areas of high erosion, for example. With this information, messages could be targeted to improve engagement. Two prominent theoretical mechanisms for targeting are the increased usefulness of relevant information and the ability to grab the attention of the recipient (Skinner et al. 1994).
Outside of agriculture, there is evidence that targeting information can increase engagement in areas of education and medicine, for example. The Social and Behavioral Science Team (SBST) collaborated with the US Education Department to learn if a targeted email could encourage borrowers to apply for an income-driven repayment program (Social and Behavioral Science Team 2016). The targeted message altered the standard wording of the email to fit the borrower’s repayment category and reminded the borrower if they had previously indicated interest in the program. The result was an 8.5% increase in applications for the program relative to the standard email, from 4.7% to 5.1%. Targeted letters in medicine encourage healthy choices or scheduling an appointment. While these studies have small sample sizes, meta-analyses suggest that targeted messages can have substantial effects. Krebs et al. (2010) found a 36% increase, relative to the baseline, across health behaviors including smoking cessation, healthy eating, and mammography screening. Targeting personal risk factors for breast cancer increased screening by 50% according to Edwards et al.’s (2003) evaluation of several studies, with the increase in screening concentrated among the high-risk.
There is little evidence on targeting messages in the agricultural domain, although increasing awareness and concern is a primary behavior change approach (Reddy et al. 2017). An experiment conducted by SBST in partnership with the USDA sent targeted letters to small-scale farmers with contact information for the recipient’s local loan officer. Messages increased program participation by 22% relative to the control group; however, the control group received no letter (Social and Behavioral Science Team 2015). The effect of the treatment cannot be solely attributed to targeting, and it is infeasible to detect a negative effect from targeting itself without an untargeted letter.
Targeted information has the potential to reduce response rates. Farmers may be especially wary of outsiders possessing personal information and be more reluctant to give additional information (Wiseman et al. 2019). Surveys and interviews with farmers confirm that they often do not trust organizations with their data (Slattery et al. 2020), and that a lack of clearly defined data rights can further reduce trust (Jakku et al. 2019). Additionally, attempts to frame conservation positively have shown negative reactions in surveys (Andrews et al. 2013). In sum, it is important to empirically and rigorously test the effect of targeted information by making a comparison to a control group because there is the potential for targeted information to either increase or decrease response rates.
We hypothesize that a targeted message, using local environmental data and coming from a known messenger, will increase engagement. We test this hypothesis with a sample of nearly 3,000 Iowa farmers in high-erosion areas using a randomized controlled trial (RCT) design. The targeted message has information on erosion and costs of erosion from the watershed in which a farmer’s field is located. Specifically, we use data from hydrologic unit code 12 (HUC-12) watersheds (~100 km2 [39 mi2]). The control message has information at the state level. Both messages use the Iowa State University logo, making the sender known but without a relationship for this specific program, as would be common for messages from land grant universities and government agencies. Both messages allowed response on paper or online, indicating the relative importance of physical mail. The results of this test provide evidence to inform organizations’ outreach strategies while simultaneously providing useful information to farmers in an important region for soil erosion.
Materials and Methods
To understand the effect of targeting messages, we contacted a large sample of farmers in high-erosion watersheds along the eastern border of Iowa. Erosion is a significant problem in this area; roughly 10 tn ac−1 (22.4 Mg ha−1) of soil are lost annually (Gelder et al. 2018), well beyond the natural replacement rate. All messages contained information about erosion, erosion-mitigating practices, and the erosion rate at either the state or watershed level. Farmers were asked to complete a brief survey included in the mailing and available online.
Soil erosion in Iowa is a major contributor to water quality issues, including the hypoxic (“dead”) zone in the Gulf of Mexico, which is caused by excess nutrients flowing out of the Mississippi River (Jones et al. 2018; Rabotyagov et al. 2010). We identified priority watersheds as those near the Mississippi River with high erosion rates, while also attempting to make the selection of watersheds contiguous (figure 1). Erosion rates were identified using data from the Daily Erosion Project (Gelder et al. 2018). Farmers were selected if they had at least 5 ac (2 ha) of corn (Zea mays L.) or soybeans (Glycine max [L.] Merr.) in any priority watershed. We included 84 watersheds in our study to reach the target of roughly 3,000 farmers, which comes from our power analysis. Assuming a 5% response rate, power was 78% to detect a three percentage point treatment effect. Farmers’ field location, crop, and contact information were obtained from the private company, FarmMarketiD. Most watersheds experience more erosion than the state average, but due to our aim to keep the selection contiguous, some watersheds had lower than average erosion. All farmers were assigned to the watershed in which they had the most acres for the randomization procedure and analysis.
HUC-12 watersheds of Iowa included in the randomized control trial.
Farmers received either the control message or the targeted message. The control message used erosion data for the state of Iowa and an image of the state. The targeted message used data for the watershed and an image of the watershed highlighted within the county. The image of the county was chosen because farmers may not be aware of what watershed they are in, but, based on personal interactions with farmers, they likely recognize their county and their farm’s location.
We use an RCT design and pre-analysis plan. An RCT allows for a causal interpretation of the targeted message effect by comparing two groups of farmers that are, in expectation, similar in every way. We create these two groups by randomly assigning farmers to either the control or targeted message. The design ensures that, aside from random chance, the difference in response rates is due to differences in our messages, rather than a result of differences in farmers. Without randomization, response rates might be explained by certain farmers being more responsive perhaps because they are environmentally conscious, for example. Randomizing farmers ensures that farmers with particular attributes (e.g., environmentally conscious) are no more likely to be in one treatment than another, even if we cannot observe their attributes. To add further credibility to our results, we posted a pre-analysis plan at https://osf.io/pdcw8/ before analyzing data. The pre-analysis plan specifies the model, covariates, and subgroups in advance, reducing the opportunity for mining data or applying numerous specifications to obtain desirable results.
Our randomization procedure includes steps to address potential confounds to the design. One is block-randomization, which improves treatment balance, and the second is variation in treatment saturation to detect spillover effects. In this study, farmers in a watershed (block) were assigned to the treatment or control group in a fixed proportion (saturation), which ensured that no watershed had, for example, all farmers assigned to the treatment. The proportion in the treatment was either two-thirds or one-third, which allows us to test if there is a spillover effect, for example, a farmer being more likely to respond because a neighbor received the targeted message. Controls for assignment block and saturation are used in the analysis. For more on the randomization process, see the pre-analysis plan. Watersheds with fewer than 20 farmers were grouped together, for a total of 76 randomization blocks. Table 1 shows that, by observable measures, the farmers in each group appear to be similar.
Balance across control and targeted groups.
There is no standard way to target a message. Our mailer emphasized two proposed mechanisms through which targeting increases engagement: more relevant information and drawing the recipient’s attention. Local erosion information is more relevant to a farmer than state erosion information, and an image of the farmer’s county with their watershed is more likely to stand out from an image of Iowa. Both the erosion information and the county/state image were displayed on the outside of the mailing and could be seen without opening the mailing (see figure 2 and figure 3; full mailing in the supplemental materials). The mailing was a trifold design, with a detachable postage-paid survey, online survey link, and erosion information inside.
Outside of mailing for the control group.
Outside of mailing for the targeted group.
Mailings were sent in March of 2020, and responses were collected through June. The survey was attached, with postage prepaid, or could be filled out online. Each farmer was given a random identification number on their mailing, which identifies both physical and online respondents. We use a linear probability model to estimate the effect of the targeted message, as specified in the pre-analysis plan:
1
The Block variable is a set of indicators for treatment assignment blocks, which is a watershed except for very small watersheds that were combined into a single block. It is standard to include treatment assignment blocks in the analysis and was specified in the pre-analysis plan.
Results and Discussion
The overall response rate was 15.1%, with 90% of responses by mail rather than the online survey (table 2). Our primary specification uses the linear probability model, which is chosen for its ease of interpretation and was specified in the pre-analysis plan. We find that the targeted message significantly increased the response rate by three percentage points (p = 0.045), or a 20% increase relative to the control group (table 3). Results are similar for a logistic regression, which estimates targeting increased the response rate by 25% (p = 0.042) (table 4). We explore variation in response rates for three subgroups: high treatment saturation, high erosion, and farm size. Despite having nearly 3,000 individual farmers in our dataset, the test is underpowered to detect small differences. We do not detect significant differences amongst the subgroups (tables 3 and 4, column 2).
Response by message and medium.
Linear probability model of response rates. High saturation is an indicator for two-thirds of farmers in that watershed being assigned to the targeted message. Erosion is measured as the amount of erosion in tons per acre reported in the message. Large farms are farms with more than the median, 91.7 ac, in the priority watersheds.
Logistic model of response rates.
This study provides a straightforward evaluation of targeting messages in real-world agricultural outreach, adding to scientific literature and practice by using a preregistered and statistically powerful design with an environmentally important population. The RCT design allows detection of both positive and negative effects, which is important because attempts to increase farmer interest in conservation practices can have a negative impact (Andrews et al. 2013). Farmers are especially wary of giving information (Wiseman et al. 2019) and may be deterred by targeting because it appears invasive. We used information that does not individually identify the farmer’s land—such an approach could be more effective or appear overreaching without an established relationship (Bronson and Knezevic 2016). We estimate the net effect of targeting in this experiment. If messengers can mitigate the negative impacts of targeting, the net effect may be even larger.
Survey responses show two distinguishing features: farmers overwhelmingly chose to complete the physical, rather than digital, survey, and were more likely to use conservation practices than typical Iowa farmers (see supplemental materials for details). The high proportion of physical responses demonstrates the continued relevance of physical mail even when digital options are available, which is corroborated by the literature on messaging farmers (Weigel et al. 2020). Physical mailings will likely remain an important medium for outreach; the Danish population shows a preference for physical mailings when responding to a public authority despite having digital mailboxes expressly for that purpose (Ebert et al. 2018).
The composition of responders is an important feature for both researchers and conservation programs. Self-selection by responders biases estimates of behaviors, such as conservation practices, in the population. Though we find the observable characteristics of responders, farm size, and erosion are not significantly different from the full sample (table 3 column 2), responders were more likely to use conservation practices. Roughly 4% of Iowa farms use cover crops (Sawadgo et al. 2021), yet 41% of our survey respondents claim to (see supplemental materials for more). Future research could explore how targeted information can increase response rates among underrepresented populations, perhaps by using destigmatizing wording (The People Lab 2021), resulting in greater external validity (Bethlehem 2010). One technique could be to target information to specifically attract those who are unlikely to respond, such as farmers who do not use conservation practices. Conversely, targeting the farmers who are most likely to participate may be most cost-effective for conservation programs.
Targeting increases the cost per message sent but could reduce the cost per response through higher response rates. In this study, targeting increased the cost of data and mailing by approximately 16%, and added hours of the researchers’ time to create differentiated messages. While cost per response was slightly higher for targeting, including the researchers’ time, in many agricultural contexts the limiting factor in recruitment is the response rate. This study messaged roughly half of the relevant farmers in Iowa, while Reddy et al. (2020) messaged approximately one-third of eligible landowners across three states; the inability to infinitely scale outreach makes response rates potentially a more important outcome measure than the total number of responses.
We are not able to measure on-farm actions in response to our mailing, but it is possible the targeted information made farmers more willing to act to control erosion. In a study on Medicare choices, Kling et al. (2012) found that providing cost information, already freely accessible, increased plan switching and saved the treatment group roughly 5% of their annual payments. Future studies testing targeted information should aim to measure real actions rather than hypothetical actions or intention to act, when possible.
Summary and Conclusions
There is no single solution to increasing engagement among farmers. Conservation programs and academic researchers need new ways to stand out from marketing mail and get useful information to farmers. Our results suggest that mail remains an important medium for farmers as more than 90% of responses were through physical mail even though an online option was provided. We find that targeting a message may be an effective and scalable way to present more useful information and catch the recipient’s attention, increasing relative response rates by 20% in our RCT. Higher response rates reduce mailing costs, especially for hard-to-reach populations (Reddy et al. 2020), and may be the only option for programs messaging an entire group without the ability to scale the number of messages. Our study is among the first to test information targeting in the agricultural context—where it is plausible that targeted information could reduce response rates.
Given the many factors that can affect how messages are received, it is important that other researchers and practitioners continue to test targeted messages. This RCT demonstrates a strong methodology to compare farmers who received a control message to those who received the targeted message, but replications are needed with different targeting strategies, populations, and outcome measures such as actual program take-up to better understand how and when targeting can be used most effectively. We encourage other researchers to adopt similar methodology, including a pre-analysis plan, which specifies, in advance, the outcomes, model, ex ante statistical power, and subpopulations to be analyzed. Using these methods can improve the credibility of research to be adopted by conservation agencies and government organizations (Rosch et al. 2021) that would benefit from improved engagement with agricultural constituents.
Supplemental Material
The supplementary material for this article is available in the online journal at https://doi.org/10.2489/jswc.2022.00145.
Acknowledgements
This project was funded by grant 1910-036 from the National Wildlife Federation and supported by funding from The Nature Conservancy.
- Received September 23, 2021.
- Revision received February 23, 2022.
- Accepted February 25, 2022.
- © 2022 by the Soil and Water Conservation Society
References
In this issue
Jump to section
Related Articles
Cited By...
- No citing articles found.