Process Evaluation
Step 1: Program Overview
Through sharing practical and relevant knowledge about sugar-sweetened beverages online, the mission of EatGoodFeelGood (EGFG), a virtual nutrition education program, is to help Rhode Island families learn and practice healthier dietary behaviors. The EGFG program utilizes the social ecological model to address multiple levels on influence to deliver more well-rounded support for families attempting behavior change. Consisting of a blog, Instagram account, and Facebook account, EGFG works to achieve its goal of reducing consumption of sugar-sweetened beverages in Central Falls (CF) families with young children, and subsequently reducing rates of childhood obesity in CF. By providing easily accessible bilingual nutrition education to parents and guardians, not only does EGFG address individual level determinants of the follower, EGFG encourages changes in the social environment of the families. Through increased knowledge and self-efficacy, parents and guardians can effectively support their entire family by teaching and modeling healthy beverage choices (Contento, 2016, p. 83).
Additionally, EGFG creates partnerships with CF convenience stores to achieve its goal of increasing availability and advertising of no added sugar beverage options in stores CF families have easily accessible. By providing stores the incentive of being promoted on EGFG platforms, stores will highlight beverages they sell that contain no added sugar through sales or signage. Most foods and beverages are bought as a direct result of advertising. Therefore, by manipulating the target audience’s environment by increasing the advertisement of beverages with no added sugars, consumers will be further influenced to opt for those instead of their sugar-sweetened counterparts. This would in turn increase demand and consequently increase supply (Contento, 2016, p. 83).
Finally, EGFG materials will work to address two sectors of influence: systems and policy. EGFG will encourage followers to critically assess the food and beverage advertisements they and their families are exposed to through blog and social media posts exposing relevant examples and providing thoughtful discussion prompts. With this information, details will be provided on how the reader can reach out to their local government representative encouraging legislation on stricter limitations for food and beverage advertising. Information will also be linked on the EGFG website and shared on where to find and how to access local resource systems, including food and healthcare assistance agencies (Contento, 2016, p. 83).
Step 2: Complete and Acceptable Delivery
To be published on EGFG platforms, all posts must consist of science-based nutrition and health information referencing credible sources from the last ten years. Additionally, the information provided in the posts must be related to and relevant to the target audience concerning or somehow related to the program’s target behavior: consuming less sugar sweetened beverages. Posts will also be translated into both English and Spanish by fluent speakers of each language who are also familiar with the subject. EGFG will post 2 blog posts per week and one post each on Facebook and Instagram per day. The content for these posts can be produced in bulk and scheduled to be posted throughout the month/week as needed.
To gauge how the materials are being received by the target audience, individuals can interact with the content (i.e., ‘liking’ the post), comment, and share. The reach of EGFG will then be measured using social media analytic software that can track information including number of views, shares, and likes. Additionally, Instagram and Facebook questionnaires will be used to elicit and gather direct audience feedback. Bi-weekly Instagram Q&A’s will be posted, encouraging followers to submit any questions they have or topics they would like to know more about. Facebook Polls will be used once a month to ask followers to rate how satisfied they are with EGFG.
EGFG will also be partnering with at least two local convenience stores in Central Falls (02863) to advertise no sugar added beverages in their stores in exchange for promotion on EGFG platforms. In order to qualify for EGFG promotion, the stores must at minimum place an approved no sugar added beverage at consumer eye-level and display an EGFG informational tag. EGFG will post 2 promotional posts per month while the store maintains the advertising. As mentioned above, the EGFG content on virtual platforms can be interacted with directly by individuals and the reach can be tracked by social media analytic software. After the 2 promotional posts have been made, EGFG staff will check in with store owners to hear anecdotally how the advertisement was received and if sales of the product went up.
Steps 3-5: Process Evaluation
Target of Question | Process- Evaluation Question | Method of Assessment for Question | Resources Required |
Recruitment | 1. How were social media and blog posts advertised? 2. How were stores recruited? | 1. Meta Insights will gather and track data on advertising statistics. Data will be gathered and analyzed monthly by EGFG staff. 2. Analysis of store partnership records | 1. Facebook/Instagram business account; EGFG staff; staff hours. 2. EGFG staff will keep detailed records of recruitment including store address and contact information; when possible, email documentation will be used. Stores will have to sign a partnership agreement consenting to being featured on EGFG in exchange for featured product promotion as dictated by partnership requirements and reporting monthly sales of these products. |
Reach | 1. How many views has each post gotten? How does this compare to previous posts? 2. How many of the promoted beverage were sold during the one-month period? How does this compare to previous months?* | 1. Meta Insights will gather and track data on advertising statistics. Data will be gathered and analyzed monthly by EGFG staff. 2. Analysis of monthly sales reports from stores by EGFG staff. | 1. Facebook/Instagram business account; EGFG staff; staff hours. 2. Monthly sales reports from stores; EGFG staff; staff hours. |
Fidelity | 1. Are posts using science-based information? Are posts relevant and relatable to the target audience and target behavior? Are posts providing advocacy information when relevant? Are posts being translated into English and Spanish? 2. Are store owners putting approved no sugar added beverages at consumer eye-level? Are store owners displaying EGFG tags next to products? Have at least 2 promotional posts been posted? | 1. Posts will be cross-examined by at least 2 different EGFG staff with at minimum one of those staff members being fluently bilingual in English and Spanish. 2. Once per month, an EGFG staff member will make an unannounced visit to the store to confirm advertising meets partnership requirements and to share insights from monthly posts | 1. EGFG staff; staff hours. 2. EGFG staff; staff hours; store owner time. |
Context | 1. Have there been any confounding events that might have affected the community?* | 1. Reports during monthly EGFG staff meetings | 1. All EGFG staff are required to report out on any confounding events they have come across out in the community, heard on the news, etc. |
Dose Delivered | 1. How many posts are being posted? When are posts being posted? 2. How many beverages are being promoted? | 1. Meta Insights will gather and track data on advertising statistics. Data will be gathered and analyzed monthly by EGFG staff. 2. Once per month, an EGFG staff member will make an unannounced visit to the store to confirm advertising meets partnership requirements. | 1. Facebook/Instagram business account; EGFG staff; staff hours; store owner time. |
Dose Received | 1. How many likes, comments, and shares has each post gotten? How does this compare to other posts?* | 1. Meta Insights will gather and track data on advertising statistics. Data will be gathered and analyzed monthly by EGFG staff. | 1. Facebook/Instagram business account; EGFG staff; staff hours. |
Step 6: Significant Process Evaluation Questions
The questions indicated above with an asterisk will be critical during process evaluation as these will provide the most information on program success and contributing factors. The question regarding the reach of the convenience store partnerships will assess the success of these partnerships. An increase in reach indicates a successful partnership while a steady or deceased reach suggests the partnerships need to be evaluated or further supported. The question regarding context and any confounding events allows the process evaluation to take into account anything outside of the program’s control. Instances, such as the continued COVID-19 pandemic and recovery, may occur that could limit or hinder EGFG efforts and should thus be taken into consideration when evaluating program results.
Finally, the dose received question pertains to the effectiveness of EGFG postings. By understanding how many posts are being received, EGFG staff can ensure and prove that content interactions continue to grow and be positive. This question can also provide insight into the material’s reach, even though that is not the primary focus of the question.
REFERENCES
Contento, I. R. (2016). Nutrition education: Linking research, theory, and practice (3rd ed.). Jones & Bartlett Learning, LLC.
Meta. (n.d.). Facebook IQ- Audience insights. https://www.facebook.com/business/insights/tools/audience-insights
Saunders, R. P., Evans, M. H., & Joshi, P. (2005). Developing a process-evaluation plan for assessing health promotion program implementation: A how-to duide. Health Promotion Practice, 6(2), 134–147. https://doi.org/10.1177/1524839904273387