Nutrition-IQ Summer Camp for Little Brothers and Sisters of Pinellas County- Process Evaluation Plan

Complete and Acceptable Delivery

Ideal implementation of the Nutrition-IQ (N-IQ) Summer Camp program depends on successful coordination between the N-IQ team and Big Brothers Big Sisters (BBBS) staff, volunteers and resources across three areas:  orientation, activities and environment.

  1. Orientation

The orientation meetings with Big/Little pairs (Matches) and the Little’s parents/caregivers are critical to the understanding of program objectives and the specific activities designed to meet them. A successful orientation meeting requires BBBS staff to have pre-screened and recruited Matches with a Little age 10+.  Accordingly, towards the end of the school year eligible Matches and corresponding parents/guardians would receive information describing N-IQ camp and encourage them to attend an orientation meeting.  Parents may e-submit consent at the N-IQ website or send/bring the consent form with the Little to orientation.

The agenda during orientation includes laying out the plan and building excitement for weekly activities taking place over the following three weeks.  This includes having Matches get familiar with navigating the N-IQ website to identify before/after surveys, locate recipes and access the tools they need in regards to food safety and kitchen equipment.  Littles will be given time at orientation to e-submit baseline surveys on knowledge, self-efficacy and attitude around preparing food at home. Lastly, Bigs will be provided with Wal-Mart gift cards at orientation to assist each Match with the costs of grocery shopping.

2. Activities

The completion of three activities by each oriented Match is entirely supported by the N-IQ website as described below:

  • Activity #1- Matches discuss a list of fruits and vegetables that provide nutrients teens are often short on, select a fruit or vegetable recipe at the website and then visit the Wal-Mart or an electronic-benefits (EBT) accepting produce store in the Little’s neighborhood.  Items to make the recipe are then taken to the Big’s house where chopping skills are practiced as part of recipe preparation.
  • Activity #2- Matches take a kitchen equipment inventory at the Big’s house and shop together at a thrift store or Good Will in the Little’s neighborhood to purchase any missing items. Matches also choose a main dish recipe for Activity #3.
  • Activity #3- Matches shop together at the neighborhood Wal-Mart to purchase ingredients and then make the main dish recipe together. Matches post pictures of their creations and Little’s e-submit the post-camp surveys.  Matches also commit to a cooking or food-related quarterly goal, i.e. prepare one new recipe a month for the next three months.  Recipes take appeal, culture, ingredients and equipment, nutrition and food allergies/intolerances into consideration, such that variety of cost conscious options are provided that include Wal-Mart’s Good Value brand.

BBBS staff and volunteers contact each Big via email once a week during the three weeks of camp to gather feedback on the prior activity and keep a record of completion.

3. Environment

A supportive environment for N-IQ Camp starts with parenteral/guardian consent.  This is assisted by the trust parents have with BBBS staff and program launch communications.  From there Big’s provide a kitchen environment with the basic equipment needed to prepare the recipes selected.  Bigs have a large role in encouraging the cross-over of the equipment inventory to the Little’s home environment. The identification of produce markets that accept EBT and gift card grocery shopping at the neighborhood Walmart also help to create an environment that improves local access to foods for preparation at home.

If these aspects of program implementation are well executed, the delivery would be considered complete and acceptable (Saunders, Evans & Joshi, 2005).

 

Process Evaluation Questions are largely formative in nature.

Target of Question Process- Evaluation Question Method of Assessment for Question Resources Required
Program Management/ Recruitment What process was used to identify and encourage eligible Matches to enroll? Database pull of Matches with a Little aged 10+. Email and phone script for recruitment BBBS staff do database pull and NIQ team writes email and script.
Reach What percentage of eligible Matches enrolled in the program? Number of Matches emailed/phoned vs. Consent Forms on file BBBS identify eligible Matches using their database and keep spreadsheet on email and phone recruitment. N-IQ team counts consent forms submitted.

 
Fidelity During orientation did the N-IQ Camp moderator reinforce how often activities are planned to occur and in which order?

 

 Logic Model Internal Observer from BBBS.  N-IQ team volunteer to record data for final report.
Activities Are week 1, 2 and 3 activities culturally sensitive and designed to meet the needs of the Little? Program evaluations completed by Bigs Evaluation forms created jointly by BBBS and N-IQ team.  Emailed by BBBS staff; data compiled by N-IQ team.
Dose Delivered What percentage of Matches completed Week 1, 2, 3 and all activities? Weekly phone/email surveys of Bigs.  Photos of Week 3 activity posted to N-IQ site. N-IQ team to keep spreadsheet tally and analyze.
Dose Received Did Littles’ scores for knowledge, self-efficacy and attitude significantly improve? Pre and post surveys completed at N-IQ website Likert scale questionnaires for measuring attitude and self-efficacy; multiple choice surveys for measuring knowledge.
Context Will BBBS make a continued commitment to supporting the program? Process evaluation report NIQ team to write with input from BBBS staff and camp participants

Plan Summary

Ultimately, this Process Evaluation Plan is designed to measure the two most important aspects to judge a program by:  quality and effectiveness.  Was the program put in place according to the rigor of its design, and in such a way that a significant portion of Littles received a sufficient educational and environmental dose of cooking as a life skill to propel them into doing more cooking and achieving a healthier diet?

The process evaluation questions above represent key measurements for the Nutrition-IQ Summer Camp program for Little Brothers and Sisters of Pinellas County.  They follow a logical order across various aspects of program implementation:  recruitment, reach, fidelity, activities and dose delivered vs. received.  For each question the method of assessment is specified to make clear how the data will be gathered, by whom, and using what resources.  Note:  the evaluation process is highly dependent upon BBBS staff that are already in place, the N-IQ team of volunteers, the Matches participating in Camp and the availability of the N-IQ website to accept e-submissions.

Because of privacy concerns, determining program eligibility and sending recruitment communications to Littles and their parents/guardians would require additional time on the part of BBBS staff.  However, the preparation and analysis of data collected from submission of surveys, questionnaires and forms via the website and otherwise would be handled by the N-IQ team.

 

Saunders, R.P., Evans, M. H., & Joshi, P. (2005). Developing a Process-Evaluation Plan for Assessing Health Promotion Program Implementation: A How-To Guide. Health Promotion Practice, 6(2), 134-14