Process Evaluation Plan

Complete and Acceptable Deliveries for SANE Program

The Star Athlete Nutrition Education (SANE) program will consist of three fundamental components: an environmental component focusing on creating a SANE intervention team, the curriculum component which is broken down into two modules: Fueling the Female Athlete and Positive Body Image, as well as a training component to help prepare coaches and squad leaders.

The Environmental Component:

A SANE team will be formed at each school to develop a strategic plan for delivering the program to female sports teams from Freshman to Varsity as acknowledged by a written plan. Team members will include the athletic director, two coaches, and one SANE program expert. Additionally, the school gym environment will be supportive of the program by providing exhibition space for projects produced by the female student-athletes.

The Curriculum Component:

The SANE curriculum is comprised of two main modules —Fueling the Female Athlete and Positive Body Image. The 8-lesson plan curriculum is designed to be delivered during each sports season (fall, winter, and spring depending on the sport) with an average time of 45 minutes to an hour per session. There will be one squad leader per 10 female athletes. The curriculum is designed to be team-based and highly participatory, which acts as an almost team-building session for the teams. Squad leaders are encouraged to engage their teammates in a way that aids in critical thinking and discussion. Participation throughout this program will be key especially in the latter portion of the intervention when athletes are required to perform some role-playing activities and work in teams. The curriculum components are designed to be delivered in sequential order because the knowledge and skills taught in the second require mastering those taught in the first.

The Fueling the Female Athlete module:

  • Squad leaders will deliver the sports nutrition curriculum as outlined in the workbook and explain how to use the nutrition guide for athletes to monitor diet.
  • Squad leaders will present materials on how nutrition affects athletic performance
  • Athletes will classify various foods according to carbohydrate, fat, and protein content
  • Squad leaders will engage team in role-playing activities in which one athlete shares their hypothetical poor diet choices and the other offers changes to better fuel their body.
  • Squad leaders will engage athletes in a team project to come up with some nutritious pre- and post-game meals and snacks

The Positive Body Image module:

  • Athletes work as a team to discuss and critique magazine advertising and other media influences on body image
  • Athletes create a vision board with messaging of what positive body image means to them.
  • Vision boards will be presented to the group by each athlete and then publicly showcased in women’s locker rooms for all female athletes to see.

The Training Component:

Team coaches and squad leaders will acquire the necessary skills through training developed by SANE program experts. Training will consist of one 2-hour training and orientation session in which they will go over the workbook/manual with scripts for all eight sessions.  Training is to be done before the start of each team’s sports season giving ample time before first session begins in case any additional consultations or support is needed. All necessary program materials will be provided at the orientation session.

The training will consist of:

  • Having one coach and at least one squad leader (depending on size of team) present
  • Outline the role of coaches (timekeepers and facilitators) and squad leaders (teachers)
  • Coverage of main themes for each session and overview of the workbook
  • Strategies to get team involved and interact during sessions
  • A role-play and 15-minute mock teaching session

 

Potential Process – Evaluation Questions for SANE

Fidelity

Were all environmental, curriculum, and training components completed as outlined in the complete and acceptable deliveries?

Dose Delivered

Did squad leaders deliver all 8 session plans in a way that was engaging?

Dose Received

Did the athletes enjoy each session of the SANE curriculum and activities?

Did athletes make any lifestyle changes based on the curriculum learned?

Reach

What percentage of athletes attended all 8 sessions, 6 sessions, or 4 sessions?

To what extent did coaches and squad leaders participate in the orientation/training?

Recruitment

What procedures were followed to recruit coaches and choose squad leaders for each team?

Context

What were the barriers to maintaining the involvement of the female high school athletes in the SANE program?

What factors such as environmental, community, or financial that interfered with the implementation of the SANE program curriculum?

Process Evaluation Methods

The process evaluation for the SANE program will be both formative and summative. Timely data collection will need to take place to acquire needed data in order to make appropriate changes to the program if necessary. SANE program experts will be in charge of collecting both formative and summative data from coaches, squad leaders, athletes, and any other necessary parties.

Potential process-evaluation methods for the SANE program curriculum are summarized below:

Fidelity

Data sources and methods include reports from coaches, squad leaders, and SANE program experts. Checklists will be given, and each party is required to check off when a component is complete. There will also be an area next to each checkbox for notes and feedback.

Dose delivered

Possible data sources and methods include reports from coaches who are present at each session. Coaches will have a checklist and a grading rubric to rate each session in terms of effectiveness.

Dose received

Possible data sources include athletes and coaches observations. Methods and tools include administering brief satisfaction scales and conducting focus groups with open-ended questions.

Reach

Possible data sources include SANE program experts as well as coaches. Methods would include an attendance sheet provided to coaches and SANE program experts in order to get a headcount for the training as well as sessions.

Recruitment

SANE program experts document all activities involved in recruiting coaches for the SANE program training and have interviews with coaches on how they chose the squad leaders.

Context 

Possible data sources include athletes, squad leaders and coaches. The main method and tool are interviews and focus groups with open-ended questions to assess barriers.

Program Resources, Context, and Characteristics

The size of the SANE program is large making the evaluation process more complex. Each school has between 300-400 female athletes with about 20 different sports in a school year. The SANE program foresees there being a potential respondent burden given that there are 8 sessions of the program to be given in a 16-week period along with extra time put in for evaluation questions, focus groups, and interviews.  SANE program experts want to minimize the burden placed on the athletes and coaches while still being able to collect accurate data without intruding in on practice and game preparations.

Final Process Evaluation Plan

After analyzing all process evaluation questions that the program planners would ideally ask and would help to improve the implementation and future program, the SANE team has decided to focus on fidelity (program components delivered), dose received (enjoyment of curriculum and any lifestyle changes made), reach (athlete attendance to each session and coach/squad leader attendance to training) and context (barriers to implementation).  By obtaining the data from these questions the SANE team will be able to improve the program for future use as well as limit the burden of coaches and athletes. As stated before, the SANE program understands the importance of the curriculum taught but also understands the athletes need to focus on practice and game preparations throughout the season as well. Most of the methods used to obtain data will come from questionnaires or attendance sheets and a few focus groups.

Information on the program’s fidelity will be discovered by asking the question, “Were all environmental, curriculum, and training components completed as outlined in the complete and acceptable deliveries?” Coaches, squad leaders and SANE program experts will be given a checklist for them to cross of the components of the program that they have delivered. Enjoyment and involvement in the program (dose received) is important to the SANE program, therefore questions to obtain this data will be implemented by administering a brief satisfaction scales and conducting focus groups with open-ended questions. These questions include “Did the athletes enjoy each session of the SANE curriculum and activities?” and “Did athletes make any lifestyle changes based on curriculum learned?” The program also needs to understand how many athletes are attending each session along with the barriers to maintain interest and involvement. The questions asked will be “What percentage of athletes attended all 8 sessions, 6 session or 4 sessions?” and “What were the barriers to maintaining involvement of the female high school athletes in the SANE program?” Data collected here will be obtained through a low participant burden attendance sheet done by the coaches along with a short questionnaire and focus group interviews.

***Reflection*** The programs evaluation is important in order to adjust the program during, as well as after, in order to improve upon the program for the future. Coming up with well thought out questions to guide the evaluation was important and I learned that getting as specific as possible with each question will strengthen the evaluation process. Finding appropriate methods to answer the questions was also important in this step.