Process Evaluation

COMPLETE AND ACCEPTABLE DELIVERY OF MATERNAL INSTINCTS

The ideally implemented Maternal Instincts program will consist of three components: an environmental component focusing on creating an intervention team; two curriculum models, “Eating for a Healthy Baby” and “How Can I Afford this Food?;” and the peer support and training component. 

The Environmental Component

An intervention team will be assembled. This will include volunteers, Maternal Instincts planners and partners as well as additional necessary staff (paid staff, tech support, statistical analysts, creators-for brochures and media). The primary team will ideally be made up of mothers and people from within the surrounding community. The local stores and doctors offices will need to be on board with the program. There will need to be use of local libraries, schools, or VFWs for meetings and space for the program. There will also be a need for a kitchen space.

The Curriculum Component

The Maternal Instincts program is comprised of two models- “Eating for a Healthy Baby” and “How Can I Afford this Food?.” “Eating for a Healthy Baby” is delivered in person. It will consist of two meetings, over the span of two weeks. The first meeting will be around 2 hours long with childcare as an option for moms with additional children. This first meeting will focus on what pregnant women need to have a healthy pregnancy- nutritional needs and how to meet them. It will be two parts with a short break in between. The second meeting will focus on recipes using the information learned in the last class and safe exercises for the pregnant mother. The informational portion of the second meeting will be shorter, so there will be plenty of time at the end for questions and connecting with other members of the community. These meetings are designed to have active participation in order to establish new learned information and stimulate discussion based on knowledge learned. The optional third meeting will be hands-on and the participants can take home free food. They will prepare a healthy dinner step-by-step with an instructor. They will be able to take home the final product.

“How Can I Afford this Food?” will be delivered via social media. This can take many forms and can be adjusted based on what fits best for certain participants (ie. Facebook, email, text message, etc). The curriculum will focus on how to eat on a budget and it will give some tips and tricks to do so. There will also be printable resources like meal planning charts and grocery lists. There will need to be local shopping centers, farmers markets and resources (aide, free vitamins, SNAP, etc.) established. 

The two curriculums are designed to be given in sequential order. The participants will learn how to shop (in the second curriculum) for the healthy foods they learned about (and possibly prepared) in the first curriculum. 

Peer Support and Training Component

Maternal Instincts will require the use of peer support from the community. These community members will be volunteers, but they will receive compensation if they complete all of their tasks. There will be translators, transportation aides, and mothers who want to help future/new moms. The translators will need to be fluent in Spanish and English and trained on the content of the program and what their position will entail. The transportation aides will need to be interviewed (ie. for driving safety- partner with local police department). The moms will be peer support regarding whatever the pregnant mothers need, but also be trained in order to provide the participants with safety and companionship attending doctors appointments. All volunteers will be trained before the start of the program.

POTENTIAL PROCESS-EVALUATION QUESTIONS FOR MATERNAL INSTINCTS

Through a series of meetings, program planners developed a list of potential process-evaluation questions that included the following:

Fidelity

  • Was peer support implemented as planned?
  • Was transportation implemented as planned?
  • Were the instructors educating the intended curriculum as planned?

Dose Delivered

  • Did peer support aides contribute as planned?
  • Are translators attending appointments with participants? 
  • Are transportation aides picking up participants as intended?
  • Did instructors deliver all units of the curriculum?

Dose Received

  • Did the participants enjoy the education curriculums?
  • Were the instructors satisfied with the curriculums?
  • Did participants utilize transportation, translators and peer support?
  • Are pregnant mothers in low-income communities safe and healthy?
  • Did pregnant mothers receive nutrition education?
  • Did pregnant mothers receive support systems they can trust?

Reach

  • Were the brochures delivered to at least 70% of local stores and doctors 
  • offices?
  • Did 50% of local pregnant women participate in education classes?
  • To what extent did instructors/volunteers/peer support aides participate in training?

Recruitment 

  • What procedures were followed to recruit trainers and instructors to develop the Maternal Instincts team?
  • What procedures were followed to recruit participants and volunteers to contribute?

Context 

  • Did the libraries/schools/VFWs allow meeting time for Maternal Instincts planning?
  • Did the doctors offices and local stores allow the program implementation?
  • Were there any barriers to implementation in the community?

PROCESS-EVALUATION METHODS

The Maternal Instincts evaluation will be both for formative uses (fine-tuning) and summative uses (info about the extent program was planned and reached). Because of this, the timing of data collection is important. It needs to be ensured that planners receive information back in a timely manner in order to determine if adjustments to the program are needed to be made. Multiple methods to collect data will be used because different conclusions may be determined. Below are some potential process-evaluation methods:

Fidelity

Possible data sources and methods include gathering reports from transportation aides, translators and peer support staff. Also, creating quick surveys with open-ended questions for the participants to take every week.

Dose delivered

Possible data sources and methods include having a log sheet with appointments to be initialed by the doctors offices when the aide is present, having transportation aides enter their mileage before and after a pick-up, and counting brochures before and after delivery. They will also be observation by staff at select appointments and activities.

Dose received 

Possible data sources and methods include having participants and instructors take a survey to determine enjoyment. There will be a focus group with the participants and another one with the instructors to determine satisfaction with the curriculum.

Reach

Reach are the three classes in person and the online class. At the beginning of each in-person class, the teacher could have a head count and a sign-in sheet at the door. The third class only requires sign up, since it is not a mandatory part of the curriculum. For the online class component, it will be determined who participates because it is them online. There will also be a quick quiz activity before each class to determine what they know/learned. 

Recruitment 

All staff will document their activities in a provided notebook. The “who, what, when, where and why” will be listed, as well as conversations of importance.  

Context

Possible data sources and methods include instructors, Maternal Instincts staff and potential partners. There will be meetings to determine proper implementation within the community. Barriers to implementation will also be discussed.

PROGRAM RESOURCES, CONTEXT AND CHARACTERISTICS

Maternal Instincts program has a long preparation piece and requires many resources including time. There are many volunteers to be monitored and evaluated as the program is unfolding. There are many interviews that will need to be conducted to determine who will take it seriously and who is up for the job. There are childcare volunteers, an instructor for “Eating for a Healthy Baby,” another potential instructor to prepare the take-home recipe, the creators of “How Can I Afford this Food?,” tech support, statistical analysts, creators-for brochures and media, translators, transportation aides, and the peer support volunteers, and the people to monitor and train all of the staff and volunteers. The amount of volunteers and the need for volunteers depends on how many participants sign on. There can be one translator and one transporter to every 10 participants. The scheduling will need to be determined by a staff member, not the volunteers themselves. There will be 2:1 ratio for peer support to participants and that can increase based on how much the volunteer wants to help their fellow mothers in their community. There will be need for a meeting space for the three in-person classes. Data collection tools will also be necessary.

THE FINAL PROCESS EVALUATION PLAN

After considering all the potential process questions that the program planners would ask as well as the complexity of the program with limited resources, the team decided to focus on fidelity, dose delivered, dose received, reach and context. The team thinks this will provide the most insight using the least amount of resources. The general curriculum of this program remains the same since it is to be implemented in low-income communities, while some must be adjusted based on location and culture. For instance, some low-income communities may have the primary language of Spanish, while others may have more than one, which requires multiple translators. Training instructors and determining ability of volunteers is critical to the success of this program. There are also many budget constraints due to the locations in which the programs are being implemented. Next, is a short summary of the process evaluation plan.

Fidelity will be measured by the question: Were the instructors educating the intended curriculum as planned? This will be determined by the use of open-ended surveys and self-reporting from volunteers. Dose delivered will be measured by the question: Did instructors deliver all units of the curriculum? The curriculum is of the most importance to enhance knowledge of pregnant mothers in the low-income communities. This will be measured by self-reporting of the instructors and by observation. Dose received will be measured by the question: Did the participants enjoy the education curriculums? This will be determined by participant surveys at the end of the in-person course and the focus group at the end of the curriculum. Reach will be measured by the question: Did 50% of local pregnant women participate in education classes? If the women are not being reached in their own communities, then the program is not successful. This will be measured by headcount logs and instructor reporting. Finally, context will be measured by the questions: Were there any barriers to implementation in the community? Were they successfully overcome? There will be barriers based on culture and differing communities and it is important for the group to overcome these prior to and during implementation of the program.