Initial Assessment - Proof of Concept
MHI ran our first Proof of Concept in the Northern region of Ghana from April to July 2023. This provided the opportunity for MHI to test our training model in practice, learn lessons for the implementation of a larger-scale pilot later in the year, and get a preliminary understanding of our potential impact.
Written by Jemima Jones on behalf of MHI.
Overview
We’re thrilled to have passed such an important milestone in our journey to maximise the quality of information and access to family planning women receive in the post-birth period. Improving family planning services requires a deep understanding of local healthcare practices and logistical possibilities – an understanding that cannot be gained by looking at published data alone.
Our first proof of concept project has been of invaluable help in understanding how family planning information is provided in northern Ghana and allows us to better tailor our training resources. We wish to extend special thanks to our local partner Savana Signatures, who implemented the majority of our on-the-ground work.
An overview of the work we did during Proof of Concept (POC) 1. We gathered 274 data samples, including 172 surveys and interviews.
Key Terms
- Clients: postpartum women attending health facilities
- Providers: midwives, staff nurses and community health nurses who provide women with family planning counselling.
- Supervising nurse: The most senior nurse in a department.
- Baseline: First round of surveys and interviews held in April.
- Endline: Second round of surveys and interviews held in May.
- Northern region: the second largest of the sixteen regions of Ghana. (See picture)
Understanding where we work through facility visits and interviews
In order to better understand how women receive family planning counselling in northern Ghana, we commissioned our partner Savana Signatures to conduct qualitative work in northern Ghana, including observations of sessions, data collection, and private qualitative interviews on our behalf.
- We heard from 36 local women and 24 staff members about the family planning counselling they receive or give during antenatal care and child immunisation appointments.
- We also observed family planning counselling sessions and gathered data from health facilities on the numbers of women provided with family planning overall and after postnatal counselling, and the number of postnatal family counselling sessions.
Themes we investigated included existing levels of contraceptive use, the prevalence of infertility myths, barriers to contraceptive use and whether women had the chance to ask questions during counselling.
Through this we gained a more informed understanding of what information to highlight and sideline during our training session, and the many questions organically asked about family planning (10/35 participants expressed a desire to know more about it) increased our confidence in the need for improving existing family planning counselling.
How this first stage informed our work
- Through analysis of data gathered we have a much deeper and up-to-date understanding of local context. This includes the percentage of women who had ever used contraception (~60-70%), the low prevalence of individual counselling, and the existing tools used by providers (overwhelmingly flipcharts).
- To ensure fuller and more reliable data in future, we moved to offline digital forms after quickly discovering our previous system, giving surveyors a spreadsheet to fill in, led to problems. Whilst the offline spreadsheet format did avoid the anticipated connection issues, missed questions were a surprisingly large issue. We now use SurveyCTO, where not only are questions unskippable, but there are many other functionalities that help us monitor the reliability of our data.
- After gathering data from facilities on the number of visitors they receive, we decided to shift from focusing on health centres and hospitals to solely focusing on hospitals. One of the two hospitals in our initial sample of six facilities received more visitors per month than any of the other facilities received in an entire year. We believe we can help many more women because of this shift.
Putting our training programme into action
Aunty Theodora, Regional Public Health Nurse from the Ghana Health Service and an MHI training facilitator (far left) with Ben from MHI and the participants at the end of one of our training session.
On the 26th and 28th April, MHI ran two training programmes, designed by MHI and led by facilitators from the Ghana Health Service. Each training programme consisted of four sections, with each section including a lecture and role-play in pairs. The goal of this training is to improve the quality of information shared by providers of family planning counselling, so women can make informed decisions and go on to have happier and healthier futures.
Trainings included information such as the benefits of family planning and how to respond to common concerns and misconceptions, principles of effective counselling, and information on how to conduct a session using a five-step counselling approach.
Before the session, there was an anonymous knowledge test, and we repeated the test both immediately after training and one-month later. Through the pre-training test, we found that existing knowledge levels were higher than we had been expecting – the average score was 79%!
Despite this, there was still room for our training to result in high changes – with absolute changes of over +10% in 11 questions, including two questions where absolute correct answers increased by over 30%.
Overall however, there wasn’t much room for improvement – questions that providers had answered incorrectly in the initial test were often those that were particularly difficult to remember. For example, some birth control methods can only be used at set points postpartum, and remembering which method at which point is comparatively tricky. For this reason, one of the tools we provided to participants at the session was a reference chart on this to use during counselling for this reason.
In the test immediately post-training, the average score had increased 3%, reaching 82%. Alternatively, it can be expressed as a 5% increase to 84% if you don’t include misinformation that was unfortunately shared by an attendee of the session and lowered the average answer to one question by 55 percentage points.
An additional benefit of running two training sessions was spending large amounts of time with our local partners and meeting two dozen healthcare providers. Through listening and talking to them, we learnt that facility buy-in was even more important than we had thought, and strengthened our view that individual counselling is uncommon.
Changes we made as a result of the proof of concept training
In response to the high levels of existing knowledge amongst the providers we trained, MHI have made two changes to our future training programmes:
- The training will focus on a specific model of counselling rather than facts about family planning. In practice, this will centre around ensuring providers consistently follow important steps like asking women if they have any concerns about using family planning methods, rather than testing them on how soon after birth IUDs can be inserted.
- Instead of training staff from family planning units, we are shifting our focus exclusively to frontline postnatal and child vaccination staff, who have lower knowledge levels. This will also better integrate our services within existing systems.
Participants in MHI’s training reviewing the World Health Organisation’s Medical Eligibility Criteria wheel, a tool integrated into our trainings to ensure providers consistently provide medically safe and correct guidance.
One-month follow up
Tests
For our training to have an impact, it has to be remembered! One month after the training sessions, Savana ran surveys and interviews on our behalf, to help us understand the effectiveness of our training programme and gather additional information on local family planning services. Here’s some of what we did:
- We quizzed fourteen of the original training session participants to test retention and surveyed them on the usefulness of the tools we provided, like the cue cards.
- Savana observed four family planning information sessions given by the healthcare workers who received our training, so that we could see the extent to which learnings were put into action.
We found that increases observed from the pre-training test to the post-training test were largely retained. Among questions where the average correct answer percentage increased, the average absolute change between the post-training test and the one month follow-up test was only -1.09%!
Counselling in practice
During our training sessions, we provided participants with two tools to improve the quality of counselling women receive. The first was a cue card with a suggested structure for family planning counselling, to ensure that women’s autonomy is fully respected through the processes. Sections in the structure include asking women about understanding the client’s needs and concerns. The second tool was a reference chart showing pictorially which family methods are safe at different points after giving birth.
We asked providers we trained about our tools at a one month follow-up survey. The cue card and reference chart were ranked as 4/5 or 5/5 for usefulness and upskilling power 100% of the time. For each of these, the percentage of people answering 5/5 ranged from 50% – 71%.
Does this reflect actual usage? In the four family-planning sessions our partners observed, one month after the tools were given out, the cue card was used in 75% of sessions and the reference chart was used in 100%.
Facility data evaluation
Has our training influenced contraceptive uptake rates in facilities where staff members underwent our training? This is the key question about the impact of our programming and one we are working hard to answer.
The Ghana Health Service gathers quarterly data on contraceptive uptake at health facilities across the country. In partnership with the Northern Region Health Directorate, MHI is working to track changes to care in the three months since our training. This will allow us to assess changes in contraceptive uptake at the six facilities from which we invited healthcare workers to the training.
At the time of writing, this data is still to be finalised but we will update this article once we have received it.
Programme design changes
Based on the proof of concept work, we've made two major changes for our future work
First, we’ve split our programming into two models. Survey data and conversations on the ground from POC 1 suggest that rates of individual counselling are lower than we expected from our engagements with other stakeholders. However, research from other countries suggests that individual counselling substantially increases family planning uptake, and this was recently corroborated by a study performed at two major hospitals in Accra, where shifting from group to individual counselling in PNC increased postpartum contraceptive uptake from 12.6% to 51% (Asah-Opoku et al 2023).
Our other programming arm will target child welfare clinic sessions specifically. These occur later in the postpartum period when the natural contraceptive benefits of breastfeeding begin to fall away and the use of family planning therefore becomes particularly salient and important. This avenue of work is based of the program implemented by Dulli et al. (2016)
Second, we’ve put an emphasis on increasing the accountability systems that we provide. We have seen how time-pressed providers are and how it is easy for best practices to fall through the cracks. We are offering small financial incentives to providers who follow best practices, selecting ‘Birth Spacing Champions’ for each facility, and increasing the rigour of our pre-post monitoring systems.
These changes are in addition to those already discussed: promoting a specific model of family planning counselling in our training sessions, and exclusively training frontline staff. Depending on our level of success with this programming, we anticipate beginning to scale one or both models across additional regions of Ghana towards the end of 2023.
Counselling in practice
During our training sessions, we provided participants with two tools to improve the quality of counselling women receive. The first was a cue card with a suggested structure for family planning counselling, to ensure that women’s autonomy is fully respected through the processes. Sections in the structure include asking women about understanding the client’s needs and concerns. The second tool was a reference chart showing pictorially which family methods are safe at different points after giving birth.
We asked providers we trained about our tools at a one month follow-up survey. The cue card and reference chart were ranked as 4/5 or 5/5 for usefulness and upskilling power 100% of the time. For each of these, the percentage of people answering 5/5 ranged from 50% – 71%.
Does this reflect actual usage? In the four family-planning sessions our partners observed, one month after the tools were given out, the cue card was used in 75% of sessions and the reference chart was used in 100%.
Facility data evaluation
Has our training influenced contraceptive uptake rates in facilities where staff members underwent our training? This is the key question about the impact of our programming and one we are working hard to answer.
The Ghana Health Service gathers quarterly data on contraceptive uptake at health facilities across the country. In partnership with the Northern Region Health Directorate, MHI is working to track changes to care in the three months since our training. This will allow us to assess changes in contraceptive uptake at the six facilities from which we invited healthcare workers to the training.
At the time of writing, this data is still to be finalised but we will update this article once we have received it.
Wrapping up
Our first Proof of Concept was invaluable. It validated key aspects of our theory of change, providing evidence that participants can retain learning from training sessions for meaningful periods and that providers are willing to adopt new materials and approaches for how they deliver care. It also affirmed our fundamental assumptions, such as there being a significant unmet need for family planning amongst our target communities.
Finally, POC 1 deepened our understanding of the local context, and revealed opportunities to iterate on our model and better improve the lives of the women we hope to serve.
Inspired by MHI's work?
Please follow our work through our newsletter, reach out to us directly through our contact form, or consider donating to help us expand our work. Thank you for your support.
