Evaluation of an MNCH Mentoring Intervention in Karnataka

Monitoring and Evaluation of an MNCH Mentoring Intervention in Two Districts of Karnataka

Our hypothesis is that an on-site mentoring program for staff at primary health care centers in northern Karnataka, will improve the quality of maternal, neonatal and child health care, and improve clinical outcomes.

Study Overview

• Status: Completed
• Conditions: Maternal-child Health Services
• Intervention / Treatment: Other: Mentoring intervention

Detailed Description

By providing on-site mentoring for improved clinical care and service delivery, we hypothesize that the quality of services and care will improve and patients will have better clinical outcomes. Specifically we believe that routine on-site clinical and service delivery mentoring will strengthen clinical knowledge, skills and practices among staff nurses and ANMs (in year 2), will strengthen appropriate referrals and follow up care, will improve other aspects of service quality, and provider and client satisfaction. The evaluation will assess change during the first year of the intervention that focuses on clinical mentoring of staff nurses and on quality improvement in the PHCs as a whole. The evaluation will focus on success indicators as shown in Appendix 1. Due to the short time frame for evaluation, success indicators will be principally at the levels of inputs, outputs, and outcomes focusing on adherence to the Sukshema Critical Care package, although we also hope to assess some key health outcomes, in terms of reduction of "near-miss" cases.

A. Pre and post MM intervention evaluation strategy The "pre-intervention" evaluation will have two phases. The first phase will take place in all 122 sites before any intervention activities begin (Figure 1). This evaluation will focus on site readiness to provide quality services, provider knowledge, attitudes and reported or observed (or simulated) practices, supervision and mentoring, record keeping, and provider and client satisfaction. The NRHM SBA guidelines, PHC 24/7 guidelines, MHFW NSSK guidelines, and the Sukshema critical care package will all be used to determine expectations of clinical quality of care at this level.

The focus of the first year of the MM intervention is enhanced clinical skills in the intra-partum and postpartum periods and thus assessments of actual clinical practices are important. However, data on actual practices in the intrapartum and postpartum periods will be difficult to collect by observation during this baseline (especially those with complications). Most of the PHCs conduct only a few deliveries each week, and it will be therefore almost impossible to obtain a representative sample of observations. Therefore we will assess intrapartum and postpartum clinical practices largely though the audit of case sheets and registers. However, these case sheets are currently either not available or not used. Thus our strategy necessitates delay of this part of the second phase of the baseline until after the update training where all staff will be given and instructed in the use of new and improved case sheets. All sites will be given the opportunity to practice using these patient case sheets for one month before they are collected by project staff and used as a basis for assessing both compliance with the new case sheets, adequacy of reporting, and actual case management. Actual case observations will be done during routine MM visits where possible. The post intervention phase of the assessment will take place in all sites once year after the first (planned 6) MM visits and will repeat the baseline survey.

Pre and post MM intervention evaluation survey tools

The survey will use four types of survey tools pre and post MM intervention and one additional tool after one year in the intervention sites only:

A1. Questionnaires for all staff nurses on duty (including night duty) at the PHCs during the evaluation that will assess knowledge of intrapartum, postpartum, newborn care and infection prevention, and solicit provider perspectives on mentoring and supervision. In some cases, the questions will be in the form of case studies or asking for simulated demonstrations of practices.

A2. Facility audits and observations in all PHCs to assess site readiness to provide quality services and provide checks of drugs used: equipment and drugs, staffing and training levels, availability and correct use of case sheets, registers and referral slips. This tool will also have a section for observing practices in the postnatal ward.

A3. Postpartum client questionnaires with all women who have delivered up to one month before the survey traced to their homes that will assess client recall of information and services given and client satisfaction. Delivery registers will be checked and all women who delivered in the PHC n the previous month will be traced to their homes for interview.

A4. Case sheet and register audits for all cases for the one month before survey dates to assess completeness of reporting and appropriate client management A5. Focus Group Discussions with SNs in a sample of intervention PHCs: During the post MM intervention survey we will conduct focus group discussions (FGDs) with staff at a selection of intervention sites to understand their experiences with the MM intervention and to gather their suggestions for improvement.

Data collection: The field staff will consist of three supervisors per district, plus 13 teams of 2 interviewers, one of whom will manage the facility audits and staff surveys, and one the postpartum client questionnaires for each of the 122 PHCs. It will be necessary for each team to spend 2 days in each PHC, thus each team will be able to complete the survey work in 3-4 weeks. Simultaneous data collection will be carried out in both districts.

A week's training will be provided to the field teams that will include lectures on technical topics, demonstration interviews, mock interviews and field practice. The training will be provided to about 10% more persons than actually required for the survey, and the required number will be selected based on their performance during the training. KHPT and SJMC staff will conduct the training and will monitor the field work activities and quality of data.

Data processing: Double data entry will be done at KHPT, who will prepare a report of the pre-intervention survey and end-line survey comparison.

B. Routine continuous monitoring: First, project staff will routinely document staffing, training, financial and other project inputs, data that will be crucial for assessing the feasibility of scale up to other districts. During bi-monthly MM visits, staff will have a proscribed list of data to collect. As well as keeping details of staff at the sites their training background and any clinical mentoring given, the mentors will have a proscribed list of clinical skills to observe; wherever possible during their 3 day visits, they will observe women in labour, after delivery and in the postpartum ward, and where this is not possible, they will use models and case studies to assess staff skills. There will be continuous monitoring of the MM interventions through reports of MM visits that will include: documentation of visits through trip reports that detail facility meetings, talks given, happenings in the PHC or area and notes on any morbidity and mortality discussions; completed clinical competency checklists for mentored individuals. They will also photocopy the case sheets, which will have been used as a basis for discussion during the visit and facility action plans, for review by KHPT clinical staff. The case sheets will also be collected quarterly from the control sites by the KHPT district staff. These staff will encourage use of the case sheets at each of their quarterly visits without any additional training or mentoring.

Informed consent Verbal, witnessed informed consent will be sought from the head of the PHC who will be asked to give permission to survey the PHC; from staff nurses who will be interviewed in the PHCs and from postnatal women traced to the community.

Districts and sites Using a matched (for client volume) case control strategy, the MM intervention will be piloted and evaluated in PHCs in two districts, Gulbarga and Bellary that have a total of 122 24x7 PHCs (Gulbarga 70; Bellary 52). Sixty one PHCs (Gulbarga 35; Bellary 26) will be randomly assigned to an MM intervention or experimental group (stratified by taluka and 6-monthly patient delivery load), while the rest will serve as control sites.

SBA + training The first part of the intervention involves all the 122 PHC 24/7 facilities in the two districts for two reasons: first we feel that this training is generally needed and will help to put the intervention and control site staff on the same level playing field in terms of basic MNCH knowledge; second, we know from the Sukshema baseline survey conducted in 2011 that few staff use adequate reporting mechanisms such as case sheets. As these are needed both programmatically as job aids, and for programme evaluative purposes, we will use this training as an opportunity to introduce new case sheets. After the first phase of baseline survey (described below) and before the mentoring programme begins in the intervention sites, Sukshema project staff will conduct a 3 day training update (called SBA+) for all staff nurses and a one day SBA+ for all M.O.'s from all 122 PHCs in the two districts, irrespective of whether they have previously attended the NRHM Skilled Birth Attendant (SBA) training. The 3 day training will focus on the essential components on intra-partum and postpartum (mother and baby) care identified in the Sukshema critical care package that evidence shows can significantly reduce maternal and newborn morbidity and mortality. The one day training for doctors will focus on identification of danger signs, pre-referral management of complications, quality of care, case audits and improvement of management and documentation. At these trainings, staff will be given, and instructed in the use of enhanced case sheets for women and babies from labour through to discharge, including use of the partograph. After this training, there will be no further interventions in the control sites, except that Sukshema staff will visit every 3 months to collect copies of the case sheets.

Sampling strategy The PHC intervention and control sites will be selected systematically in each taluka, after stratification and matching by client volume per month.

MNCH mentoring In each district MNCH mentors (MMs), a new staff cadre, will be recruited and trained (Gulbarga 5; Bellary 4), each responsible for the intervention in 6-7 experimental/intervention PHCs. We will also train 3 additional MMs to act as a back-up (2 in Gulbarga and one in Bellary), total 12 mentors. The MMs will attend some of the PHC staff training but will also be trained for 4 weeks by project staff at St John's Medical College in essential clinical competencies and in how to mentor staff in clinical skills, team building and problem solving, service delivery improvement, as well as in how to be effective mentors. The MNCH mentors will be senior nurses, selected based on specific criteria such as clinical experience, training and mentoring abilities, and experience/familiarity with government health systems. They will be guided by a Mentoring Coordinator in each district who will support the mentors' activities and liaise between them, the PHCs, district government staff and KHPT district staff. During the first year, the mentors will be trained by the Project to (a) provide clinical mentoring in safe delivery to staff nurses by hands-on coaching; (b) provide mentoring in how to improve the quality of services by building teamwork and introducing problem solving initiatives to address other aspects of service delivery (including especially referral processes); and in year 2 to mentor ANMs who will come to the PHC during the MNCH mentor visits, in providing antenatal, postpartum and newborn care, and referrals and follow up.

• Study Type: Interventional
• Enrollment (Actual): 295
• Phase: Not Applicable

Contacts and Locations

Study Locations

• India
  • Karnataka
    • Bangalore, Karnataka, India
      • Bellary and Gulbarga districts

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 18 years and older (ADULT, OLDER_ADULT)
• Accepts Healthy Volunteers: No
• Genders Eligible for Study: All

Description

Inclusion Criteria:

• All physicians and staff nurses in PHCs in the two districts.

Exclusion Criteria:

• Physicians and staff nurses in PHCs who do not consent to participate.

Study Plan

How is the study designed?

Design Details

• Primary Purpose: HEALTH_SERVICES_RESEARCH
• Allocation: RANDOMIZED
• Interventional Model: PARALLEL
• Masking: NONE

Number of Arms

2

Arms and Interventions

Participant Group / Arm	Intervention / Treatment
EXPERIMENTAL: Mentoring intervention; The mentoring intervention involves periodic supportive visits and education to health center staff by nurse mentors.	Other: Mentoring intervention; The mentoring intervention involves periodic supportive visits and education to health center staff by nurse mentors.
NO_INTERVENTION: No mentoring intervention; The no mentoring intervention arm does not have supportive visits and education to health center staff by nurse mentors.

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Changes in provider knowledge and performance from baseline on intrapartum, postpartum and newborn care	Outcome measures with respect to knowledge of intrapartum care include, among others: % providers who know how to do AMTSL; % providers who know how to use a partograph; % providers who know what are danger signs needing referral; % providers who know to identify and manage PPH; % providers who know to identify and manage obstructed labour; % providers who know to identify and manage fetal distress. There are also several indicators that will be measured with respect to provider performance during labour, such as % labours with fully completed essential general history taking documentation. Outcome measures with respect to knowledge of and performance in postpartum and newborn care include, among others: % providers who know what to monitor in the mother in the immediate PP period; % postpartum women staying in the facility for at least 12 hours and for 2 days or more; % postpartum women properly monitored 2 and 4 hourly; % PHC clients who were given FP counselling.	12-24 months

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Change from baseline in quality of care provided at primary health care centres	Outcome measures with respect to quality of care at primary health care centres include: % PHCs having action plans; % PHCs having all needed drugs; % PHCs having all needed equipment and supplies for delivery; % PHCs having all needed equipment and supplies for infection prevention; % PHCs having all needed equipment on emergency tray in labour room; % PHCs that have a complete newborn corner; % PHCs that have all needed IEC and learning materials; % PHCs that have a list of referral facilities/displayed referral chart; % PHCs that have a functioning feedback mechanism for women referred to FRU; % PHCs that follow correct steps in infection control; % PHCs that have an M&M audit meeting at least once per quarter with notes recorded.	12-24 months

Collaborators and Investigators

• Sponsor: Karnataka Health Promotion Trust
• Collaborators: University of Manitoba; Bill and Melinda Gates Foundation
• Investigators: Principal Investigator: Ramesh Banadakoppa, PhD, Karnataka Health Promotion Trust

Publications and helpful links

General Publications

• Bradley J, Jayanna K, Shaw S, Cunningham T, Fischer E, Mony P, Ramesh BM, Moses S, Avery L, Crockett M, Blanchard JF. Improving the knowledge of labour and delivery nurses in India: a randomized controlled trial of mentoring and case sheets in primary care centres. BMC Health Serv Res. 2017 Jan 7;17(1):14. doi: 10.1186/s12913-016-1933-1.
• Jayanna K, Bradley J, Mony P, Cunningham T, Washington M, Bhat S, Rao S, Thomas A, S R, Kar A, N S, B M R, H L M, Fischer E, Crockett M, Blanchard J, Moses S, Avery L. Effectiveness of Onsite Nurse Mentoring in Improving Quality of Institutional Births in the Primary Health Centres of High Priority Districts of Karnataka, South India: A Cluster Randomized Trial. PLoS One. 2016 Sep 22;11(9):e0161957. doi: 10.1371/journal.pone.0161957. eCollection 2016.

Helpful Links

• Website of Karnataka Health Promotion Trust

Study record dates

Study Major Dates

• Study Start: September 1, 2012
• Primary Completion (ACTUAL): September 1, 2015
• Study Completion (ACTUAL): September 1, 2015

Study Registration Dates

• First Submitted: November 27, 2013
• First Submitted That Met QC Criteria: December 3, 2013
• First Posted (ESTIMATE): December 9, 2013

Study Record Updates

• Last Update Posted (ESTIMATE): November 28, 2016
• Last Update Submitted That Met QC Criteria: November 25, 2016
• Last Verified: November 1, 2016

More Information

Terms related to this study

• Keywords: child; India; neonatal; health services; maternal; health care; Karnataka
• Other Study ID Numbers: 38/2012

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: YES
• IPD Plan Description: Peer-reviewed publications and end-of-project workshop held in February 2016