Assessing the reliability of phone surveys to measure reproductive, maternal and child health knowledge among pregnant women in rural India: a feasibility study

Abstract

Objectives: Efforts to understand the factors influencing the uptake of reproductive, maternal, newborn, child health and nutrition (RMNCH&N) services in high disease burden low-resource settings have often focused on face-to-face surveys or direct observations of service delivery. Increasing access to mobile phones has led to growing interest in phone surveys as a rapid, low-cost alternatives to face-to-face surveys. We assess determinants of RMNCH&N knowledge among pregnant women with access to phones and examine the reliability of alternative modalities of survey delivery.

Participants: Women 5-7 months pregnant with access to a phone.

Setting: Four districts of Madhya Pradesh, India.

Design: Cross-sectional surveys administered face-to-face and within 2 weeks, the same surveys were repeated among two random subsamples of the original sample: face-to-face (n=205) and caller-attended telephone interviews (n=375). Bivariate analyses, multivariable linear regression, and prevalence and bias-adjusted kappa scores are presented.

Results: Knowledge scores were low across domains: 52% for maternal nutrition and pregnancy danger signs, 58% for family planning, 47% for essential newborn care, 56% infant and young child feeding, and 58% for infant and young child care. Higher knowledge (≥1 composite score) was associated with older age; higher levels of education and literacy; living in a nuclear family; primary health decision-making; greater attendance in antenatal care and satisfaction with accredited social health activist services. Survey questions had low inter-rater and intermodal reliability (kappa<0.70) with a few exceptions. Questions with the lowest reliability included true/false questions and those with unprompted, multiple response options. Reliability may have been hampered by the sensitivity of the content, lack of privacy, enumerators' and respondents' profile differences, rapport, social desirability bias, and/or enumerator's ability to adequately convey concepts or probe.

Conclusions: Phone surveys are a reliable modality for generating population-level estimates data about pregnant women's knowledge, however, should not be used for individual-level tracking.

Trial registration number: NCT03576157.

Keywords: community child health; maternal medicine; neonatal intensive & critical care.

Conflict of interest statement

Competing interests: All authors have completed the Unified Competing Interest form (available on request from the corresponding author) and declare that the research reported was funded by the Bill and Melinda Gates Foundation. AG, SC, PD are employed by BBC Media Action, one of the entities supporting programme implementation. The authors do not have other relationships and are not engaged in activities that could appear to have influenced the submitted work.

© Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY. Published by BMJ.

Figures

Figure 1

Completed interviews for face-to-face and CATI surveys. CATI, caller-attended telephone interview.

Figure 2

Conceptual framework for identifying the determinants of pregnant women’s RMNCH&N knowledge. ANC, antenatal care; ASHA, accredited social health activist; RMNCH&N, reproductive, maternal, newborn, child health and nutrition.

Figure 3

Measurement of degree in which repeated measurements in pregnant women interviewed (test–retest) provide similar answers.

Figure 4

Average knowledge scores for composite knowledge domains.