INTRODUCTION

Nigeria currently has one of the worst maternal, newborn and child health indices in Africa and globally ^1,2. This is partly due to challenges in effective health care service delivery particularly at the primary health centers which serve the rural populations ^3,4.

Major disparities exist between rural and urban populations in terms of poor maternal and newborn child outcomes with the rural populations bearing majority of the burden. This is in part due to the skewed distribution of health care facilities and health workers in favour of the urban zones ^5,6. This inequitable distribution of skilled birth attendants appears much more distinctly in the northern parts of the country and primary health care facilities in rural areas ⁶.

Major challenges identified in the effective operationalization of the primary healthcare in Nigeria towards improved quality of life for members of the communities include unavailability of trained human resources, poor distribution of health workers , poor quality of health-care services , high dropout rates in interventions requiring reasonable degree of continuity in order to attain the required quality coverage, unavailability of essential medicines, poor condition of infrastructure and inadequate equipment ^7,8. The inability of primary health care centers (PHCs) in Nigeria to provide essential obstetric care services which are vital towards reducing maternal and newborn morbidities and mortalities have also been previously reported ^9,10.

The use of innovative digital health technologies has potential to address the above challenges and improve maternal, newborn and child health outcomes in Nigeria. The World Health Organization (WHO) defines digital health technologies as the cost-effective and secure use of routine and innovative forms of ICTs to support health and health-related fields ¹¹. These benefits have not been fully harnessed in Sub Saharan Africa as the use of digital health tools has largely been limited to their application for improving utilization of health care services via reminder text messages ^12-14. There is a dearth of evaluations on innovative approaches and the role of digital health tools in training frontline health workers in remote areas that lack regular telecommunications network connectivity in Nigeria at a large scale. Its use in improving quality of data generated at the PHCs and timely transmission of such data in hard-to-reach areas using satellite devices has also not been evaluated in Nigeria.

The “Extending Nigerian health services to rural population using SatCom (EXTEND) project in Nigeria sought to address logistic and technical challenges of providing care to those hardest to reach (the so-called last mile challenge), by using satellite technology to extend communications infrastructure to rural areas in three states of Nigeria (Federal Capital Territory (FCT), Ondo and Kano). for a period of 24 months (March 2017-March 2019). The study protocol ¹⁵ and overall findings from the main project that involved three states in Nigeria (Kano, Ondo and FCT) have been published elsewhere ¹⁶. This report represents analysis of state level quantitative and qualitative data from the FCT,Abuja. The study aimed to evaluate the impact of the use of digital innovations (satellite connectivity infrastructure, video training apps and CliniPAK) in strengthening primary health care and improving utilization of MNCH services in the Federal Capital Territory by comparing findings of end line assessment with that of baseline.

MATERIALS AND METHODS

Study setting

Nigeria has an estimated population of 206,139,589 people at midyear of 2020 according to UN data with 48.8% of these residing in rural areas. The country occupies a total land area of 910,770 Km^{2 17}. The Federal Capital Territory (FCT) Abuja which is the Nation’s Capital city is located between latitudes 8◦25′ and 9◦ 25′ north of the equator and longitudes 6◦45′ and 7◦ 45′ east of Greenwich. The territory covers an area of 8,000 square kilometres and has an estimated population of 3,564,100 in 2016 ¹⁸. The territory is made up of six area councils which corresponds to the local government areas in other states of the Federation that supervises and funds the PHC facilities as well as have overall responsibility for this level of health care service delivery in Nigeria. The area councils are namely: Abuja Municipal, Kwali, Bwari, Gwagwalada, Kuje and Abaji.

Two clusters corresponding to two area councils in the FCT were selected. These are Gwagwalada (intervention) and Kuje (non-intervention) area councils respectively. The intervention area council had health facilities benefiting from e-Health innovations (VTR, CliniPAK), and the non-intervention area council did not have facilities benefiting from e-Health innovations. The two area councils were selected in close discussion with state policymakers and PHC programme managers as well as based on other criteria that included similarity in maternal mortality rates and existence of effective and committed district health leadership and ministry of health’s plans for scaling up.

Gwagwalada area council had an estimated population of 402,000 in 2016 ¹⁸. The area council is made up of ten ¹⁰wards namely: Gwagwalada central, Zuba, Kutunku, Tungan -maje, Dobi, Paiko, Ibwa, Ikwa and Gwarko. On the other hand, Kuje LGA had an estimated population of 246,400 in 2016¹⁸. It also consists of ten wards namely: Kuje, Kabi, Kujekwa, Chibiri, Kwakwu, Gwargwada, Gudunkarya, Rubochi and Yenche.

Study Design

The study used a mixed-methods design to evaluate the acceptability and effects of novel e-Health tools implementations in the FCT. The quantitative part of the study used a non-randomised cluster trial design, collecting longitudinal data on utilization of MNCH services before (baseline assessment) during (midline) and after (endline assessment) the implementation of eHealth tools in 24 PHCs in intervention (Gwagwalada) and non-intervention (Kuje) area councils. Utilization of MNCHs services were compared in the intervention and non-intervention area councils during the three assessment periods (baseline, midline and endline) as well as with intervention and non-intervention area councils. The facility data was also compared with longitudinal National Health Management Information System (NHMIS) data during these periods in the intervention and non-intervention facilities to understand the impact of the e-Health tools on health systems functions and health outcomes.

The qualitative part of the study involved in-depth interviews (IDIs) with stakeholder groups (service users, frontline health workers/ heads of PHCs and policy makers) in both intervention and non-intervention sites. This enabled assessment of perceptions of health workers, patients and policy makers on the usefulness of the e-Health tools in improving health outcomes for pregnant women and their newborns as well as feasibility for scale up of these interventions to other area councils in the FCT.

Description of the interventions in the “EXTEND Project” in FCT

A detailed description of the Video training (VTR) application and Clinical Patient Administration Kit (CliniPAK) in the EXTEND project in Nigeria has been previously described ¹⁶. The VTR tablets and Satcom technology were effectively deployed to 24 Primary health care centres in Gwagwalada (intervention) LGA in March 2017 following two key workshops and training of frontline health care workers and heads of facilities on the use of the video training enabled tablets. The CliniPAK software provides an electronic medical record that incorporates data on patient registration and demographics, vital signs, diagnosis, treatment, case review and administrative task support. The frontline health workers and their facility heads were visited on monthly basis by the researchers and more frequently by a technical team to ascertain their use of the tools as well as possible challenges faced and solutions while using these e-health innovations.

Methods of data collection/evaluation

Baseline, midline and endline assessments

Baseline, midline and endline assessments were conducted at commencement of the study, one year after, and at the end of the project in both intervention and control sites to ascertain the status of target key performance indicators (Maternal newborn child health services number of pregnant women attending ANC: number of women delivering in health facilities and attended to by health professionals and numbers of women accessing PNC), before, during and after full implementation of the project. The quantitative component involved analysis of yearly historical data (Baseline: January 2016 to December 2016, Midline: January 2017 to December 2017, Endline: January 2018 to December 2018). At end line, the data retrieved were also compared with CliniPak data at eight randomly selected intervention health facilities. This was aimed at assessing the completeness and quality of data received at the LGA which is usually in hard copies in comparison with CliniPak data which is usually transmitted “real-time” and stored in the devices and electronic platforms. For non-intervention health facilities, facility-level data were compared with data at the LGA for the same health facility only as CliniPaK devices were not deployed to them.

For qualitative data, a total of 28 in-depth interviews (14 IDIs from each area council: 4 clients, 4 Heads of facilities,4 for frontline health workers and 2 policy makers.) were conducted during each phase of the study across three stakeholder groups involving frontline health workers and facility heads, pregnant women/women receiving MNCH services, and policymakers. The individuals interviewed were purposively selected from eight randomly selected PHCs from intervention and six randomly selected PHCs from non-intervention sites. Same PHCs were involved at the three phases of the qualitative assessments and efforts were made to interview same head of facilities and frontline health workers if they were still working in the PHCs during subsequent interviews. For clients and policy makers, it was not possible to interview the same persons at midline and endline evaluations. The distribution of stakeholder groups interviewed is shown in table 1.

Data collection process for all the 3 phases of assessments commenced with a refresher training exercise, which enabled researchers from the University of Abuja to familiarize themselves with the tools of the In-depth interview guides and expected historical data, prior to commencement of fieldwork. Interviews were conducted by three members of the research team [OD, GOA and RMY], trained in qualitative interviewing. The potential participants were provided with the study information sheet at least 24 hours before the scheduled interview time to explain study objectives and help participants decide whether (or not) to partake in the study. The interviews which lasted 20–30 minutes were conducted in a private setting in a health facility, audio recorded, transcribed verbatim, and where appropriate translated into English for analysis.

Data analysis

Quantitative data analysis was done using Statistical Package for Social Sciences (SPSS), IBM SPSS statistics for windows, Armonk, NY: IBM Corporation 2011 Version 20. Proportion of change in percentage between baseline and endline assessment was determined by calculating the difference between the actual number recorded for the variable at endline and at baseline, divided by the baseline figure and multiplied by 100.The means between variables in the intervention and non-intervention groups were also compared when possible, using student t-test. Statistical significance (P-value) was set at p≤ 0.05. Additionally, baseline results were compared with midline and endline results to delineate the trends in the project Log frame indicators for intervention sites alone, non-intervention sites alone and between intervention and non-intervention sites. Results were also represented using charts, graphs and tables where necessary.

Regarding qualitative analysis, the framework approach was used for analysis, while allowing for the emergence of new themes. The framework analysis involves the stages of familiarization with data, coding (done by the 3 interviewers above and members of the research team from the University of Leeds: BE, MJA), indexing and charting, mapping, and interpretation ¹⁹. The analysis was performed manually.

Integration of data and emergent themes

We used a contiguous and narrative approach for data integration in which the results are presented separately ²⁰. The quantitative findings are presented first and followed by the qualitative findings, and thereafter discussion of the results together ^21,22.Thus, the discussion section combines the results of both phases to more fully appreciate the impact of the EXTEND project on utilization of maternal-neonatal-child-health services (MNCHS) in the FCT, Abuja.

Ethical approval and consent to participate

Approval for the study was granted by the University of Leeds School of Medicine Research Ethics Committee (MREC16-178), and the Federal Capital Health Research Ethics Committee (FHREC/2017/01/42/12-05-17). Written informed consent was obtained from all participants involved in this study.

RESULTS

Quantitative Findings

HMIS data for the years 2016, 2017 and 2018 were collected from the FCT authority and supplemented by data from the monitoring and evaluation unit for

Table 1: Distribution of the in-depth interviews between intervention (Green) and non-intervention (Red) LGAs

Table 2: Proportion of change reflected in FCT endline data compared against baseline data using number of patients. Actual baseline and endline figures are reported, with percentage magnitude of change between baseline and endline.

*LGA = local government area, ANC = antenatal care, PNC = postnatal care

Colour code = Green suggests desired change to percentage; Red suggests undesirable change to percentages.

Gwagwalada and Kuje. The data elements include antenatal care attendance, number of deliveries, deliveries attended by skilled birth attendance, family planning attendance, post-natal clinic attendance and pregnancy related complications including maternal deaths and still births. Also, data on human resources for health were collected from the two LGAs.

A positive increase in number of women utilizing maternal and child health services in the intervention LGA was noticed for all Log frame indicators. Compared to baseline, total ANC visits increased by 148.2%, % of births by skilled health personnel increased by 36.3%, total deliveries increased by 110.2 %, total PNC visits increased by 22.7%, % PNC visits within ≥7 days increased by 133.6%, number of clients counselled for family planning increased by 21.5% and % females accessing modern family planning methods also increased by 57.2%.

Contrastingly, at the non-intervention LGA, all log frame indicators except for № of clients counselled for family planning and % females accessing modern family planning methods showed a decline at endline when compared to baseline.

Total ANC visits decreased by 20.1%, percentage of births by skilled health personnel decreased by 32.1%, total deliveries decreased by 34.9 %, total PNC visits decreased by 29.6% and % PNC visits within ≥7 days decreased by 74.4%. The number of clients counselled for family planning and % females accessing modern family planning methods increased by +47.4% and 26.9% respectively. This is shown in table 2.

It was also observed that the patterns of change showed similar trend when mean differences in the utilization of the MNCH services between the two groups were compared. This is shown in Table 3.

Table 3: Log frame Indicators for Gwagwalada and Kuje Local Government Areas for 2016, 2017 and 2018

(Statistic = t-test) ⃰are statistically significant

Figure 1: Total antenatal attendance in Gwagwalada vs. Kuje 2016,2017 and 2018

Trend analysis of utilization of MNCHS services during the three phases of the study in both intervention and non-intervention LGAs for the Log frame indicators also showed higher utilization at endline compared to base line between intervention and non-intervention site as well as in the intervention site when endline was compared to midline and baseline. Exceptions to the above trend was noticed in relation to utilization of partograph for monitoring of labour and skilled attendance at birth. These

Figure 2 Primary Health Care Human Resources Gwagwalada vs Kuje in 2018

two variables showed a decline in utilization at the end line compared to base line. These are shown in figures 1 and 2 ,3-16 in the supplementary file.

Overall, Kuje LGA (Control) had a higher number of health workforce and more importantly a higher number of Community Health Extension Workers (CHEW=100) when compared to Gwagwalada (74). The CHEWs are the predominant frontline primary health care workers in Northern Nigeria. This is shown in Figure 2 below.

Qualitative Data Analysis

Main Themes

Four main themes emerged from analysis of the interview transcripts at endline: (1) Perceived benefits of the E-Health applications, (2) acceptability of E-Health applications, (3) challenges experienced during the use of the E-Health application, and (4) plans for sustainability and scale up.

Perceived benefits of the E-Health applications

The stakeholders reported several benefits attributed to the E-Health applications with positive impacts on the patients, health workforce and the health system in general. In their opinion, these benefits resulted in improved quality of care as well as utilization of health services. The highlighted benefits were in the following areas;(i) Staff training and capacity building (ii)Standard of care (iii)Attitudes of health workers (iv) Supervision of staff/monitoring and evaluation (v) Data collection, transmission and management.

Staff training and capacity building

The video training application was reported to be useful in updating staff on the current trends of managing diseases and preventive health. The unique benefits included being trained on the job and staff did not need to leave their job post. There was no structure for training in Gwagwalada area council and thus the video training application was the main stay of training and capacity building. The apps improved staff knowledge as they were exposed to continuous learning and better capacity to care for their patients. They had learnt new concepts and improved techniques for administering MNCH services. Prominent skills of interest mentioned by most stakeholders that were learnt included; conduct of antennal care labour and delivery, neonatal resuscitation, active management of third stage of labour, administration of magnesium sulfate for managing preeclampsia and need for early referral of patients.

“I think I have benefited from the video through improvement in my skills on management of maternal child health issues. There was a day a woman delivered and there was eh…. bleeding. The bleeding was too much and all of us were confused on what to do but later on we suggest let’s use our video, we were even about calling our in-charge that day, I could remember we said let’s use our video and we used the video, really, we succeeded and the bleeding stopped".Frontline health worker

“It has really helped us in training ourselves and in rendering service here. You know at the time we look at it and we see any mistake or maybe in the services we are rendering if we go back to the video we tried to correct our self… taking of delivery, all the necessary precautions that we need to take before we take delivery… by the time the baby is coming out, normally we just cut the umbilical cord and just keep the baby aside, but in the video by the time we deliver the baby, they will put the baby on the mothers abdomen, in order to create warmth and bonding before we can cut the umbilical cord”. Facility Head

“I must tell you it has really helped us. For a staff to be able to open into the system, into the video and update him or herself on the latest trends of disease management and evaluation is fulfilling. Also, if you have a case in front of you and you don’t know how to go about it, just open the video and get yourself updated, you can get that done within seconds. I have had that experience with my staff in tugan-maje PHC, who had a case of primary postpartum haemorrhage and watched the video to be able to manage the patient until the patient was referred. It was impressive. Policymaker

“There is a case that I handle like that, a woman gave birth to a baby. So, the baby cannot breathe. So, I have to remember what I watched from the video and I put it into practice and the baby was able to breath...the baby is alive now. So that's why I really appreciate the video training apps”. Frontline health worker

Standard of care

This project was reported by policy makers, healthcare staff and clients as being helpful in improving quality of clinical management, reporting and referral.

“The mortality and the mobility have reduced tremendously from the facility based by the report we receive now. Because as I said earlier the patient or the client are also carried along, they show them this is how it’s being done and they see it themselves, so they now know the ideal thing, they key into it and also participate and surprisingly even the husbands, the spouse now followed them to the clinics”. Policymaker

The majority of service users were satisfied with the level of competence displayed by health workers in their facilities. Service users were also satisfied with the attitude of health workers as they reported how caring and friendly, they were to them when they visited the facilities.

“There are ok to me like I said before, they’ve been rendering the services well, if not I won’t come back if it was not well”. Service user

Clients were also attended to in a cordial manner and promptly too. The speed of services was said to have improved after the introduction of the data entry tablet computers.

“What I just noticed now is that compared to before, I think the system is really faster now, like they answer us, not that we would stay longer than expected. Service user

Attitude of health workers

The use of video training applications was considered by many heath workers to have boosted their self-esteem and confidence to handle cases in the facilities.

“I think it has increased our level of confidence, I would like to use myself as an example, I studied biology and I’m actually the one entering patients’ data. I was oblivious to most of the terminologies and variables that needed to be entered into the registers but after watching the video with the nurses I can now comprehend the reasons for the questions and information iam entering into the tablet computers. I now do my work as a professional… it has increased my confidence. It has affected my motivation strongly, because each day I am open, I want to learn new things and there is always something to watch, it’s motivating me to want to do something, I don’t just sit down and say this is the only part, this is my role in the clinic, I also tend to assist in other things when help is needed”. Frontline health worker

Participants reported being more satisfied with their jobs because of the video training apps and data management tool. These tools made their work easier and more rewarding. The task of data management was not limited to the M&E officers but other health workers were now involved in the process.

“it has had a positive effect, I’m more satisfied using the video training apps because prior to this time as an N-power staff who is an adhoc staff, I can’t go for the trainings outside so I know nothing about updates in management of pregnant women and their newborn but now with this video apps, I am able to get the knowledge and skills to help me in my daily work. Rather than sitting down and discussing with my colleagues when there is not patient, I now watch the videos”.Frontline health worker

Supervision of staff/monitoring and evaluation

Heads of health facilities found it easier to monitor patients and staff and make decisions about clinical outcomes when they compared parameters from previous visits. They were also able to make better decisions about referral in emergencies because they were more informed about situations they could handle in their facilities. For example, some heads of facilities gave examples of monitoring morbidity data to inform increase in prevention activities as needed or notification of disease surveillance officers for action.

“It has helped because in that aspect, if there's any disease, this thing will notify or that it's getting higher, like in the issue of malaria for example or measles, if we're seeing the cases they are getting more. From there we can easily notify the monitoring and evaluation unit about what is happening and they will easily notify the disease surveillance officers for action”. Head of facility

The video training aps has helped improved our supervision and health outcome as the health staff now know what to do even in emergencies like referring patients early. If we do our own intervention and we feel it is something we cannot handle, we refer to the other tertiary Institution. Head of facility

Data collection, transmission and management

The Clinipak tool was preferred to manual filling of data within the clinic because it facilitated real-time electronic transmission to the area councils. The tablets made it very easy to collect and process data, improving the timeliness of data collection and transmission. The pages of the manual registers sometimes got missing, damaged or torn resulting in data loss and poor record keeping. The data collection with the tablet addressed all these issues in the facility.

“There are lots of advantages using the tablet. Before we used to submit data late, but when they brought this tablet to collect the data is very easy, we used to send it easily”. Frontline health worker

“…when you put data inside the register that we were using before, some will get lost because the register may tear off, were you wrote something may tear off and it will get lost but when you put it inside the tablet computer it will be stored there. So whenever you need it you go back and pick it and you will see your data complete”. Frontline health worker

The tablet computer also made it easier to submit harmonized records from different departments in the facility and the health workers could transmit the data to the local government headquarters without getting up from their desk. The convenience that the e-health CliniPak tool afforded health workers in Gwagwalada area council made the data management process more efficient compared with the use of paper-based entry. Although manual data collection continued because they were required along with the electronic entries for state level and national level monitoring, the entire process was more efficient. Health workers could send electronic copies ahead of the paper entries if they had challenges with physically moving data (hardcopies) monthly to the PHC headquarters.

“Before now that we use the register, the personal data is not serial the way it is in the CliniPAK, is not like that, now so everything is serial, you know who is entering the data which nurse or which mid-wife that enter this data like the employer ID that is there you know who is entering it, the patients name. It follows serially like that till the end, so everything comes one after the other”. Frontline health worker

Data validation improved with the use of clinipak because it was easier to reconcile the records and correct any omissions before final submission.

“Yes, it has improved, the most important effect in our facility is that all the record… all the data we are submitting, all the data we are giving are correct data, and it reduces much load in our head and we appreciate because doing this paper work, it was not easy for one to carry the paper registers, I think the registers we have in this facility are up to 15 in numbers or more than. now, everything is in one place”. Head of facility

“Yes, smooth, more smooth reportage, you have my words smooth reportage, you don’t have to start waiting for any hard copy of anything, just go online and you will see all the data. You can take that from me”. Policymaker

Acceptability of E-Health applications

The narratives of facility heads and policy makers mirrored those of health workers regarding the acceptability of the video training application and data collection tool. The majority of the participants reported that the video training application was considered acceptable because it facilitated a convenient means of improving knowledge and skills at work. The video training was considered practical and available as reference material real time. The videos could be watched as many times as possible, especially during emergencies. It was considered easy to use, comprehensive and had practical applications that were commonly seen in the clinic. Junior and senior staff were able to learn together, and junior staff could ask us questions and get clarification from colleagues without feeling shy about expressing themselves in large face to face trainings.

“I must confess that video training is the best than face to face because for the video training you are given enough chances, you do your training at your convenient time, so it doesn’t take you away from doing any other work. When you have time, you do your video training unlike in face-to-face training in which sometimes there are lots of people, there are crowds and you can hardly hear what the instructor is saying. To me I feel the video training is still the ultimate best”.Facility Head

“... the most significant part of using that video app is that it is cheap and enables staff to get continuous medical education. Even if you wake up today and have doubts you about something, you can bring the video clips and still go over it, to remember what you forgot.so is very important and is something much more better than going for a training outside. Frontline health worker

The majority of the participants found the video training apps and data management tool highly rewarding because it was practical and made the work easier. They were happy to use the video training apps despite not getting incentives to do so.

“This one is more efficient because everything is practicalized in the video training, you can see, you can watch, you can do it at that very time and at any time you feel you are missing something in what you are doing, you can quickly go and pick up your video to play back, to do the right thing”. Frontline health worker

“The money allowances are not my problem. For me, I am much more concerned and interested in getting the training from the video apps. So, money allowances are not the problem If our clients and patients will be okay, it is better than the money. Frontline health worker

Challenges experienced during the use of the E-Health applications.

The main challenge experienced by the health workers in the effective use of the e-health applications was the unavailability of some of the equipment and gadgets used in the video training apps for patients’ management in the PHCs thereby making the practice of all the skills learnt difficult in some instances. There was also logistic challenge in many staff sharing just one tablet computer supplied to the facility, difficulty in using the tablet computer for data entry by some staff who were not computer literate and lack of power to charge the tablet computers.

“Well, you know in this our primary health facility where we don't have most of these modern things for some of these observation like the BP and other things to use in the care of patients. Though we have some but there are some modern ones that we used to see inside the video that is more authentic than all these old ones that we have”.

“…the challenge we have is that in terms of taking record in the tablet, it is not easy you know, like for those who are not a little bit computer literate or understand something that are technical.” Facility Head

“The negative is when there is no light, so we find it difficult to send the report promptly”. Facility Head

Plans for sustainability and scale up.

Sustainability of the e-health project was extensively discussed by the health workers; heads of facilities and policy makers were keen to see the project continue beyond the lifespan of donor funding. The importance of political will by government at state and national level for programmatic scale up was emphasized.

“If there is political will the e-health interventions can be sustained, that is political will in the sense that the administrators should make sure that ownership of such program should be taken seriously and there should be financial backing not in the situation no money no money no money”. Policymaker

“The impact is awesome, and the impact is something we have to sustain at all cost and spread to other Area Councils in FCT to all the States and of course Nationwide but with the same zeal that we are seeing now to make it work and more impactful. Policymaker

“I can advise the government to put in money... and this project should not go down. ...Sustain the program continuously. We thank the people that joined hands together to make this program a success, it is helping us well. Facility head

DISCUSSIONS

Digital tools have the potential to dramatically enhance primary health care performance, but their potential is only beginning to be channeled by health systems ²³. In Nigeria, the role of digital health and its impact on strengthening primary health care system at scale is still at its developmental stage. Our study reports the FCT, Abuja level data on how a National innovative e-health interventions (that comprised of installation of Satellite devices (Satcom) to improve internet connectivity in rural communities, deployment of video training apps for training of health workers and use of tablet computers with data monitoring tools for data capture, transmission and storage) helped to improve utilization of maternal, newborn and child health services utilization at the primary health care level. The paper also highlights the ease of acceptability and use of the e-health interventions by health workers, challenges in implementations and opportunities for sustainability and scale up.

At endline evaluation, we found remarkable improvement in utilization of maternal, newborn and child health services in PHCs in the intervention LGA compared to the non-intervention LGA at the endline evaluation. This trend of improved utilization of services was similar when findings at endline were compared with base line at the intervention LGA.

Frontline health workers knowledge, skills, and confidence in the management of conditions related to pregnancy, labour and puerperium were cited as benefits of the video training. The health workers cited practical examples in which they have used the knowledge and skills gained from the video training to manage their patients with immediate positive results. This positive outcome in patients care encouraged them to further watch and recommend the videos to their colleagues.

Service users in the intervention LGA also found the treatment and care they received following the commencement of the project as satisfactory and were happy that frontline health workers referred to relevant videos to guide their provision of appropriate care. They saw this practice as highly commendable as it reduced morbidity and mortality that could have arisen from misdiagnosis, inappropriate treatments and logistic challenges when they are referred to other facilities.

Poor quality of care resulting from inadequate knowledge and skill of health workers and unavailability of health workers at the PHCs have been documented as major barriers to women accessing maternal, newborn and child health services in sub-Saharan Africa ^24-26. A good strategy to mitigate this challenge is in the area of health workers training. A Previous study ²⁷ had highlighted the benefits of health workers training on improvement in utilization of maternal, newborn and child health services. While the aforementioned study used the traditional face-face, out of station training methodology, our study deployed a low-cost digital technology with tremendous advantages of peer-to-peer learning. flexibility for time of training, immediate on the job application of lessons learnt, opportunities for revision and updates. The increased utilization of MNCH services following the use of video training apps for training of health workers in the FCT was similar to findings from Ondo State, Nigeria where the project was also implemented ²⁸.

Health information system is one of the six core components or “building blocks” of the health system ²⁹. A well-functioning health information system is one that ensures the production, analysis, dissemination and use of reliable and timely information on health determinants, health system performance and health status ³⁰. The data management tool (Clinipak) utilized in this study enabled accurate, confidential and prompt availability of patient’s data for their routine care as well as for monitoring and auditing process in compliance with the WHO definition of a well-functioning health information system ³⁰.Service users were happy for their data to be entered into the tablets computers as it made retrieval of information’s relating to them easier during their next visits when their case files couldn’t be assessed for several reasons ranging from misplacement to damage in rainy seasons. They also felt that their records were more secured in the tablet computers than in the case notes.

The logistic challenges in the use of multiple hard copy registers, storage of patient’s data in unsecured cabinets, difficulty in transporting relevant data from rural health posts to the LGA for monthly audits were overcome by the use of the Clinipak tool. The study showed ease of acceptability and satisfaction in the use of the digital health innovations by both service users and health workers. This is similar to a previous National data report from the EXTEND project ¹⁶. The ease of acceptability of digital health technology by stakeholders in our study was however different from reports from other studies where some health workers had reluctance in the use of mobile technology in patients care ^31,32.The positive response to these innovations in our study site may include prior training to prepare FHWs to embrace technology, access to technology in health facilities, and availability of regular technical supports which were all embedded in the digital health intervention packages. This has also been previously reported ¹⁶. Additionally, the previous studies where the participants showed reluctance in the use of mobile apps explored using participants smart phones in healthcare which raised several concerns like safety of information, costs of service, and professional dilemmas unlike our study that used a tablet computer that was deployed to the hospitals and supported with internet connectivity.

Despite the enormous advantages of the video training application for building the capacity of health workers and the Clinipak data tool which translated to improved utilization of MNCH services and management of clinical data respectively, there were observed challenges which needs to be considered in future programming to ensure ease of translation of interventions to palpable impacts. Although our study utilized the Satcom devices to overcome the challenges posed by unavailability of 3G mobile networks in some of the rural communities’ other problems like lack of stable electricity still hampered the uninterrupted use of the video training apps and Clinipak tool in these facilities. Consideration of alternative sources of power supply like the installation of solar panels and inverters must be considered for continuous use of digital technologies for service delivery in primary health care settings in Nigeria.

Discussions with policy makers and stakeholders in the FCT suggested that the EXTEND Project in the FCT was contributing significantly to improved standard of patients care in intervention health facilities as well as in the quality and promptness of data submission to the NHMIS. They expressed their willingness to support the expansion of the project to involve other LGAs in the FCT. With the lessons learnt from the COVID-19 pandemic on the relevance of digital health technology in health service provision coupled with the current global economic crises which makes funding for health care training challenging as well as the increasing dearth of human resources for health in Nigeria due to migration of health workers, the need for further scale up of this laudable project becomes imperative towards realization of the sustainable development goals in Nigeria.

Study strengths and limitations

Our study used a mixed methods approach to evaluate the role of a package of e-health intervention on the utilization of MNCH services. The mixed methods approach enabled wholistic evaluation of the impact of the intervention as well as other contextual factors that may have affected the results. The quantitative data received from the facilities was compared with the state level data to ensure its correctness while the qualitative data involved several stakeholder groups to allow for heterogenous information gathering.

The study findings may however be limited by social desirability bias. Given that the interviews were conducted at the health facility where the participants and health workers received health services and provided services respectively, these factors may have influenced their responses.

CONCLUSION

E-health innovations when implemented at scale with the involvement of relevant stakeholders improved MNCH service delivery in PHCs in Abuja, Nigeria. The importance of political will by the government at local, state, and national level for further programmatic scale-up and sustainability cannot be over-emphasized.

Data Availability Statement

The original contributions presented in the study are included in the article

Ethics Statement

Ethical approval for the study was granted by the University of Leeds, School of Medicine Research Ethics Committee (MREC16-178), the Federal Capital Territory Health Research Ethics Committee, Abuja, Nigeria (FHREC/2017/01/42/12-05-17). The patients/participants provided their written informed consent to participate in this study.

Author Contributions

BE and GOA jointly conceived the study. GOA, OD, RY, BE, MJA were involved in data collection and analysis of study findings. GOA developed the manuscript and led the writing of this paper with contributions from BE, OD, RY and MJA. All authors read and approved the final version of the manuscript.

Funding

This article presents independent research funded by the UK Space Agency International Partnership Program (Grant reference number IPPC1-30). The sponsor, funders and the company that implemented the technologies were not involved in the study design; collection, management, analysis, and interpretation of data; writing of the report; and the decision to submit the report for publication.

Disclaimer

All views expressed in this publication are of the authors only.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Trial Registration: ISRCTN32105372

REFERENCES

World Health Organization (‎2019)‎. Trends in maternal mortality 2000 to 2017: estimates by WHO, UNICEF, UNFPA, World Bank Group and the United Nations Population Division: executive summary. World Health Organization. https://apps.who.int/iris/handle/10665/327596. Accessed on May 12, 2022.
Lawn JE, Blencowe H, Waiswa P, Amouzou A, Mathers C, Hogan D, et al. Stillbirths: rates, risk factors, and acceleration towards 2030. Lancet. 2016; 387(10018): 587-603.
Abubakar I, Dalglish SL, Angell B, Sanuade O, Abimbola S, Adamu AS et al. The Lancet Nigeria Commission: Investing in health and future of the nation. The Lancet.2022;399(10330):1155-1200. https://doi.org/10.1016/ S0140-6736(21)02488-0
Okereke, E., Ishaku, S.M., Unumeri, G. et al. Reducing maternal and newborn mortality in Nigeria—a qualitative study of stakeholders’ perceptions about the performance of community health workers and the introduction of community midwifery at primary healthcare level. Hum Resour Health,2019; 17: 102 https://doi.org/10.1186/s12960-019-0430-0
Adewuyi EO, Auta A, Khanal V, Bamidele OD, Akuoko CP, Adefemi K, Tapshak SJ, Zhao Y. Prevalence and factors associated with underutilization of antenatal care services in Nigeria: A comparative study of rural and urban residences based on the 2013 Nigeria demographic and health survey. PLoS One. 2018 May 21;13(5):e0197324. doi: 10.1371/journal.pone.0197324.
National Population Commission (NPC) [Nigeria] and ICF. Nigeria Demographic and Health Survey (NDHS) 2018. Abuja, Nigeria, and Rockville, Maryland, USA: NPC and ICF.
Aregbeshola BS, Khan SM. Primary Health Care in Nigeria: 24 Years after Olikoye Ransome-Kuti's Leadership. Front Public Health. 2017;13; 5:48. doi:10.3389/fpubh.2017.00048.
World Health Organization (WHO). Primary health care systems (PRIMASYS): case study from Nigeria. Geneva: World Health Organization; 2017. Licence: CC BY-NC-SA 3.0 IGO.
Nnebue CC, Ebenebe UE, Adogu PO, Adinma ED, Ifeadike CO, Nwabueze AS. Adequacy of resources for provision of maternal health services at the primary health care level in Nnewi, Nigeria. Niger Med J. 2014;55(3):235-41. doi: 10.4103/0300-1652.132056.
Ijadunola KT, Fatusi AO, Orji EO, Adeyemi BA, Owolabi OO, Ojofeitimi EO, et al. Unavailability of essential obstetric care services in a Local Government Area of south-west Nigeria. J Health Popul Nutr. 2007; 25:94–100.
WHO guideline: recommendations on digital interventions for health system strengthening. Executive summary. Geneva: World Health Organization; 2019. (WHO/RHR/19.8). Licence: CC BY-NC-SA 3.0 IGO.
Kaunda-Khangamwa, B.N, Steinhardt, L.C, Rowe A.K. et al. The effect of mobile phone text message reminders on health workers’ adherence to case management guidelines for malaria and other diseases in Malawi: lessons from qualitative data from a cluster-randomized trial. Malar J.2018; 17: 481. https://doi.org/10.1186/s12936-018-2629-2
Adanikin AI, Awoleke JO, Adeyiolu A. Role of reminder by text message in enhancing postnatal clinic attendance. Int J Gynaecol Obstet. 2014;126(2):179-180. doi:10.1016/j.ijgo.2014.02.009
Kebede AS, Ajayi IO, Arowojolu AO. Effect of enhanced reminders on postnatal clinic attendance in Addis Ababa, Ethiopia: a cluster randomized controlled trial. Glob Health Action. 2019;12(1):1609297. doi: 10.1080/16549716.2019.1609297.
Ebenso B, Allsop MJ, Okusanya B, Akaba G, Tukur J, Okunade K, et al. Impact of using eHealth tools to extend health services to rural areas of Nigeria: protocol for a mixed-method, non-randomised cluster trial. BMJ Open. (2018) 8:e022174. doi: 10.1136/bmjopen-2018-022174
Ebenso B, Okusanya B, Okunade K, Akeju D, Ajepe A, Akaba GO, et al. What are the contextual enablers and impacts of using digital technology to extend maternal and child health services to rural areas? Findings of a qualitative study from Nigeria. Front Glob Womens Health. (2021) 2:1– 8. doi: 10.3389/fgwh.2021.670494
Worldiometer: Population of Nigeria 2020.Available at: https://www.worldometers.info/world-population/nigeria-population/. Accessed 21/05/2022
Federal Capital Territory City Population. Available at: https://www.citypopulation.de/php/nigeria-admin.php?adm1id=NGA015. Accessed 21/05/2022
Ritchie J, Spencer L. Qualitative data analysis for applied policy research. Qual Res Companion. (2002) 573:305–29. doi: 10.4135/9781412986274.n12
Creswell, J. W., & Plano Clark, V. L. (2017). Designing and conducting mixed methods research (3rd. ed.). SAGE Publications Inc.
Fetters, M. D., & Freshwater, D. (2015). Publishing a methodological mixed methods research article. Journal of Mixed Methods Research, 9(3), 203– 213. https://doi.org/10.1177/1558689815594687
Fetters, M. D., Curry, L. A., & Creswell, J. W. (2013). Achieving integration in mixed methods designs-principles and practices. Health Services Research, 48(6 Pt 2), 2134– 2156. https://doi.org/10.1111/1475-6773.12117
Cramptona N, Donga A , Agarwala P, Jia C, Bhattacharyyaa O. Primary health reform for the digital era. In: Roadmap to Successful Digital Health Ecosystems.2022:499-523. https://doi.org/10.1016/B978-0-12-823413-6.00009-4
Vidler M, Ramadurg U, Charantimath U, Katageri G, Karadiguddi C, Sawchuck D, Qureshi R, Dharamsi S, Joshi A, von Dadelszen P, Derman R, Bellad M, Goudar S, Mallapur A; Community Level Interventions for Pre-eclampsia (CLIP) India Feasibility Working Group. Utilization of maternal health care services and their determinants in Karnataka State, India. Reprod Health. 2016 Jun 8;13 Suppl 1(Suppl 1):37. doi: 10.1186/s12978-016-0138-8.
Machira K, Palamuleni M. Women's perspectives on quality of maternal health care services in Malawi. Int J Womens Health. 2018 Jan 9;10:25-34. doi: 10.2147/IJWH.S144426.
Dadi LS, Berhane M, Ahmed Y, Gudina EK, Berhanu T, Kim KH, Getnet M, Abera M. Maternal and newborn health services utilization in Jimma Zone, Southwest Ethiopia: a community based cross-sectional study. BMC Pregnancy Childbirth. 2019 May 22;19(1):178. doi: 10.1186/s12884-019-2335-2.
Shen Y, Li Q, Liu X, Xiao S, Yan H. Training and financial intervention for encouraging maternal health service utilization: Results of cluster randomized trials in Shaanxi Province. Medicine (Baltimore). 2019 Nov;98(45):e17709. doi: 10.1097/MD.0000000000017709.
Akeju D, Okusanya B, Okunade K, Ajepe A, Allsop MJ and Ebenso B. Sustainability of the Effects and Impacts of Using Digital Technology to Extend Maternal Health Services to Rural and Hard-to-Reach Populations: Experience From Southwest Nigeria. Front. Glob. Womens Health.2022; 3:696529. doi: 10.3389/fgwh.2022.696529
World Health Organization (WHO). Everybody's business - strengthening health systems to improve health outcomes: WHO's framework for action. WHO; Geneva:2007. http://www.who.int/healthsystems/strategy/everybodys_business.pdf
Manyazewal T. Using the World Health Organization health system building blocks through survey of healthcare professionals to determine the performance of public healthcare facilities. Arch Public Health. 2017 Aug 31;75:50. doi: 10.1186/s13690-017-0221-9.
Giles-Smith L, Spencer A, Shaw C, Porter C, Lobchuk M. A study of the impact of an educational intervention on nurse attitudes and behaviours toward mobile device use in hospital settings. J Can Health Libr Assoc. 2017 Apr 06;38(1):12–29. doi: 10.5596/c17-003.
Payne KB, Wharrad H, Watts K. Smartphone and medical related App use among medical students and junior doctors in the United Kingdom (UK): a regional survey. BMC Med Inform Decis Mak. 2012 Oct 30;12:121. doi: 10.1186/1472-6947-12-121.