The management of suspected critical congenital heart defects (CCHD) relies on timely echocardiographic diagnosis. The availability of experienced echocardiographers is limited or even non-existent in many hospitals with obstetric units. This study evaluates remote-mentored echocardiography performed by physicians without experience in imaging of congenital heart defects (CHD).
Methods
The setup included a pediatric cardiologist in a separate room, guiding a physician without experience in echocardiographic imaging of CHD in the examination of a symptomatic newborn. This remote-mentoring pair was blinded to the diagnosis of the newborn and presented with a simplified patient history. The echocardiographic images were streamed to the laptop of the mentor, along with a webcam feed showing the probe position. The task was to identify CCHD in need of immediate transfer to a pediatric cardiac surgical center. The result was compared to the previously completed echocardiographic report and the clinical decision of the patient-responsible pediatric cardiologist.
Results
During 17 months, 15 newborns were recruited. All six newborns with CCHD were correctly labeled by the remote-mentoring pair. One newborn with Tetralogy of Fallot was erroneously labeled as needing immediate transfer. Eight newborns without CCHD were correctly labeled.
Conclusions
Remote-mentored echocardiography performed by examiners without experience in imaging CHD identified all newborns with CCHD in need of immediate transfer for specialist care. The setup shows promising results for improving the management of CCHD in hospitals without continuous pediatric cardiology service.
Remote-mentoringechocardiographytelemedicinecongenital heart defectEuropean Union’s Project Horizon 2020 and 5G HEART857034Norwegian Association for Children with Congenital Heart DiseaseIntroduction
The prevalence of congenital heart defects (CHD) is approximately 0.8 per 1.000 births [1,2]. Approximately 25% of these are critical congenital heart defects (CCHDs). These newborns often show signs of CHD soon after birth and need prompt echocardiographic evaluation. If performed correctly, these echocardiograms will identify the newborns with CCHD who need transfer to a pediatric cardiac surgical center for specialist care. In smaller hospitals with births clinics and neonatal intensive care units (NICUs), experienced echocardiographers may be unavailable during on-call hours. To meet this challenge, remote-mentored echocardiography could be a possible solution. Previous studies have demonstrated an accuracy of around 50%–60% of pediatrician-performed echocardiography, compared to standard pediatric cardiologist echocardiography in larger centers, including both false positive and negative diagnoses of CCHD [3–6]. Remote-mentored echocardiography is reported to increase diagnostic accuracy to above 90%, only missing minor CHD, typically clinically insignificant ventricular septal defects, or mild pulmonary valvular stenosis. Previous publications all report an excellent ability to identify CCHD by remote mentoring [3–7]. Newborns need transportation for echocardiography to obtain a diagnosis in centers without remote consultation services or in-house pediatric cardiologists. Several publications report a significant reduction in unnecessary transport with the use of remote mentoring, which is beneficial for the sick newborn and cost-saving for the healthcare system [3–5,8]. Previous remote-mentoring publications include neonatologists, pediatricians or sonographers of adult cardiology to perform echocardiography [3–12]. These mentees either received preparative training in imaging CHD or a significant training volume during the data collection. To the best of our knowledge, there are no previous publications on remote-mentored neonatal echocardiography by physicians without experience in imaging CHD. The included physicians have varying levels of experience in echocardiography, which corresponds to real-life scenarios in many smaller hospitals.
The primary aim of this study was to evaluate the feasibility of echocardiography performed by remote-mentored physicians without experience in imaging CHD, in identifying newborns with CCHD. Our secondary aims were to assess the diagnostic performance of physicians with varying echocardiographic experience, as measured by mentor evaluation, self-evaluation, and examination time. In addition to this, we evaluated the technical feasibility of image transmission over a 4G network.
Materials and MethodsPatient Inclusion
Parents and their newborns were included by written consent from August 2021 to December 2022 at a level IV NICU at Oslo University Hospital (Oslo, Norway). Inclusion criteria were (1) that the patient was admitted to the NICU, and (2) that a clinical echocardiogram was previously performed. Exclusion criteria were: patient not being stable enough for study inclusion as assessed by the responsible neonatologist, or if any part of the remote-mentoring pair was not blinded to the patient history before the study echocardiogram (Fig. 1).
Flow chart of the inclusion process
Remote Mentoring
After patient identification, a remote-mentoring pair was approached for echocardiographic evaluation. The pair consisted of one of two pediatric imaging cardiologists and one of seven physicians without experience in echocardiographic imaging of CHD. The physicians without experience with CHD had varying levels of echocardiographic training, ranging from anaesthesiologists experienced in point-of-care ultrasound to fully trained adult cardiologists. The remote-mentoring pair was blinded to the newborn’s diagnosis and presented with a simplified clinical history and physical findings. The bedside infusion pumps were covered so that the remote-mentoring pair could not see what was administered. The task at hand was to decide whether the newborn had CCHD requiring transfer for specialized care, given the theoretical scenario that the newborn was at a local hospital. If the newborn was classified as in need of transfer, the remote-mentoring pair was asked to record an image loop demonstrating the pathology in question. The remote-mentoring pair was asked to save an apical four-chamber 2D loop if no transfer was needed. After guiding the examination, the mentoring pediatric cardiologist, sitting at home or in another part of the hospital submitted a report of the anatomy and decision for transfer. The report from the remote-mentoring pair was compared to the echocardiographic report of the patient-responsible pediatric cardiologist at the tertiary level.
Remote mentoring was performed using REACTS (Philips), a designated remote-mentoring software, and a 4G network link. The ultrasound machine (Philips EPIQ G7) was connected to the laptop at the end of the pediatric cardiologist showing the live ultrasound video stream and a webcam showing probe position. Voice communication was done by cell phone (Fig. 2). The parents were asked to be present, along with the patient-responsible nurse.
Illustration of the remote-mentoring pair and the equipment used for remote-mentored echocardiography
Equipment
The technical setup has been described in a previous publication [13]. The probes used were Philips Cardiac Sector Probe S12-4 (4–12 MHz) and S8-3 (3–8 MHz), Koninklijke Philips, Amsterdam, Holland. The webcam used with the Phillips EPIQ to show probe position during remote mentoring was a Logitech C920S PRO HD WEBCAM, Max Resolution: 1080–720 p/30fps, Camera megapixel: 3 (Logitech, Lausanne, Switzerland). The remote-mentoring software used on the Phillips EPIQ was REACTS and Collaboration Live, INNOVATIVE IMAGING TECHNOLOGIES INC., and REACTS (Montreal, Quebec, Canada) (Koninklijke Philips, Amsterdam, Holland). The ultrasound machine with guidance software was connected to an ASUS ROG STRIX G laptop (ASUSTeK Computer Incorporated, Taipei City, Taiwan). The network router used was the Huawei H138-380 wireless 4G router (Shenzhen, Guangdong Province, China).
Participants and Assessments
Information regarding the examining physicians’ clinical specialties and their experience with echocardiography was gathered. They rated their own user experience immediately after completion of studies, by a 6-point Likert scale questionnaire. The rated topics included their ability to solve the task, communication with the mentor, ease of orientation within the cardiac anatomy, ease of producing the required images, and their own overall performance. Similarly, the mentor rated their user experience in terms of ease of communication with the examiner, cooperation with the examiner, ease of directing the probe position, ease of instructing image optimization, and if image transmission quality was sufficient. The examination time was noted. Parents were asked about their theoretical willingness to let their child undergo a similar remote-mentored examination as an alternative to immediate transportation, in the absence of a pediatric cardiologist at their hospital.
Examining Physicians
– One consultant cardiologist and two resident doctors of adult cardiology.
– Two consultant anaesthesiologists.
– One resident doctor of pediatrics with basic echocardiographic training.
– One MD with basic experience in general ultrasound.
The three cardiologists had an average of 9.3 years of experience with echocardiography, and the resident doctor of pediatrics had 0.5 years of experience. Neither the anaesthesiologists nor the MD with basic experience in general ultrasound had any echocardiographic experience. None of the examining physicians had experience with echocardiographic evaluation of CHD.
Statistics
All analyses were performed in STATA 16.0 (StataCorp LLC, Texas, USA). Gwet’s AC1 was used to evaluate the agreement between remote-mentored echocardiography and the clinical plan. The examining physicians were divided into two groups depending on if they had previous echocardiographic experience or not. A difference in the rating of examining physician and mentor user experience depending on previous echocardiographic experience in the examinators was analyzed by a Student’s t-test.
Results
Fifteen newborns were included for remote-mentored echocardiography. Evaluation of cardiac anatomy could be performed during all 15 examinations. Six newborns with CCHD were correctly labeled as in need of immediate transfer to a pediatric cardiac surgery center. All six newborns had surgery during the first week of life. No cases of CCHD were missed. Another eight newborns were correctly labeled as without the need for immediate transfer. One newborn with stable TOF was erroneously labeled as in need of transfer (Tables 1 and 2). By Gwet’s AC1, the remote mentoring and clinician plans showed an agreement of 93%, and a corresponding coefficient of 0.87 (0.59–1.00), p < 0.001. The resultant sensitivity and specificity of remote-mentored echocardiography in this study were 100% and 89% (73%–103%), respectively.
Agreement in the necessity of transfer between the patient-responsible clinician and the remote-mentoring pair in participants with CCHD
Patient ID
Mentee
Mentor
Clinician diagnosis
Clinician plan
Remote-mentoring plan
1
Cardiologist 1
A
TGA
Transfer
Transfer
2
Cardiologist 1
A
TGA
Transfer
Transfer
3
Cardiologist 1
A
CoA and VSD
Transfer
Transfer
4
Anaesthesiologist 1
B
TGA
Transfer
Transfer
5
MD
B
Pulmonary atresia
Transfer
Transfer
6
MD
B
Hypoplastic left heart syndrome
Transfer
Transfer
Agreement in the necessity of transfer between the patient-responsible clinician and the remote-mentoring pair in participants without CCHD. Erroneous labeling of one participant is marked in <bold>bold</bold>
Patient ID
Mentee
Mentor
Clinician diagnosis
Clinician plan
Remote-mentoring plan
1
Cardiologist 2
A
Intrauterine suspicion of CoA, no evident CoA in echocardiography
No transfer
No transfer
2
Pediatrician
A
AVSD with a small VSD component
No transfer
No transfer
3
Cardiologist 1
A
Mild increase in flow velocity in the aortic arch with normal dimension
No transfer
No transfer
4
Cardiologist 3
B
Mild TOF
No transfer
No transfer
5
Cardiologist 1
B
Mild aortic arch hypoplasia
No transfer
No transfer
6
Anaesthesiologist 2
A
Pulmonary hypertension
No transfer
No transfer
7
Cardiologist 1
A
Pulmonary hypertension
No transfer
No transfer
8
Anaesthesiologist 1
A
Stable TOF with hypoplastic pulmonary artery tree
No transfer
Transfer
9
Cardiologist 1
B
Situs ambiguus and mild hypoplasia of the aortic arch
No transfer
No transfer
Examining Physician User Experience
Average physician experience across all categories was rated at 4.3 (3.8–4.7), with a non-statistically significant difference between physicians with and without echocardiographic experience at 4.5 (4.0–4.9) and 3.8 (2.7–4.9), respectively (p = 0.11).
Mentor User Experience
The average mentor experience across all categories was 4.6 (min 3.8 and max 5.5), with a standard deviation of 0.7. There was a non-significant difference in mentor rating between the examinations performed by physicians with and without echocardiographic experience, 4.8 (4.3–5.2) and 4.2 (3.3–5.1), respectively (p = 0.13).
Parent Experience
All parents answered that they were willing to let their child undergo remote-mentored echocardiography if a pediatric cardiologist was not available at their hospital in a similar situation.
Examination Time
The mean examination time was 26.8 min (range 16 to 50 min), with a standard deviation of 9.7 min. No differences were found between the examiners with and without previous echocardiographic experience.
Image Transfer Quality
The rating of the transmission speed of echocardiographic images varied between 1 and 5, with a mean of 2.9 and a standard deviation of 1.4.
Discussion
This study is the first to evaluate the feasibility of remote-mentored echocardiography performed by physicians without experience with imaging of CHD and variable levels of echocardiographic training. All 15 newborns underwent successful evaluation of cardiac anatomy by remote-mentored echocardiography, and no CCHD was missed. One newborn with stable TOF with hypoplastic pulmonary arteries was selected for transfer although not strictly indicated.
Our findings support the safety of using remote-mentoring echocardiography to evaluate newborns with signs and symptoms warranting an echocardiogram [4,7–10,12,14]. Unnecessary transportation has been reported in other studies utilizing remote mentoring of a relatively inexperienced examinator for a complete evaluation of cardiac anatomy in newborns [4,5]. The reason for the single erroneous suggestion of transfer of the child with TOF was the inability to achieve diagnostic images of the aortic arch. This newborn had an arterial oxygen saturation of 85% the first week of life and had already been transported from a local hospital due to uncertainty of the cardiac anatomy. In a multidisciplinary meeting, it was decided to perform a percutaneous balloon valvuloplasty on the pulmonary valve in the second week of life. This newborn did not need urgent transferral, but opting for transportation was not completely wrong, and could well have been the decision, even in the presence of a local pediatric cardiologist.
The total specificity in our study was 89%, which can be translated into the test’s ability to avoid unnecessary transportation. Other studies with bigger sample sizes report numbers between 75% and 100% specificity [3–5]. These studies included a higher proportion of patients with normal cardiac anatomy, and one could argue that it is easier to exclude CCHD in these cohorts. Our remote-mentoring study faces a challenge in terms of specificity, as only three newborns exhibited normal cardiac anatomy and the distribution of CCHD and non-CCHD was 1:1.5. The specificity might have been higher with a larger study sample and a higher proportion of patients with normal cardiac anatomy.
In previous studies, pediatricians [3–5], neonatologists [6], and adult sonographers [7,9,12,14] with training in the imaging of infants performed the echocardiograms. The significance of echocardiographic or sonographic experience in remote-mentored neonatal echocardiography has not been previously evaluated. Physicians who typically are available during on-call hours were included, in efforts to produce data valid for real life in rural Norwegian hospitals. No introduction to neonatal echocardiography was provided. We do not have sufficient statistical power to calculate sensitivity and specificity across different specialties and levels of echocardiographic experience in our sample size. In our data, more experienced echocardiographers tended to rate their own performance higher. Unsurprisingly, they were also rated easier to mentor, compared to physicians with less experience, although not statistically significant. This highlights the importance of studying less experienced examinators, in an effort to identify the minimum requirement of echocardiographic experience to perform this task with sufficient accuracy and precision.
Large variation was shown in the mentor evaluation of image transfer quality throughout the study. The 4G network link applied had download and upload speeds of around 60 Mbps, which is another order of magnitude compared to earlier publications counting one single or a few ISDN lines of 0.3 Mbps [4,5]. Despite capable download and upload speeds, we experienced randomly distributed problems with disconnection of the remote-mentoring conference, degrading of transmitted image quality, and frozen image transmission. This could introduce a random bias in our small sample size when comparing the subgroups of examining physicians. Unfortunately, we failed at identifying the limiting factor to image transmission, during this project, but applying new 5G networks could probably be of interest in future studies.
The study is designed to mimic the scenario of a newborn with suspected CCHD in a local hospital, a situation in which the remote-mentoring pair might experience as stressful. In our study the newborns were stabilized, the patients with TGA had undergone percutaneous atrial septostomy and the helping nurses and parents were calm. In a real situation, the stress could possibly lead to different results and more frequent transportation of newborns.
Some publications utilized the store and forward technique, where an echocardiogram was acquired by a relatively inexperienced echocardiographer before sending it for expert review, and possibly a subsequent additional real-time remote mentored session [4–7,9,10]. Two publications showed that real-time mentoring became less used, in favour of store and forward, as echocardiographers became more experienced [6,10]. For the physicians in our study, inexperienced with imaging CHD, real-time mentoring is probably the most suitable method. Another drawback to using store and forward is the longer time to evaluation in time-critical situations. The examination times in the present study demonstrate that our method is feasible for use in clinical situations where a prompt echocardiographic evaluation is important.
Study Limitations
The small sample size of this study is a limitation when interpreting the results. As mentioned above, the study setup with calm parents and healthcare professionals supporting the newborn, allowed for good imaging conditions not necessarily achievable in real clinical settings. On the other hand, most sonographers working in Norway are used to ultrasound machines from General Electric Company. Therefore, they are not acquainted with the Philips machine applied and this may have led to poorer results of the study, for example longer examination times.
Future Perspectives
Although the studied technique seems feasible, we have yet to study the effects of remote-mentored echocardiography performed by physicians without experience with imaging of CHD used in clinical working life. With increased sample sizes, sensitivity, specificity, and harder endpoints could be evaluated.
Conclusion
This study shows that remote-mentored echocardiography of newborns performed by physicians without experience in imaging of CHD is feasible. The method successfully identified newborns with CCHD, and thus, could improve the management of newborns with suspected CCHD in hospitals without on-call availability of a pediatric cardiologist. Further clinical research is needed to evaluate the potential benefits in patient care and cost savings for the healthcare system and to identify the exact level of training to serve as the remote-mentored examiner.
AbbreviationsCHD
Congenital heart defect
CCHD
Critical congenital heart defect
CoA
Coarctation of the aorta
MD
Medical Doctor
NICU
Neonatal intensive care unit
TGA
Transposition of the great arteries
TOF
Tetralogy of Fallot
None.
Funding Statement
This study was funded through a grant from the European Union’s Project Horizon 2020 and 5G HEART, under Grant Agreement Number 857034 [15]; and the Norwegian Association for Children with Congenital Heart Disease.
Author Contributions
Solvin has designed the study, collected and processed the data, and written the manuscript. Diab contributed to data collection and critical review of the manuscript. Elle contributed to data interpretation and critical review of the manuscript. Holmstrøm contributed to data interpretation and critical review of the manuscript. Brun contributed by study design, data interpretation and critical review of the manuscript. All authors read and approved the final manuscript.
Availability of Data and Materials
The data will be available upon reasonable request.
Ethics Approval
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. All participants were included by informed written consent. The study was approved by Regional Ethics Committee of South-Eastern Norway, application number 126124.
Conflicts of Interest
Philips cardiovascular ultrasound in Eindhoven, Netherlands was part of the same project and provided some technical support according to the project plan. Philips is the owner of the REACTS software that was used in the study. Oslo University Hospital purchased a time-limited research license. None of the authors have any conflicts of interest to declare.
ReferencesHoffman, J. I., Kaplan, S. (2002). The incidence of congenital heart disease. Journal of the American College of Cardiology,39(12),1890–1900.; 12084585van der Linde, D., Konings, E. E., Slager, M. A., Witsenburg, M., Helbing, W. A.et al. (2011). Birth prevalence of congenital heart disease worldwide: A systematic review and meta-analysis. Journal of the American College of Cardiology,58(21),2241–2247; 22078432Mulholland, H. C., Casey, F., Brown, D., Corrigan, N., Quinn, M.et al. (1999). Application of a low cost telemedicine link to the diagnosis of neonatal congenital heart defects by remote consultation. British Medical Journal Heart,82(2),217–221.Grant, B., Morgan, G. J., McCrossan, B. A., Crealey, G. E., Sands, A. J.et al. (2010). Remote diagnosis of congenital heart disease: The impact of telemedicine. Archives of Disease in Childhood,95(4),276–280; 19948507McCrossan, B. A., Grant, B., Morgan, G. J., Sands, A. J., Craig, B.et al. (2008). Diagnosis of congenital heart disease in neonates by videoconferencing: An eight-year experience. Journal of Telemedicine and Telecare,14(3),137–140; 18430281Mattos, S. D. S., Moser, L., Diogenes, T. C. P., Severi, R., de Araújo, J. S. S., et al. (2020). Echocardiography learning by pediatricians while screening congenital heart disease with the aid of telemedicine. Telemedicine Journal and e-Health,26(12),1449–1454; 32212988Makkar, A., Milsten, J., McCoy, M., Szyld, E. G., Lapadula, M. C.et al. (2021). Tele-echocardiography for congenital heart disease screening in a level II neonatal intensive care unit with hybrid telemedicine system. Journal of Telemedicine and Telecare,27(10),1136–1142.Webb, C. L., Waugh, C. L., Grigsby, J., Busenbark, D., Berdusis, K.et al. (2013). Impact of telemedicine on hospital transport, length of stay, and medical outcomes in infants with suspected heart disease: A multicenter study. Journal American Society of Echocardiography,26(9),1090–1098.Randolph, G. R., Hagler, D. J., Khandheria, B. K., Lunn, E. R., Cook, W. J.et al. (1999). Remote telemedical interpretation of neonatal echocardiograms: Impact on clinical management in a primary care setting. Journal of the American College of Cardiology,34(1),241–245; 10400017Krishnan, A., Fuska, M., Dixon, R., Sable, C. A. (2014). The evolution of pediatric tele-echocardiography: 15-year experience of over 10,000 transmissions. Journal of Telemedicine and Telecare,20(8),681–686.de Araújo, J. S., Regis, C. T., Gomes, R. G., Mourato, F. A., Mattos, S. D. (2016). Impact of telemedicine in the screening for congenital heart disease in a center from Northeast Brazil. Journal of Tropical Pediatrics,62(6),471–476.Sable, C. A., Cummings, S. D., Pearson, G. D., Schratz, L. M., Cross, R. C.et al. (2002). Impact of telemedicine on the practice of pediatric cardiology in community hospitals. Pediatrics,109(1),e3.Solvin, H., Lippert, M., Holmstrøm, H., Elle, OJ., Brun, H. (2023). Real-time remote expert-guided echocardiography by medical students. The Ultrasound Journal,15(1),28; 37266713Woodson, K. E., Sable, C. A., Cross, R. R., Pearson, G. D., Martin, G. R. (2004). Forward and store telemedicine using motion pictures expert group: A novel approach to pediatric tele-echocardiography. Journal American Society of Echocardiography,17(11),1197–1200.Homepage 5G HEART, Horizon (2020). European Union. www.5gheart.org