Ethical issues on artificial intelligence in radiology: how is it reported in research articles? The current state and future directions
DOI:
https://doi.org/10.20883/medical.e513Keywords:
Ethics, Radiology, Artificial Intelligence, Diagnostics, Medical ImagingAbstract
Background. This paper evaluates the status of reporting information related to the usage and ethical issues of artificial intelligence (AI) procedures in clinical trial (CT) papers focussed on radiology issues as well as other (non-trial) original radiology articles (OA).
Material and Methods. The evaluation was performed by three independent observers who were, respectively physicist, physician and computer scientist. The analysis was performed for two groups of publications, i.e., for CT and OA. Each group included 30 papers published from 2018 to 2020, published before guidelines proposed by Liu et al. (Nat Med. 2020; 26:1364-1374). The set of items used to catalogue and to verify the ethical status of the AI reporting was developed using the above-mentioned guidelines.
Results. Most of the reviewed studies, clearly stated their use of AI methods and more importantly, almost all tried to address relevant clinical questions. Although in most of the studies, patient inclusion and exclusion criteria were presented, the widespread lack of rigorous descriptions of the study design apart from a detailed explanation of the AI approach itself is noticeable. Few of the chosen studies provided information about anonymization of data and the process of secure data sharing. Only a few studies explore the patterns of incorrect predictions by the proposed AI tools and their possible reasons.
Conclusion. Results of review support idea of implementation of uniform guidelines for designing and reporting studies with use of AI tools. Such guidelines help to design robust, transparent and reproducible tools for use in real life.
Downloads
References
Kruskal JB, Eisenberg R, Sosna J, Yam CS, Kruskal JD, Boiselle PM. Quality improvement in radiology: basic principles and tools required to achieve success. RadioGraphics 2011; 31(6):1499-1509. DOI: 10.1148/rg.316115501
Academy of Royal Medical Colleges: Artificial Intelligence in Healthcare. The report published on 28.01.2019 at the Academy website: http://www.aomrc.org.uk/reports-guidance/artificial-intelligence-in-healthcare
Hosny A, Parmar C, Quackenbush J, Schwartz LH, Aerts HJWL. Artificial intelligence in radiology. Nat Rev Cancer 2018; 18:500–10. DOI: 10.1038/s41568-018-0016-5
Kazmierska J, Hope A, Spezi E, Beddar S, Nailon WH, Osong B, et al. From multisource data to clinical decision aids in radiation oncology: the need for a clinical data science community. Radiotherapy and Oncology 2020; (Online ahead of print). DOI: 10.1016/j.radonc.2020.09.054
Neri E, Coppola F, Miele V, Bibbolino C, Grassi R. Artificial intelligence: Who is responsible for the diagnosis? La Radiol Med 2020; 125:517–521. DOI: 10.1007/s11547-020-01135-9
Geis JR, Brady A, Wu CC, Spencer J, Ranschaert E, Jaremko JL, et al. Ethics of artificial intelligence in radiology: summary of the joint European and North American multisociety statement. Radiology 2019; 293:436-440. DOI: 10.1148/radiol.2019191586
Brady AP, Neri E. Artificial intelligence in radiology – ethical considerations. Diagnostics (Basel) 2020; 10:231. DOI: 10.3390/diagnostics10040231
Kohli M, Geis R. Artificial Intelligence and Radiology. J Am Coll Radiol. 2018; 15:1317–1319. DOI: 10.1016/j.jacr.2018.05.020
Liu X, Rivera SC, Moher D, Calvert MJ, Denniston AK, SPIRIT-AI and CONSORT-AI Working Group. Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension. Nat Med. 2020; 26:1364-1374. DOI: 10.1038/s41591-020-1034-x.
Baskaran L, Ying X, Xu Z, Al'Aref SJ, Lee BC, Lee SE, et al. Machine learning insight into the role of imaging and clinical variables for the prediction of obstructive coronary artery disease and revascularization: An exploratory analysis of the CONSERVE study. PLoS One. 2020; 15(6):e0233791. DOI: 10.1371/journal.pone.0233791.
Leung DG, Bocchieri AE, Ahlawat S, Jacobs MA, Parekh VS, Braverman V, et al. Longitudinal functional and imaging outcome measures in FKRP limb-girdle muscular dystrophy. BMC Neurol. 2020; 20(1):196. DOI: 10.1186/s12883-020-01774-5.
Eresen A, Li Y, Yang J, Shangguan J, Velichko Y, Yaghmai V, et al. Preoperative assessment of lymph node metastasis in Colon Cancer patients using machine learning: a pilot study. Cancer Imaging. 2020; 20(1):30. DOI: 10.1186/s40644-020-00308-z.
Huang Z, Liu D, Chen X, Yu P, Wu J, Song B, et al. Retrospective imaging studies of gastric cancer: Study protocol clinical trial (SPIRIT Compliant). Medicine (Baltimore). 2020; 99(8):e19157. DOI: 10.1097/MD.0000000000019157.
Eisenberg E, McElhinney PA, Commandeur F, Chen X, Cadet S, Goeller M, et al. Deep learning-based quantification of epicardial adipose tissue volume and attenuation predicts major adverse cardiovascular events in asymptomatic subjects. Circ Cardiovasc Imaging. 2020; 13(2):e009829. DOI: 10.1161/CIRCIMAGING.119.009829.
Wang D, Xu J, Zhang Z, Li S, Zhang X, Zhou Y, et al. Evaluation of rectal cancer circumferential resection margin using faster region-based convolutional neural network in high-resolution magnetic resonance images. Dis Colon Rectum. 2020; 63(2):143-151. DOI: 10.1097/DCR.0000000000001519.
Hilbert A, Ramos LA, van Os HJA, Olabarriaga SD, Tolhuisen ML, Wermer MJH, et al. Data-efficient deep learning of radiological image data for outcome prediction after endovascular treatment of patients with acute ischemic stroke. Comput Biol Med. 2019; 115:103516. DOI: 10.1016/j.compbiomed.2019.103516.
Sheth SA, Lopez-Rivera V, Barman A, Grotta JC, Yoo AJ, Lee S, et al. Machine learning-enabled automated determination of acute ischemic core from computed tomography angiography. Stroke. 2019; 50(11):3093-3100. DOI: 10.1161/STROKEAHA.119.026189.
Bielak L, Wiedenmann N, Nicolay NH, Lottner T, Fischer J, Bunea H, et al. Automatic tumor segmentation with a convolutional neural network in multiparametric MRI: influence of distortion correction. Tomography. 2019; 5(3):292-299. DOI: 10.18383/j.tom.2019.00010.
Bhattarai S, Klimov S, Aleskandarany MA, Burrell H, Wormall A, Green AR, et al. Machine learning-based prediction of breast cancer growth rate in vivo. Br J Cancer. 2019; 121(6):497-504. DOI: 10.1038/s41416-019-0539-x.
Toivonen J, Montoya Perez I, Movahedi P, Merisaari H, Pesola M, Taimen P, et al. Radiomics and machine learning of multisequence multiparametric prostate MRI: Towards improved non-invasive prostate cancer characterization. PLoS One. 2019; 14(7):e0217702. DOI: 10.1371/journal.pone.0217702.
van Gastel MDA, Edwards ME, Torres VE, Erickson BJ, Gansevoort RT, Kline TL. Automatic measurement of kidney and liver volumes from MR images of patients affected by autosomal dominant polycystic kidney disease. J Am Soc Nephrol. 2019; 30(8):1514-1522. DOI: 10.1681/ASN.2018090902.
Kurata A, Fukuyama N, Hirai K, Kawaguchi N, Tanabe Y, Okayama H, et al. On-site computed tomography-derived fractional flow reserve using a machine-learning algorithm - clinical effectiveness in a retrospective multicenter cohort. Circ J. 2019; 83(7):1563-1571. DOI: 10.1253/circj.CJ-19-0163.
Park A, Chute C, Rajpurkar P, Lou J, Ball RL, Shpanskaya K, et al. Deep learning-assisted diagnosis of cerebral aneurysms using the HeadXNet model. JAMA Netw Open. 2019; 2(6):e195600. DOI: 10.1001/jamanetworkopen.2019.5600.
Shayesteh SP, Alikhassi A, Fard Esfahani A, Miraie M, Geramifar P, Bitarafan-Rajabi A, et al. Neo-adjuvant chemoradiotherapy response prediction using MRI based ensemble learning method in rectal cancer patients. Phys Med. 2019; 62:111-119. DOI: 10.1016/j.ejmp.2019.03.013.
Fiehler J, Thomalla G, Bernhardt M, Kniep H, Berlis A, Dorn F, et al. ERASER. Stroke. 2019; 50(5):1275-1278. DOI: 10.1161/STROKEAHA.119.024858.
Gates EDH, Lin JS, Weinberg JS, Hamilton J, Prabhu SS, Hazle JD, et al. Guiding the first biopsy in glioma patients using estimated Ki-67 maps derived from MRI: conventional versus advanced imaging. Neuro Oncol. 2019; 21(4):527-536. DOI: 10.1093/neuonc/noz004.
Shan QY, Hu HT, Feng ST, Peng ZP, Chen SL, Zhou Q, et al. CT-based peritumoral radiomics signatures to predict early recurrence in hepatocellular carcinoma after curative tumor resection or ablation. Cancer Imaging. 2019; 19(1):11. DOI: 10.1186/s40644-019-0197-5.
Cho H, Lee JG, Kang SJ, Kim WJ, Choi SY, Ko J, et al. Angiography-based machine learning for predicting fractional flow reserve in intermediate coronary artery lesions. J Am Heart Assoc. 2019; 8(4):e011685. DOI: 10.1161/JAHA.118.011685.
Tu Y, Ortiz A, Gollub RL, Cao J, Gerber J, Lang C, et al. Multivariate resting-state functional connectivity predicts responses to real and sham acupuncture treatment in chronic low back pain. Neuroimage Clin. 2019; 23:101885. DOI: 10.1016/j.nicl.2019.101885.
Togo R, Hirata K, Manabe O, Ohira H, Tsujino I, Magota K, et al. Cardiac sarcoidosis classification with deep convolutional neural network-based features using polar maps. Comput Biol Med. 2019; 104:81-86. DOI: 10.1016/j.compbiomed.2018.11.008.
Stuckey TD, Gammon RS, Goswami R, Depta JP, Steuter JA, Meine FJ 3rd, et al. Cardiac phase space tomography: a novel method of assessing coronary artery disease utilizing machine learning. PLoS One. 2018; 13(8):e0198603. DOI: 10.1371/journal.pone.0198603.
Wang XH, Jiao Y, Li L. Identifying individuals with attention deficit hyperactivity disorder based on temporal variability of dynamic functional connectivity. Sci Rep. 2018; 8(1):11789. DOI: 10.1038/s41598-018-30308-w.
de Jong EEC, van Elmpt W, Rizzo S, Colarieti A, Spitaleri G, Leijenaar RTH, et al. Applicability of a prognostic CT-based radiomic signature model trained on stage I-III non-small cell lung cancer in stage IV non-small cell lung cancer. Lung Cancer. 2018; 124:6-11. DOI: 10.1016/j.lungcan.2018.07.023.
Jun Y, Eo T, Kim T, Shin H, Hwang D, Bae SH, et al. Deep-learned 3D black-blood imaging using automatic labelling technique and 3D convolutional neural networks for detecting metastatic brain tumors. Sci Rep. 2018; 8(1):9450. DOI: 10.1038/s41598-018-27742-1.
Citak-Er F, Firat Z, Kovanlikaya I, Ture U, Ozturk-Isik E. Machine-learning in grading of gliomas based on multi-parametric magnetic resonance imaging at 3T. Comput Biol Med. 2018; 99:154-160. DOI: 10.1016/j.compbiomed.2018.06.009.
Mokhtari F, Rejeski WJ, Zhu Y, Wu G, Simpson SL, Burdette JH, et al. Dynamic fMRI networks predict success in a behavioral weight loss program among older adults. Neuroimage. 2018; 173:421-433. DOI: 10.1016/j.neuroimage.2018.02.025.
Dey D, Gaur S, Ovrehus KA, Slomka PJ, Betancur J, Goeller M, et al. Integrated prediction of lesion-specific ischaemia from quantitative coronary CT angiography using machine learning: a multicentre study. Eur Radiol. 2018; 28(6):2655-2664. DOI: 10.1007/s00330-017-5223-z.
Johnson C, Price G, Khalifa J, Faivre-Finn C, Dekker A, Moore C, et al. A method to combine target volume data from 3D and 4D planned thoracic radiotherapy patient cohorts for machine learning applications. Radiother Oncol. 2018; 126(2):355-361. DOI: 10.1016/j.radonc.2017.11.015.
Ambale-Venkatesh B, Yang X, Wu CO, Liu K, Hundley WG, McClelland R, et al. Cardiovascular event prediction by machine learning: the multi-ethnic study of atherosclerosis. Circ Res. 2017; 121(9):1092-1101. DOI: 10.1161/CIRCRESAHA.117.311312.
van der Velden BHM, Janse MHA, Ragusi MAA, Loo CE, Gilhuijs KGA. Volumetric breast density estimation on MRI using explainable deep learning regression. Sci Rep. 2020; 10(1):18095. DOI: 10.1038/s41598-020-75167-6.
Hsu YC, Tsai YH, Weng HH, Hsu LS, Tsai YH, Lin YC, et al. Artificial neural networks improve LDCT lung cancer screening: a comparative validation study. BMC Cancer. 2020; 20(1):1023. DOI: 10.1186/s12885-020-07465-1.
Kwon G, Ryu J, Oh J, Lim J, Kang BK, Ahn C, et al. Deep learning algorithms for detecting and visualising intussusception on plain abdominal radiography in children: a retrospective multicenter study. Sci Rep. 2020; 10(1):17582. DOI: 10.1038/s41598-020-74653-1.
Fu F, Wei J, Zhang M, Yu F, Xiao Y, Rong D, et al. Rapid vessel segmentation and reconstruction of head and neck angiograms using 3D convolutional neural network. Nat Commun. 2020; 11(1):4829. DOI: 10.1038/s41467-020-18606-2.
Winther H, Hundt C, Ringe KI, Wacker FK, Schmidt B, Jürgens J, et al. A 3D deep neural network for liver volumetry in 3T contrast-enhanced MRI. Rofo. 2020; (Online ahead of print). DOI: 10.1055/a-1238-2887.
Shao L, Yan Y, Liu Z, Ye X, Xia H, Zhu X, et al. Radiologist-like artificial intelligence for grade group prediction of radical prostatectomy for reducing upgrading and downgrading from biopsy. Theranostics. 2020; 10(22):10200-10212. DOI: 10.7150/thno.48706.
Lång K, Dustler M, Dahlblom V, Åkesson A, Andersson I, Zackrisson S. Identifying normal mammograms in a large screening population using artificial intelligence. Eur Radiol. 2020; (Online ahead of print). DOI: 10.1007/s00330-020-07165-1.
Olive-Gadea M, Crespo C, Granes C, Hernandez-Perez M, Pérez de la Ossa N, Laredo C, et al. Deep learning based software to identify large vessel occlusion on noncontrast computed tomography. Stroke. 2020; 51(10):3133-3137. DOI: 10.1161/STROKEAHA.120.030326.
Zhang C, Zhao J, Niu J, Li D. New convolutional neural network model for screening and diagnosis of mammograms. PLoS One. 2020; 15(8):e0237674. DOI: 10.1371/journal.pone.0237674.
Kim JH, Kim JY, Kim GH, Kang D, Kim IJ, Seo J, et al. Clinical validation of a deep learning algorithm for detection of pneumonia on chest radiographs in emergency department patients with acute febrile respiratory illness. J Clin Med. 2020; 9(6):1981. DOI: 10.3390/jcm9061981.
Bhat CS, Chopra M, Andronikou S, Paul S, Wener-Fligner Z, Merkoulovitch A, et al. Artificial intelligence for interpretation of segments of whole body MRI in CNO: pilot study comparing radiologists versus machine learning algorithm. Pediatr Rheumatol Online J. 2020; 18(1):47. DOI: 10.1186/s12969-020-00442-9.
Kim MS, Park HY, Kho BG, Park CK, Oh IJ, Kim YC, et al. Artificial intelligence and lung cancer treatment decision: agreement with recommendation of multidisciplinary tumor board. Transl Lung Cancer Res. 2020; 9(3):507-514. DOI: 10.21037/tlcr.2020.04.11.
Drozdov I, Forbes D, Szubert B, Hall M, Carlin C, Lowe DJ. Supervised and unsupervised language modelling in Chest X-Ray radiological reports. PLoS One. 2020; 15(3):e0229963. DOI: 10.1371/journal.pone.0229963.
Kann BH, Hicks DF, Payabvash S, Mahajan A, Du J, Gupta V, et al. Multi-institutional validation of deep learning for pretreatment identification of extranodal extension in head and neck squamous cell carcinoma. J Clin Oncol. 2020; 38(12):1304-1311. DOI: 10.1200/JCO.19.02031.
Mall S, Brennan PC, Mello-Thoms C. Can a machine learn from radiologists' visual search behaviour and their interpretation of mammograms - a deep-learning study. J Digit Imaging. 2019; 32(5):746-760. DOI: 10.1007/s10278-018-00174-z.
Seidler M, Forghani B, Reinhold C, Pérez-Lara A, Romero-Sanchez G, Muthukrishnan N, et al. Dual-energy CT texture analysis with machine learning for the evaluation and characterization of cervical lymphadenopathy. Comput Struct Biotechnol J. 2019; 17:1009-1015. DOI: 10.1016/j.csbj.2019.07.004.
Mohammadi S, Mohammadi M, Dehlaghi V, Ahmadi A. Automatic segmentation, detection, and diagnosis of abdominal aortic aneurysm (AAA) using convolutional neural networks and hough circles algorithm. Cardiovasc Eng Technol. 2019; 10(3):490-499. DOI: 10.1007/s13239-019-00421-6.
Akselrod-Ballin A, Chorev M, Shoshan Y, Spiro A, Hazan A, Melamed R, et al. Predicting breast cancer by applying deep learning to linked health records and mammograms. Radiology. 2019; 292(2):331-342. DOI: 10.1148/radiol.2019182622.
Yanagawa M, Niioka H, Hata A, Kikuchi N, Honda O, Kurakami H, et al. Application of deep learning (3-dimensional convolutional neural network) for the prediction of pathological invasiveness in lung adenocarcinoma: A preliminary study. Medicine (Baltimore). 2019; 98(25):e16119. DOI: 10.1097/MD.0000000000016119.
Gan K, Xu D, Lin Y, Shen Y, Zhang T, Hu K, et al. Artificial intelligence detection of distal radius fractures: a comparison between the convolutional neural network and professional assessments. Acta Orthop. 2019; 90(4):394-400. DOI: 10.1080/17453674.2019.1600125.
Lassau N, Estienne T, de Vomecourt P, Azoulay M, Cagnol J, Garcia G, et al. Five simultaneous artificial intelligence data challenges on ultrasound, CT, and MRI. Diagn Interv Imaging. 2019; 100(4):199-209. DOI: 10.1016/j.diii.2019.02.001.
Tran GS, Nghiem TP, Nguyen VT, Luong CM, Burie JC. Improving accuracy of lung nodule classification using deep learning with focal loss. J Healthc Eng. 2019; 2019:5156416. DOI: 10.1155/2019/5156416.
Ko SY, Lee JH, Yoon JH, Na H, Hong E, Han K, et al. Deep convolutional neural network for the diagnosis of thyroid nodules on ultrasound. Head Neck. 2019; 41(4):885-891. DOI: 10.1002/hed.25415.
Li S, Xiao J, He L, Peng X, Yuan X. The tumor target segmentation of nasopharyngeal cancer in CT images based on deep learning methods. Technol Cancer Res Treat. 2019; 18:1533033819884561. DOI: 10.1177/1533033819884561.
Hae H, Kang SJ, Kim WJ, Choi SY, Lee JG, Bae Y, et al. Machine learning assessment of myocardial ischemia using angiography: Development and retrospective validation. PLoS Med. 2018; 15(11):e1002693. DOI: 10.1371/journal.pmed.1002693.
Adams M, Chen W, Holcdorf D, McCusker MW, Howe PD, Gaillard F. Computer vs human: Deep learning versus perceptual training for the detection of neck of femur fractures. J Med Imaging Radiat Oncol. 2019; 63(1):27-32. DOI: 10.1111/1754-9485.12828.
Chilamkurthy S, Ghosh R, Tanamala S, Biviji M, Campeau NG, Venugopal VK, et al. Deep learning algorithms for detection of critical findings in head CT scans: a retrospective study. Lancet. 2018; 392(10162):2388-2396. DOI: 10.1016/S0140-6736(18)31645-3.
Sumathipala Y, Lay N, Turkbey B, Smith C, Choyke PL, Summers RM. Prostate cancer detection from multi-institution multiparametric MRIs using deep convolutional neural networks. J Med Imaging (Bellingham). 2018; 5(4):044507. DOI: 10.1117/1.JMI.5.4.044507.
Tong N, Gou S, Yang S, Ruan D, Sheng K. Fully automatic multi-organ segmentation for head and neck cancer radiotherapy using shape representation model constrained fully convolutional neural networks. Med Phys. 2018; 45(10):4558-4567. DOI: 10.1002/mp.13147.
Kim T, Heo J, Jang DK, Sunwoo L, Kim J, Lee KJ, et al. Machine learning for detecting moyamoya disease in plain skull radiography using a convolutional neural network. EBioMedicine. 2019; 40:636-642. DOI: 10.1016/j.ebiom.2018.12.043.
Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med. 2009; 6(7): e1000097. DOI:10.1371/journal.pmed1000097
Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33 (1):159–174. DOI:10.2307/2529310.
McKinney MS, Sieniek M, Godbole V, Godwin J, Antropova N, Ashrafian H, et al. International evaluation of an AI system for breast cancer screening. Nature 2020; 577:89-94. DOI: 10.1038/s41586-019-1799-6
Zbontar J, Knoll F, Sriram A, Muckley M, Bruno M, Defazio A, Parente M, et al. fastMRI: an open dataset and benchmarks for accelerated MRI. arXiv:1811.08839v2
Benaich N, Hogarth I. https://www.stateof.ai access: 19 Nov. 2020
Haibe-Kains B, Adam GA, Hosny A, Khodakarami F, Massive Analysis Quality Control (MAQC) Society Board of Directors, Waldron L, et al. Transparency and reproducibility in artificial intelligence. Nature 2020; 586:14-16. DOI: 10.1038/s41586-020-2766-y
Downloads
Published
Issue
Section
License
Copyright (c) 2021 The copyright to the submitted manuscript is held by the Author, who grants the Journal of Medical Science (JMS) a nonexclusive licence to use, reproduce, and distribute the work, including for commercial purposes.
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
