Quality control of B-lines analysis in stress Echo 2020

Abstract

Background: The effectiveness trial "Stress echo (SE) 2020" evaluates novel applications of SE in and beyond coronary artery disease. The core protocol also includes 4-site simplified scan of B-lines by lung ultrasound, useful to assess pulmonary congestion. Purpose: To provide web-based upstream quality control and harmonization of B-lines reading criteria. Methods: 60 readers (all previously accredited for regional wall motion, 53 B-lines naive) from 52 centers of 16 countries of SE 2020 network read a set of 20 lung ultrasound video-clips selected by the Pisa lab serving as reference standard, after taking an obligatory web-based learning 2-h module (http://se2020.altervista.org). Each test clip was scored for B-lines from 0 (black lung, A-lines, no B-lines) to 10 (white lung, coalescing B-lines). The diagnostic gold standard was the concordant assessment of two experienced readers of the Pisa lab. The answer of the reader was considered correct if concordant with reference standard reading ±1 (for instance, reference standard reading of 5 B-lines; correct answer 4, 5, or 6). The a priori determined pass threshold was 18/20 (≥ 90%) with R value (intra-class correlation coefficient) between reference standard and recruiting center) > 0.90. Inter-observer agreement was assessed with intra-class correlation coefficient statistics. Results: All 60 readers were successfully accredited: 26 (43%) on first, 24 (40%) on second, and 10 (17%) on third attempt. The average diagnostic accuracy of the 60 accredited readers was 95%, with R value of 0.95 compared to reference standard reading. The 53 B-lines naive scored similarly to the 7 B-lines expert on first attempt (90 versus 95%, p = NS). Compared to the step-1 of quality control for regional wall motion abnormalities, the mean reading time per attempt was shorter (17 ± 3 vs 29 ± 12 min, p <.01), the first attempt success rate was higher (43 vs 28%, p < 0.01), and the drop-out of readers smaller (0 vs 28%, p <.01). Conclusions: Web-based learning is highly effective for teaching and harmonizing B-lines reading. Echocardiographers without previous experience with B-lines learn quickly.