Automatic lung cancer subtyping using rapid on-site evaluation slides and serum biological markers

Abstract

Background

Rapid on-site evaluation (ROSE) plays an important role during transbronchial sampling, providing an intraoperative cytopathologic evaluation. However, the shortage of cytopathologists limits its wide application. This study aims to develop a deep learning model to automatically analyze ROSE cytological images.

Methods

The hierarchical multi-label lung cancer subtyping (HMLCS) model that combines whole slide images of ROSE slides and serum biological markers was proposed to discriminate between benign and malignant lesions and recognize different subtypes of lung cancer. A dataset of 811 ROSE slides and paired serum biological markers was retrospectively collected between July 2019 and November 2020, and randomly divided to train, validate, and test the HMLCS model. The area under the curve (AUC) and accuracy were calculated to assess the performance of the model, and Cohen’s kappa (κ) was calculated to measure the agreement between the model and the annotation. The HMLCS model was also compared with professional staff.

Results

The HMLCS model achieved AUC values of 0.9540 (95% confidence interval [CI]: 0.9257–0.9823) in malignant/benign classification, 0.9126 (95% CI: 0.8756–0.9365) in malignancy subtyping (non-small cell lung cancer [NSCLC], small cell lung cancer [SCLC], or other malignancies), and 0.9297 (95% CI: 0.9026–0.9603) in NSCLC subtyping (lung adenocarcinoma [LUAD], lung squamous cell carcinoma [LUSC], or NSCLC not otherwise specified [NSCLC-NOS]), respectively. In total, the model achieved an AUC of 0.8721 (95% CI: 0.7714–0.9258) and an accuracy of 0.7184 in the six-class classification task (benign, LUAD, LUSC, NSCLC-NOS, SCLC, or other malignancies). In addition, the model demonstrated a κ value of 0.6183 with the annotation, which was comparable to cytopathologists and superior to trained bronchoscopists and technicians.

Conclusion

The HMLCS model showed promising performance in the multiclassification of lung lesions or intrathoracic lymphadenopathy, with potential application to provide real-time feedback regarding preliminary diagnoses of specimens during transbronchial sampling procedures.

Clinical trial number

Not applicable.

Introduction

Lung cancer is the leading cause of cancer death worldwide [1]. Accurate pathological subtyping is essential for guiding personalized therapy strategies, especially for small biopsies and cytology specimens from inoperable lung cancer patients [2]. Transbronchial sampling is the most common modality to obtain specimens of lung lesions and intrathoracic lymphadenopathy for the accurate diagnosis, and has proved to be safe and effective [3,4,5,6]. However, whether the specimens obtained were satisfactory and sufficient was indeterminate during the procedure. If the specimens obtained fail to meet the needs for an accurate diagnosis, patients may need to undergo additional invasive examinations. To overcome this problem, rapid on-site evaluation (ROSE) has emerged as a crucial auxiliary technique for transbronchial sampling [7,8,9,10]. With ROSE, the specimen undergoes an immediate cytopathologic evaluation, and the bronchoscopist receives feedback regarding specimen quality and preliminary diagnosis.

For ROSE, comprehensive evaluation of specimens is typically performed by experienced cytopathologists directly in the bronchoscopy suite. However, the widespread adoption of ROSE faces a significant challenge due to the shortage of cytopathologists, especially in areas with less medical resources. Meanwhile, the manual evaluation process is time-consuming and depends on the experience of cytopathologists [11]. With the development of artificial intelligence (AI), various methods have been proposed for automatic pathological image assessments [12,13,14,15]. Nevertheless, only a few studies have explored AI-aided ROSE for transbronchial sampling [16,17,18,19]. Besides, existing studies only focused on the differential diagnosis of benign and malignant, and none have investigated the classification of lung cancer subtypes. Moreover, it is essential to acknowledge that the diagnosis of lung cancer should not solely rely on image analysis. Recently, some researches have proved that AI models that combine medical imaging with serum biological markers can promote the accuracy of lung cancer subtyping [20, 21].

In this study, we proposed a hierarchical multi-label lung cancer subtyping (HMLCS) model that combines whole slide images (WSIs) of ROSE slides and serum biological markers, with the purpose of achieving performance comparable to cytopathologists in discriminating between benign and malignant lesions and recognizing different subtypes of lung cancer.

Materials and methods

Study design

This is a retrospective observational study. The overview of this study is illustrated in Fig. 1. Lung cancer is mainly divided into non-small cell lung cancer (NSCLC) and small cell lung cancer (SCLC). About 80–85% of lung cancer cases are NSCLC, and the subtypes of NSCLC include lung adenocarcinoma (LUAD), lung squamous cell carcinoma (LUSC), and some other types. To achieve better performance in classifying the lung cancer subtypes as mentioned above, we proposed the HMLCS model which could be divided into three modules: benign or malignant classification (B/M) module, malignancy subtyping (M-sub) module, and NSCLC subtyping (NSCLC-sub) module. Specifically, WSIs of ROSE slides and paired serum biological markers were input to the HMLCS model. We first classified them as benign or malignant with the B/M module. The classification stopped here if the output of this step was benign. Otherwise, the input was forwarded to the M-sub module and was classified into three sub-categories: NSCLC, SCLC or other malignancies. Finally, if the output of the second step was NSCLC, we took one more step with the NSCLC-sub module to classify the input into three deeper sub-categories: LUAD, LUSC or NSCLC not otherwise specified (NSCLC-NOS). WSIs of ROSE slides and paired serum biological markers were retrospectively collected and randomly divided into training, validation and test sets to develop and test our model. In addition, we conducted a comprehensive comparison between our model and professional staff (including cytopathologists, trained bronchoscopists and technicians) to further evaluate the performance of our model. This study was approved by the Ethics Committee of Shanghai Chest Hospital (No. KS2023), and the requirement for informed consent was waived. The study was performed in accordance with the Declaration of Helsinki.

Fig. 1

Overview of this study. (A) The strategy of the HMLCS model. The HMLCS model can be divided into three modules: benign or malignant classification (B/M) module, malignancy subtyping (M-sub) module, and NSCLC subtyping (NSCLC-sub) module. (B) The details of the HMLCS model. The WSIs were segmented to patches, and then the patches were encoded into features by a pre-trained network. For each module, the attention layer was utilized to calculate attention scores for each patch feature, and the features of patches within a WSI were aggregated according to attention scores. Finally, the aggregated feature was concatenated with five serum biological markers for the classifier to perform the classification. (C) The potential application of the HMLCS model. The HMLCS model could provide real-time feedback regarding primary diagnoses of specimens during transbronchial sampling. LUAD: lung adenocarcinoma, LUSC: lung squamous cell carcinoma, NOS: not otherwise specified, NSCLC: non-small cell lung cancer, SCLC: small cell lung cancer, ROSE: rapid on-site evaluation, WSIs: whole slide images

Data collection

The ROSE slides of patients with lung lesions or intrathoracic lymphadenopathy who underwent transbronchial sampling in Shanghai Chest Hospital between July 2019 and November 2020 were retrospectively collected. One of the following transbronchial sampling techniques was performed: transbronchial needle aspiration (TBNA), transbronchial lung biopsy (TBLB), and transbronchial biopsy (TBB). TBB was performed in endobronchial lung lesions, TBLB was performed in lung lesions invisible during bronchoscopy, and TBNA was performed in intrathoracic lymphadenopathy and central lung lesions adjacent to airways. During the procedure, the specimen was stamped on glass slides for immediate ROSE. For specimens from forceps, the specimens were picked up using tweezers and spread in concentric circles about 1 cm in diameter on the glass slides. For specimens from brushes, the brush tip was pushed out, and the specimens were smeared on the glass slides, forming a rectangle of about 1 cm × 2 cm. For specimens from needles, the tissue was picked up using tweezers and spread in concentric circles, and one drop of the liquid specimen was placed on the glass slide and spread by pressing using another slide. The slides were air-dried and stained using a Diff-Quik stain kit (Baso Diagnostics Inc., Zhuhai, China). The slides were immersed in solution A for 20–30 s and rinsed with phosphate-buffered saline (PBS), followed by immersion in solution B for 20–30 s and rinsed with PBS. Owing to the convenience of Diff-Quik staining, it finds extensive application in ROSE and allows for rapid differentiation of benign and diverse malignancy subtypes (Fig. 2). The remaining specimens after ROSE slide preparation were sent for pathological examinations.

Only patients with a diagnostic transbronchial sampling procedure were included in this study. A biopsy that resulted in a malignant or specific benign (e.g., tuberculosis, fungal infection, etc.) process was considered diagnostic. Biopsy specimens with non-specific benign findings were considered diagnostic only if: (1) the diagnosis was confirmed by a subsequent surgery, mediastinoscopy, or CT-guided biopsy; or (2) follow-up imaging demonstrated stability or improvement of the lesion. All patients received follow up for at least 6 months. Patients with non-diagnostic transbronchial sampling procedures were excluded. All specimens in this study had undergone immunohistochemistry examinations. Specifically, cell block pathology of TBNA and histopathology of TBLB or TBB were confirmed by immunohistochemistry. Patients with unclear pathological subtypes were excluded in this study.

Fig. 2

Typical cytological images of different benign and malignant lesions from rapid on-site evaluation slides staining with Diff-Quik (×200). (A) Granulomatous structure with necrosis. (B) Pigment-laden macrophages. (C) Dense inflammatory background with neutrophil. (D) Anthracotic. (E) Lung adenocarcinoma. (F) Lung squamous cell carcinoma. (G) Small cell lung cancer. (H) Lymphoma

The ROSE slides were scanned using the Digital Micro Image Analysis System (Shanghai Aitrox Technology Corporation Limited, Shanghai, China) with a ×20 objective lens to produce WSIs. The WSIs were saved by the Microscope Image Information System (Shanghai Aitrox Technology Corporation Limited, Shanghai, China) in SVS format. WSIs with insufficient scan clarity were excluded. Since not all specimens were used for ROSE preparation and the specimens were only stamped on the slides during ROSE preparation, the diagnosis of ROSE slide and the diagnosis of corresponding transbronchial sampling procedure could be inconsistent. For example, a lung lesion was diagnosed as adenocarcinoma by transbronchial sampling, but its ROSE slide could be negative. All WSIs were labeled by one experienced cytopathologist under the guidance of final diagnoses of transbronchial sampling procedures. To guarantee annotation accuracy, the annotation for one WSI was accepted only when it was consistent with the final diagnosis. Otherwise, the WSI was excluded. The corresponding serum biological markers were also collected, including carcinoembryonic antigen (CEA), soluble fragment of cytokeratin 19 (CYFRA21-1), squamous cell carcinoma antigen (SCC), neuron-specific enolase (NSE), and carbohydrate antigen 125 (CA125). Flowchart of data collection and splitting was shown in Fig. 3.

Fig. 3

Flowchart of data collection and splitting. LUAD: lung adenocarcinoma, LUSC: lung squamous cell carcinoma, NOS: not otherwise specified, NSCLC: non-small cell lung cancer, SCLC: small cell lung cancer, WSIs: whole slide images

Data preprocessing

The WSIs were tiled into patches of 224 × 224 pixels, and all background patches were discarded. To efficiently discard all background patches, we converted patches into binary format, with white areas denoting valid cells and black areas denoting invalid background. Patches with white areas in their binary formats were retained, while the other patches were considered as background and discarded. Although each patch may be treated as an input, the feature dimension of 224 × 224 is computationally too expensive as there may exist more than 150, 000 patches for one WSI. To improve computing efficiency, we derived low-dimensional features of size 1024 for each patch for further steps.

Deep convolutional neural network

Like CLAM [12], a method that aims to do data-efficient weakly-supervised learning for WSIs by embedding instances using a frozen encoder, we used the ResNet50 pre-trained on the ImageNet to extract low-dimensional features of patches. Then, for each WSI, we concatenated the extracted features of patches within a WSI as an input for the subsequent steps. This feature extraction procedure preserved both a local representation with low-dimensional features of each patch and a global representation with ensemble features of the whole WSI.

Attention module

The attention module aggregated the features of patches within a WSI to create WSI-level representations. Specifically, the attention module assigned one attention score to each patch to indicate its contribution to the WSI-level representation. We show such calculation in Eq. 1:

$$\:{a_i} = \frac{{exp({f_{reg}}\:(tanh(fea{t_i})\: \odot \:sigmoid\:(fea{t_i})\:)\:)}}{{\sum {\:_{j = 1}^N} exp({f_{reg}}(tanh\:(fea{t_j})\: \odot \:sigmoid\:(fea{t_j})\:)\:)}}\:$$

(1)

where $\:{a}_{i}$ represents the attention score for patch $\:i$, $\:{feat}_{i}$ stands for the feature extracted by the pre-trained ResNet50 from patch $\:i$, and $\:N$ is the number of valid patches within the WSI. We mapped the $\:{feat}_{i}$ by a $\:tanh$ function and a $\:sigmoid$ function, respectively, and the mapped outputs were aggregated via dot product. We then applied a regression function on the result and normalized it to [0, 1] by a softmax function. To make the equation easy to follow, we only show the calculation of one attention score here.

Classifier

Each module, namely the B/M module, the M-sub module, and the NSCLC-sub module, was equipped with a dedicated fully connected layer serving as a classifier. To establish these classifiers, we initially aggregated the features of patches within a WSI by computing the weighted average based on the attention scores. It is noteworthy that, for each classifier, this feature fusion process was carried out independently to ensure that the attention score was learned in a focused manner specific to the respective classifier. In addition, we also concatenated five serum biological markers (CEA, CYFRA21-1, SCC, NSE, CA125) with the aggregated feature for the classifier to perform the classification. In summary, we employed a fully connected layer as the classifier for each module and trained the classifiers using the WSI-level labels.

Training protocol

The detailed training parameters of our model were as follows: a batch size of 4, a learning rate of 1× e^− 5, an epoch of 100, an input size of 224 × 224 × 3, and a patch size of 8. Cross entropy loss was used as the loss function. The development of the model was implemented using Python version 3.6 and PyTorch version 1.3 based upon Ubuntu 16.04 on a workstation with two Xeon E5-2650 CPU (12C24T) at 2.2 GHz (Intel), 192GB RAM, and eight NVIDIA GTX 1080 Ti GPUs with 12GB GPU memory (NVidia).

Comparison between the HMLCS model and professional staff

The diagnostic performance of the HMLCS model was compared with professional staff, including two senior cytopathologists specializing in chest diseases, two bronchoscopists, and two technicians, on the same test set. The bronchoscopists and technicians were previously trained on ROSE regarding chest diseases. Professional staff independently made their predictions using the WSIs of ROSE slides and paired serum biological markers blinded to the final diagnoses.

Statistical analysis

The difference of clinical data from different datasets was compared by Mann-Whitney U test. The receiver operating characteristic (ROC) curve and the area under the curve (AUC) were introduced as graphical representations to showcase the balance between true positive and false positive rates. In addition, the accuracy and Cohen’s kappa were calculated to assess the performance of the model and professional staff. We separately measured the agreement between professional staff and the annotation, as well as the agreement between the model and the annotation on the test set. The kappa value is denoted as κ, and the formula is as follows:

$$\:{\upkappa\:}=\frac{{\text{P}}_{\text{A}}-{\text{P}}_{\text{e}}}{1-{\text{P}}_{\text{e}}}$$

with

$$\:{\text{P}}_{\text{A}}=\frac{1}{\text{N}}\sum\:_{\text{i}=1}^{\text{N}}\frac{\left(\sum\:_{\text{j}=1}^{\text{k}}{\text{x}}_{\text{i}\text{j}}^{2}\right)-\text{m}}{\text{m}(\text{m}-1)}$$

And

$$\:{\text{P}}_{\text{e}}=\sum\:_{\text{j}=1}^{\text{k}}{\left(\frac{{\sum\:}_{\text{i}=1}^{\text{N}}{\text{x}}_{\text{i}\text{j}}}{\text{N}\bullet\:\text{m}}\right)}^{2}$$

, in which k is the number of categories, N is the number of datasets, and m is the number of experts.

The κ values were interpreted as follows: 0.00–0.20, poor agreement; 0.21–0.40, fair agreement; 0.41–0.60, moderate agreement; 0.61–0.80, substantial agreement; and 0.81–1.00, almost perfect agreement. The test statistics were approximated by a normal distribution to calculate the p-value and the 95% confidence interval (CI). p < 0.05 was considered statistically significant. Statistical analysis was carried out with MedCalc (Version 22.009, MedCalc Software Ltd, Belgium) and IBM SPSS Statistics for Windows, Version 25.0 (IBM Corp., Armonk, NY, USA).

Results

Clinical characteristics

A total of 811 WSIs and corresponding serum biological markers were included in this study, including benign lesions (n = 137), LUAD (n = 325), LUSC (n = 130), NSCLC-NOS (n = 17), SCLC (n = 135), and other malignancies (n = 67). The detailed information is presented in Table 1. The results of biomarker values for different diseases are shown in Table S1 in the Supplementary Materials.

Table 1 Clinical characteristics

P values were calculated based on training and test groups. LUAD: lung adenocarcinoma, LUSC: lung squamous cell carcinoma, SCLC: small cell lung cancer, NSCLC: non-small cell lung cancer, NSCLC-NOS: non-small cell lung cancer not otherwise specified, TBB: transbronchial biopsy, TBLB: transbronchial lung biopsy, TBNA: transbronchial needle aspiration, IQR: interquartile range, CEA: carcinoembryonic antigen, CYFRA21-1: soluble fragment of cytokeratin 19, SCC: squamous cell carcinoma antigen, NSE: neuron-specific enolase, CA125: carbohydrate antigen 125.

Diagnostic performance of the HMLCS model

To evaluate the diagnostic performance of each classification module, the confusion matrix and AUC of each module on the test set were calculated as shown in Fig. 4. For the confusion matrix, each column represents the number of WSIs in each predicted class according to the model, while each row represents the actual number of WSIs in each class according to the annotation. The B/M module achieved an AUC of 0.9540 (95% CI: 0.9257–0.9823) in classifying malignant and benign cases (Fig. 4A). The M-sub module exhibited an AUC of 0.9126 (95% CI: 0.8756–0.9365) in distinguishing subtypes of malignancies (Fig. 4B). The NSCLC-sub module obtained an AUC of 0.9297 (95% CI: 0.9026–0.9603) in classifying subtypes of NSCLC (Fig. 4C). In total, the HMLCS model achieved an AUC of 0.8721 (95% CI: 0.7714–0.9258) and an accuracy of 0.7184 in the six-class classification (benign, LUAD, LUSC, NSCLC-NOS, SCLC, or other malignancies) (Fig. 4D). Additionally, the predictions of the HMLCS model demonstrated substantial agreement with the annotated labels, with a κ value of 0.6183 (p < 0.0001).

Fig. 4

The diagnostic performance of the HMLCS model on the test set. The diagnostic performance of each module and the total model is presented. The first row displays the ROC curve with the AUC value, while the second row shows the confusion matrix with the accuracy and κ value. (A) The diagnostic performance of the B/M module. (B) The diagnostic performance of the M-sub module. (C) The diagnostic performance of the NSCLC-sub module. (D) The diagnostic performance of the total HMLCS model. P1 to P6 indicate the performance of two cytopathologists, two bronchoscopists, and two technicians, respectively. acc: accuracy, AUC: aera under the curve, LUAD: lung adenocarcinoma, LUSC: lung squamous cell carcinoma, NOS: not otherwise specified, NSCLC: non-small cell lung cancer, ROC: receiver operating characteristic, SCLC: small cell lung cancer

We analyzed the performance of our model on lymph node and lung lesion data respectively. The HMLCS model achieved an AUC of 0.8347 (95% CI: 0.7882–0.8606) and an accuracy of 0.6923 in the six-class classification on lymph node data, and achieved an AUC of 0.8961 (95% CI: 0.8616–0.9317) and an accuracy of 0.7295 on lung lesion data. Although there were differences in imaging characteristics between lung and lymph node samples, such as the differences in background cell composition, the performance of our model on lung and lymph node data was similar.

To further analyze the significance of each component in our algorithm, we conducted comprehensive studies to assess the effect of individual elements. Initially, we trained hierarchical models using solely WSIs. The AUC for classifying malignant and benign was 0.9792 (95% CI: 0.9127–0.9838), the AUC for classifying SCLC, NSCLC, and other malignancies was 0.8907 (95% CI: 0.8628–0.9236), and the AUC for classifying LUAD, LUSC, and NSCLC-NOS was 0.8464 (95% CI: 0.8012–0.8829). Overall, the hierarchical model using solely WSIs achieved an AUC of 0.8067 (95% CI: 0.7489–0.8256) and an accuracy of 0.6494 in the six-class classification (Fig. S1 in the Supplementary Materials displays the ROC curve and the confusion matrix of the hierarchical model using solely WSIs). At most levels of classification, our method (using both WSIs and serum biological markers as the input) outperformed the model using solely WSIs, demonstrating the additional predictive power gained through the inclusion of serum biological markers. Additionally, to analyze the significance of our hierarchical classification approach, we attempted using a flat classification model instead of the hierarchical approach, where the six leaf classification labels were equally represented in the output. The flat model achieved an AUC of 0.7967 (95% CI: 0.7601–0.8333) and an accuracy of 0.4770 in the six-class classification (Fig. S2 in the Supplementary Materials displays the ROC curve and the confusion matrix of the flat classification model). Our method outperformed the flat model, underscoring the significance of our hierarchical classification approach.

Comparison between the HMLCS model and professional staff

To further evaluate the performance of the HMLCS model, we conducted a comprehensive comparison with professional staff, including two cytopathologists, two bronchoscopists and two technicians. Fig. 5 illustrates the Cohen’s kappa and accuracy analysis for the cytopathologists (P1 and P2), bronchoscopists (P3 and P4) and technicians (P5 and P6) in relation to the annotated labels. In this analysis, we treated the problem as a simple six-class classification problem to calculate the κ value and the accuracy. Higher κ values of 0.7074 and 0.7422 were found in P1 and P2, demonstrating substantial agreement between cytopathologists and the annotated labels. Moderate agreement was found in P5 (κ = 0.4483). Only weak agreement was shown in P3, P4, and P6, with κ values of 0.3443, 0.3514, and 0.3362, respectively. The accuracy analysis showed similar trends as depicted in Fig. 5C. In comparison, the κ value of our HMLCS model was 0.6183, which was comparable to cytopathologists and superior to trained bronchoscopists and technicians.

Fig. 5

The diagnostic performance of professional staff on the test set. (A) Confusion matrixes of professional staff. (B) κ values of professional staff. (C) Accuracies of professional staff. P1 and P2: two cytopathologists, P3 and P4: two bronchoscopists, P5 and P6: two technicians, LUAD: lung adenocarcinoma, LUSC: lung squamous cell carcinoma, NOS: not otherwise specified, SCLC: small cell lung cancer

Discussion

In this study, a three-step model named HMLCS model was proposed to discriminate between benign and malignant lesions and recognize different subtypes of lung cancer. A large-scale dataset containing 811. ROSE slides and paired serum biological markers from patients with lung lesions and intrathoracic lymphadenopathy was constructed to train and evaluate the model. The HMLCS model demonstrated promising performance, achieving AUC values of 0.9540, 0.9126, and 0.9297 in benign/malignant classification, malignancy subtyping (NSCLC, SCLC, or other malignancies), and NSCLC subtyping (LUAD, LUSC, or NSCLC-NOS), respectively. In total, the HMLCS model achieved an AUC of 0.8721 and an accuracy of 0.7184 in the six-class classification task (benign, LUAD, LUSC, NSCLC-NOS, SCLC, or other malignancies). In addition, the κ value of the HMLCS model in relation to the annotated labels was 0.6183, which was comparable to cytopathologists and superior to trained bronchoscopists and technicians.

Previous studies have only focused on the malignant and benign discrimination on ROSE slides using deep learning [16,17,18,19]. Accuracies of 84.6-95.5% were obtained in these studies. The accuracy of our model in malignant and benign discrimination was 89.08%, which was similar to previous studies. To our knowledge, this is the first study to develop a deep learning model that can identify lung cancer subtypes, and promising performance in lung cancer subtyping was obtained. A large-scale dataset containing both lung lesions and intrathoracic lymphadenopathy was collected to train and evaluate the model.

The proposed HMLCS model addressed the lung cancer subtyping problem by a novel hierarchical framework with three modular classification levels: (1) a B/M module that distinguishes benign and malignant lesions, (2) a M-sub module that subtypes malignancies into SCLC, NSCLC, and other malignancies, and (3) a NSCLC-sub module that further classifies NSCLC subtypes. To formalize hierarchical learning, the HMLCS model calculated independent attention scores at each level, ensuring the model allocated representational capacity based on each decision. According to our results, this cascading structure provided advantages over a single multi-class classifier. Dividing the problem into successive sub-modules allows focused learning on narrowly defined tasks, potentially capturing subtle differences obscured in a holistic approach.

The HMLCS model has the potential to provide real-time feedback regarding preliminary diagnoses of specimens during transbronchial sampling. If the lesion is likely to be malignant according to the patient’s history and preoperative examinations but the specimen is predicted to be benign by the HMLCS model, the specimen obtained may be unqualified, and the biopsy site should be adjusted and new specimens should be obtained. Otherwise, if the lesion is likely to be malignant and the specimen is also predicted to be malignant by the HMLCS model, the specimens are satisfactory. AI-assisted ROSE evaluation could help compensate for the shortage of experienced cytopathologists in the field and address disparities in expertise among medical institutions. Furthermore, diagnostic results of the HMLCS model may be useful to facilitate an early clinical decision, especially for malignant results.

This study acknowledges certain limitations that warrant further investigation. First, this is a single-center retrospective study. To evaluate the generalizability of our model, it is necessary to carry out a multicenter and prospective research. Second, the dataset used in this study came from a specialized hospital, resulting in a high proportion of lung cancer, especially LUAD. However, this phenomenon was reasonable in lesions scheduled for an invasive examination. In a future study, larger dataset including sizeable cases of different diseases should be collected to optimize our model. Third, although the performance of our model was similar on lymph node and lung lesion data, the number of lymph node samples is not large enough (26.51% of the total dataset). The generalizability of our model on lymph node and lung lesion samples needs to be further validated. Finally, although the diagnostic performance of our model was promising, its clinical value needs to be assessed prospectively in real clinical scenarios.

Conclusion

In summary, by utilizing ROSE slides and serum biological markers as the input, our HMLCS model demonstrated cytopathologist-level diagnostic performance in benign and malignant discrimination and lung cancer subtype classification for lung lesions and intrathoracic lymphadenopathy. The HMLCS model could be a useful tool during transbronchial sampling procedures, providing real-time feedback regarding preliminary diagnoses of specimens to the bronchoscopist.