Abstract
Objective
The characteristics of metastatic lymph nodes (MLNs) have been investigated as important predictors of recurrence and progression in papillary thyroid cancer (PTC). However, clinically applicable risk stratification systems are limited to the assessment of size and number of MLNs. This study investigated the predictive value of detailed characteristics of MLNs in combination with currently used risk stratification systems.
Design and methods
We retrospectively characterized 2811 MLNs from 9014 harvested LNs of 286 patients with N1 PTC according to the maximum diameter of MLN (MDLN), maximum diameter of metastatic focus (MDMF), ratio of both diameters (MDMFR), lymph node ratio (LNR, number of MLNs/number of total harvested LNs), presence of extranodal extension (ENE), desmoplastic reaction (DR), cystic component, and psammoma body.
Results
Factors related to the size and number of MLNs were associated with increased risk of recurrence and progression. Extensive presence of ENE (>40%) and DR (≥50%) increased the risk of recurrence/progression. The combination of MDLN, LNR, ENE, and DR had the highest predictive value among MLN characteristics. Combination of these parameters with ATA risk stratification or 1-year response to therapy improved the predictive power for recurrence/progression from a Harrell’s C-index of 0.781 to 0.936 and 0.867 to 0.960, respectively.
Conclusions
The combination of currently used risk stratification systems with detailed characterization of MLNs may improve the predictive accuracy for recurrence/progression in N1 PTC patients.
Introduction
Thyroid cancer is the most common type of endocrine tumor, and it is usually associated with favorable clinical outcomes (1, 2, 3, 4). Recent observational studies have suggested that patients with small sized papillary thyroid cancer (PTC), especially papillary microcarcinoma (PTMC), might be ideal candidates for active surveillance, considering clinicopathological findings such as tumor location and the age of patients (5, 6). However, the clinical manifestations of patients with PTC are not always favorable. Although a large proportion of PTC patients show excellent therapeutic responses, 10–15% of patients develop tumor recurrence or progression (7, 8). The presence of aggressive clinicopathological features and molecular markers, such as BRAFV600E mutation and TERT promoter mutation, have recently been used to predict recurrence and disease progression in PTC (9, 10, 11). For example, the 2015 American Thyroid Association (ATA)-modified initial risk stratification system (RSS) is widely used in the clinical setting. This classification system focuses on tumor size, invasive features, and the characteristics of metastatic lymph nodes (MLNs) in differentiated thyroid cancer (12).
The 2015 ATA RSS requires detailed pathological examination of MLNs, including assessment of the number and size (maximal diameter of the MLN, MDLN) of MLNs (13, 14). In addition to size and number, various pathological features of MLNs predict recurrence and progression (15, 16, 17). The size of metastatic focus in MLN (maximum diameter of metastatic focus, MDMF) reflects the exact tumor burden. The lymph node ratio (LNR), which is the ratio of the number of positive lymph nodes (LNs) to the total number of surgically removed LNs, reflects an adequate intraoperative evaluation (18, 19, 20, 21). Extranodal extension (ENE) is a marker of clinically significant MLNs (22, 23, 24). Desmoplastic reaction (DR), which is the growth of dense connective tissue or stroma around malignant neoplasms, is a predictor of LN recurrence and can be present around the actual metastatic lesions in MLNs (25). In addition to these pathological features, cystic metastasis (CM) and psammoma bodies (PB) are interesting features of PTC that can be pathologically or clinically associated with MLNs (26, 27, 28, 29). These two indicators may facilitate early detection of MLNs, since they present with typical sonographic features.
In this study, we investigated the pathological characteristics of MLNs capable of predicting tumor recurrence and progression in patients with N1 PTC. Although the presence of CM and PB had no impact, detailed examination of MLNs including MDLN, LNR, ENE, and DR improved the predictive accuracy for tumor recurrence and progression, especially when combined with 2015 ATA RSS or 1-year response to therapy (RTT) reclassification.
Methods
Patients
The clinicopathologic characteristics of patients with PTC who underwent total thyroidectomy between January 2006 and December 2010 were analyzed retrospectively via complete review of medical charts and pathology reports. All patients under comprehensive review had histologically proven conventional PTC with LN metastasis. To ensure adequate LN dissection yield, patients with more than six central LNs harvested by central cervical node dissection (CCND) or more than 18 LNs harvested by modified radical neck dissection (MRND) were included (n = 286). Of the 286 enrolled patients, 88 (30.8%) initially underwent total thyroidectomy (TT) with prophylactic CCND without clinically suspicious MLNs; 10 (3.5%) underwent TT with therapeutic CCND; and 188 (65.7%) underwent TT with therapeutic MRND for clinically suspicious or cytologically confirmed lateral LNs. The median follow-up duration was 138.0 months (range, 116–152 months). All enrolled patients received 131 radioactive iodine ablation therapy (RAIT) at 8–20 weeks postoperatively using the dose recommended by ATA guidelines. The follow-up protocol was also based on ATA guidelines. PTC recurrence and progression were confirmed using imaging modalities and/or pathologic diagnosis by ultrasound-guided fine-needle aspiration biopsy. Some patients were included in previous analyses published in prior papers (21, 30).
Characterization of MLNs
A total of 9014 LNs derived from central and lateral neck compartments were analyzed (2440 and 6574, respectively), including 2811 MLNs from central and lateral compartments (855 and 1956, respectively). All MLNs were re-evaluated independently by two experienced pathologists for detailed characterization of nodes. The MDLN and LNFD as well as the ratio of MDMF to MDLN (MDMFR) were calculated. The LNR was defined as the number of MLNs divided by the total number of LNs retrieved from the central compartment with or without nodes from lateral compartments. ENE was assigned pathologically according to the presence of tumor cells extending beyond the lymph node capsule into the perinodal fibroadipose tissue. Thus, microscopic or gross disease beyond the nodal capsule was regarded as the presence of ENE. MLNs were classified into three categories according to the percentage of those with ENE among the total number of MLNs as follows: low (<10%), moderate (10–40%), and high (>40%). According to the presence of fibrous components, MLNs were classified into three categories as follows: none, low (<50%), and moderate to high (≥50%). When several different DR degrees in LNs were detected in one patient, DR status was defined based on the highest degree of DR in all LNs from one patient. MLNs were divided into three categories according to the percentage of MLNs with PB as follows: none, low (<10%), and moderate to high (≥10%). When at least one cystic nodule was observed in all MLNs, it was classified as CM positive group.
Statistical analysis
The Mann–Whitney U-test and Chi-square test or Wilcoxon rank sum test were used to compare groups as appropriate. Categorical cut-off values for the size and number of positive LNs were defined according to frequently used cut-offs from the literature, as well as by receiver operating characteristic (ROC) curves. ROC curves were plotted for all LN number and size cut-off points. The values maximizing sensitivity and specificity for predicting the recurrence and progression were selected for subsequent analyses. Univariable analysis was performed using the log-rank test, and multivariable analysis was performed using the Cox proportional hazard model to identify independent predictors of recurrence and progression. P < 0.05 indicated statistical significance. To estimate the performance of the 2015 ATA RSS and 1-year RTT reclassification, three statistical parameters were calculated: Harrell’s C-index, time-dependent ROC curve (incremental area under the curve, iAUC), and Akaike information criterion (AIC). Comparison of Harrell’s C-index and iAUC was performed using 1000 bootstrap samples, and if the 95% CI did not contain 0, it was interpreted as statistically significant. All statistical analyses were performed using IBM SPSS statistics 23.0 (SPSS Inc.), SAS (version 9.4, SAS Inc., Cary, NC, USA), and R package version 3.1.3 (http://www.R-project.org).
Ethical approval and informed consent
This study was approved by the institutional review board of Severance Hospital and was conducted in accordance with the recommendations of the institutional review board, which waived the requirement for informed consent due to the retrospective nature of this study.
Results
Baseline characteristics of the study patients
This study included 286 patients with a median age of 44 years (interquartile range (IQR) = 33–58). Median tumor size was 1.8 cm (IQR = 1.2–2.5). According to the inclusion criteria, all patients underwent LN dissection including CCND (n = 98), unilateral MRND (n = 154), and bilateral MRND (n = 34). In 88 cases (30.8%), MLNs were not detected preoperatively (i.e. occult MLNs), whereas 198 cases (69.2%) were diagnosed as metastatic disease according to preoperative imaging studies and/or cytological examination (i.e. clinical MLNs). The median MDLN was 1.24 cm (IQR = 0.85–1.75), and the median MDMF was 0.56 cm (IQR = 0.25–1.35). The MDMFR was 0.7 (IQR = 0.42–0.95), and the median LNR was 0.3 (IQR = 0.03–1.00). More than 40% ENE was identified in 44 cases (15.4%). Twenty eight patients (9.8%) had at least one MLN showing >50% DR. Sixteen cases (5.6%) had more than 10% MLNs with PB, and 64 (22.4%) patients had at least one MLN harboring CM. Postoperative RAIT, TNM stage, 2015 ATA RSS, and 1-year RTT reclassification of study patients are described in Supplementary Table 1 (see section on supplementary materials given at the end of this article). Since this study included 88 patients with occult MLNs and 198 patients with clinical MLNs, we first investigated the difference in clinicopathologic features, including detailed MLN characteristics, between the two groups (Supplementary Table 2). Patients with clinical MLNs had more aggressive pathological features of primary tumors compared to those with occult MLNs, such as more frequent microscopic extrathyroidal extension (ETE) and infiltrative tumor margin. The size and MDMFR of clinical MLNs were larger than those of occult MLNs. ENE, DR, and PB, but not CM, were more frequently observed in clinical MLNs, suggesting that these characteristics represent the aggressive features of MLNs. Other clinical features, such as TNM stage, postoperative off-Tg level (serum thyroglobulin level after T4 withdrawal or rhTSH stimulation), 2015 ATA RSS, 1-year RTT reclassification, and recurrence/progression, also showed more aggressive characteristics in patients with clinical MLNs (Supplementary Table 2). Based on these data, the study cohort reflected the current clinical situation for N1 stage patients, suggesting that detailed MLN characteristics indicate the clinical severity of the disease.
Frequent ENE and DR in MLNs of patients with recurrence/progression
Next, we compared the clinicopathological features, including the detailed characteristics of MLNs, according to recurrence/progression. In 34 (11.9%) patients who developed recurrence/progression during the follow-up period, the initial evaluation showed larger primary tumor size, frequent microscopic extension, and infiltrative margin of primary tumors (Table 1). Evaluation of the detailed characteristics of MLNs in these patients showed larger MDLN and MDMF, and MDMFR and LNR were also higher in recurrence/progression group. ENE and DR were more frequently detected in MLNs from patients with recurrence/progression, whereas no difference was found in the frequency of CM and PB (Table 1). The high postoperative off-Tg level (≥10 ng/mL) showed statistically significant association with recurrence/progression group. The 2015 ATA RSS and 1-year RTT reclassification showed different risk distribution levels. High ATA risk and biochemical incomplete response were more frequently observed in recurrence/progression group (Table 1).
Clinicopathological features predicting recurrence/progression in N1 PTC patients (n = 286). Data are presented as n (%) or as median (IQR).
| LN recurrence/Disease progression |
P-value |
||
|---|---|---|---|
| No (n = 252) |
Yes (n = 34) |
||
| Age (years) | 44 (33–58) | 49 (36–55) | 0.324* |
| Gender (F) | 188 (74.6) | 28 (82.4) | 0.485** |
| Tumor size (cm) | 1.7 (1.2–2.5) | 2.2 (1.5–3.3) | 0.008* |
| Microscopic ETE | 0.021** | ||
| No | 58 (23.0) | 2 (5.9) | |
| Yes | 194 (77.0) | 32 (94.1) | |
| Primary tumor margin | <0.001** | ||
| Encapsulated | 146 (57.9) | 4 (11.8) | |
| Infiltrative | 106 (42.1) | 30 (88.2) | |
| Multiplicity | 0.423** | ||
| No | 128 (50.8) | 12 (35.3) | |
| Multifocal | 30 (11.9) | 10 (29.4) | |
| Bilateral | 94 (37.3) | 12 (35.3) | |
| Size of LNs | |||
| MDLN (C) | 1.08 (0.78–1.55) | 2.45 (1.96–2.75) | <0.001* |
| MDMF (D) | 0.51 (0.25–1.05) | 1.70 (1.29–2.08) | <0.001* |
| MDMFR (D/C) | 0.67 (0.40–0.93) | 0.92 (0.64–1.00) | <0.001* |
| LN size according to ATA risk stratification (cm) | <0.001** | ||
| <0.2 | 84 (33.3) | 0 | |
| 0.2–3 | 152 (60.3) | 26 (76.5) | |
| >3 | 16 (6.3) | 8 (23.5) | |
| Number of LNs | |||
| Total LNs harvested (A) | 33 (9–47) | 27 (22–41) | 0.543* |
| LNs involved (B) | 8 (3–13) | 12 (10–24) | <0.001* |
| LNR (B/A) | 0.26 (0.20–0.43) | 0.54 (0.39–0.72) | <0.001* |
| ENE (%) | <0.001** | ||
| <10 | 138 (54.8) | 2 (5.9) | |
| 10–40 | 88 (34.9) | 14 (41.2) | |
| >40 | 26 (10.3) | 18 (52.9) | |
| DR (%) | <0.001** | ||
| No | 122 (48.4) | 2 (5.9) | |
| Capsule <50 | 112 (44.4) | 22 (64.7) | |
| Capsule ≥50 | 18 (7.1) | 10 (29.4) | |
| PB (%) | 0.217** | ||
| <10% | 124 (49.2) | 10 (29.4) | |
| 10–40% | 112 (44.4) | 24 (70.6) | |
| >40% | 16 (6.3) | 0 | |
| CM | 0.295** | ||
| No | 198 (78.6) | 24 (70.6) | |
| Yes | 54 (21.4) | 10 (29.4) | |
| T stage† | <0.001** | ||
| T1 | 166 (65.9) | 14 (41.2) | |
| T2 | 72 (28.6) | 6 (17.6) | |
| T3 | 4 (1.6) | 8 (23.5) | |
| T4 | 10 (4.0) | 6 (17.6) | |
| N stage† | 0.030** | ||
| N1a | 92 (36.5) | 6 (17.6) | |
| N1b | 160 (63.5) | 28 (82.4) | |
| M stage† | 0.601** | ||
| M0 | 242 (96.0) | 32 (94.1) | |
| M1 | 10 (4.0) | 2 (5.9) | |
| TNM stage group† | 0.896** | ||
| I | 190 (75.4) | 24 (70.6) | |
| II | 14 (5.6) | 4 (11.8) | |
| III | 8 (3.2) | 0 | |
| IVa | 34 (13.5) | 6 (17.6) | |
| IVb | 6 (2.4) | 0 | |
| Postoperative 131RAI dose | 0.597** | ||
| 30–100 mCi | 92 (36.5) | 14 (41.2) | |
| 150–200 mCi | 160 (63.5) | 20 (58.8) | |
| Postoperative off-Tg level†† | 0.015** | ||
| <1.0 ng/mL | 112 (44.4) | 10 (29.4) | |
| 1.0–10 ng/mL | 82 (32.5) | 9 (26.5) | |
| ≥10 ng/mL | 58 (23.1) | 15 (44.1) | |
| 2015 ATA risk stratification | <0.001** | ||
| Low | 84 (33.3) | 2 (5.9) | |
| Intermediate | 124 (49.2) | 8 (23.5) | |
| High | 44 (17.5) | 24 (70.6) | |
| 1-year response to therapy | <0.001** | ||
| Excellent response | 216 (85.7) | 4 (11.8) | |
| Indeterminate response | 4 (1.6) | 2 (5.9) | |
| Biochemical incomplete response | 22 (8.7) | 26 (76.5) | |
| Structural incomplete response | 10 (4.0) | 2 (5.9) | |
*P-values calculated by Independent t-test or Mann–Whitney U-test. Data are expressed as the mean (IQR). **P-values calculated by χ2 test or linear-by-linear association. †T-, N-, M-, and TNM- stage: followed by AJCC TNM staging system 8e. ††off-Tg level: serum thyroglobulin level after T4 withdrawal or rhTSH stimulation.
131RAIT, radioactive iodine ablation therapy; CM, cystic metastasis; DR, desmoplastic reaction; ENE, extranodal extension; ETE, extrathyroidal extension; IQR, interquartile range; LN, lymph node; LNR, lymph node ratio; MDLN, maximal diameter of metastatic lymph node; MDMF, maximal diameter of metastatic focus; MDMFR, ratio of MDMF to MDLN; MLN, metastatic lymph node; No., number; PB, psammoma body.
To estimate the hazard ratio (HR) for recurrence/progression, we performed univariable and multivariable regression analyses using clinicopathological features and detailed MLN characteristics. Before performing univariable analyses, we calculated the threshold for continuous variables such as MDLN, MDMF, MDMFR, and LNR using the Contal and O’Quigley methods, and these variables were converted into binary variables. As shown in Table 2, univariable analysis indicated that large tumor size, infiltrative tumor margin, and microscopic and macroscopic ETE, including structural invasion, significantly increased the HR. Detailed characteristics of MLNs such as MDLN, MDMF, MDMFR, LNR, ENE, and DR were also related to increased HR. The high off-Tg level, 2015 ATA RSS, and 1-year RTT reclassification showed a significant association with increased HR. Structural incomplete response in 1-year RTT reclassification was not associated with increased HR, which might be due to the small number of cases in this group. The statistical significance of HR for primary tumor characteristics was decreased in multivariable analyses, although infiltrative margin and T4 lesion remained significantly associated with increased HR. Multivariable analysis showed a statistically significant increase of HR for detailed MLN characteristics including LNR, MDLN, MDMF, MDMFR, ENE, and DR, consistent with the results of univariable analysis.
Univariable and multivariable Cox proportional hazard models to predict recurrence/progression in N1 PTC patients.
| Univariable | Multivariable* | |||
|---|---|---|---|---|
| HR (95% CI) | P-value | HR (95% CI) | P-value | |
| Age, ≥55 years | 1.032 (0.528–3.935) | 0.652 | ||
| Gender, female | 1.054 (0.714–19.845) | 0.512 | ||
| Tumor characteristics | ||||
| Size, >4 cm | 2.125 (1.184–4.087) | 0.022 | 1.915 (0.083–2.657) | 0.255 |
| Multiplicity | 0.782 (1.624–1.217) | 0.624 | ||
| Infiltrative margin | 18.057 (4.987–29.182) | 0.001 | 2.858 (1.224–14.248) | 0.039 |
| Microscopic ETE | 6.514 (1.512–14.178) | 0.033 | 1.725 (0.518–3.993) | 0.307 |
| Macroscopic T3 | 1.871 (1.114–2.514) | 0.012 | 1.309 (0.447–1.759) | 0.204 |
| Structural invasion, T4 | 13.673 (2.651–21.671) | 0.008 | 10.007 (3.152–17.632) | 0.012 |
| Nodal characteristics | ||||
| MDLN ≥1.6 | 5.284 (2.584–26.215) | <0.001 | 4.928 (3.981–22.144) | <0.001 |
| MDMF ≥1.1 | 3.297 (1.645–23.214) | 0.002 | 2.851 (1.502–20.078) | 0.008 |
| MDMFR ≥0.7 | 2.357 (1.187–13.276) | 0.001 | 2.204 (1.315–13.337) | 0.006 |
| LNR ≥0.3 | 4.357 (1.920–19.585) | 0.001 | 3.541 (1.702–16.270) | 0.002 |
| ENE >40% | 6.017 (3.087–30.820) | <0.001 | 4.962 (3.704–21.998) | <0.001 |
| DR ≥50% | 2.842 (1.703–13.769) | 0.0031 | 2.545 (1.017–19.257) | 0.007 |
| PB ≥10% | 0.881 (0.119–10.298) | 0.597 | ||
| CM, yes | 1.064 (0.547–18.415) | 0.314 | ||
| Distant metastasis | 0.961 (0.015–3.145) | 0.764 | ||
| Off-Tg level ≥10 ng/mL | 2.632 (1.182–5.858) | 0.018 | 2.031 (0.921–4.479) | 0.079 |
| 2015 ATA risk stratification | ||||
| Low | Ref. | |||
| Intermediate | 3.648 (1.955–38.571) | 0.001 | 2.157 (0.304–22.741) | 0.471 |
| High | 17.592 (2.098–69.452) | <0.001 | 7.778 (2.517–42.229) | 0.005 |
| 1-year response to therapy | ||||
| Excellent response | Ref. | |||
| Indeterminate response | 6.619 (1.688–34.214) | 0.017 | 4.278 (1.112–19.524) | 0.007 |
| Biochemical incomplete | 14.362 (4.321–52.298) | 0.001 | 3.745 (1.991–20.241) | 0.019 |
| Structural incomplete | 1.142 (0.854–17.248) | 0.352 | 1.004 (0.548–11.074) | 0.576 |
*Multivariable analyses were performed using backward elimination of variables with P < 0.05 in univariable analyses.
DR, desmoplastic reaction; ENE, extranodal extension; ETE, extrathyroidal extension; HR, hazard ratio; LN, lymph node; LNR, lymph node ratio; MDLN, maximal diameter of metastatic lymph node; MDMF, maximal diameter of metastatic focus; MDMFR, ratio of MDMF to MDLN; MLN, metastatic lymph node; No., number.
Predictive accuracy of the combination of detailed MLN characteristics
Based on data indicating the predictive role of each individual MLN characteristic in N1 PTC patients, we calculated Harrell’s C-index, iAUC, and AIC to estimate the performance of individual and combined MLN characteristics. As shown in Table 3, MDLN as a continuous variable showed the highest Harrell’s C-index (0.864), and MDLN as a binary variable (cut-off ≥1.6 cm) also showed the highest iAUC and lowest AIC. Therefore, MDLN showed the best performance as a single variable, although the other MLN characteristics also showed good performance for predicting recurrence/progression. Since the performance of MDLN was superior to that of MDMF, we generated combination models using MDLN as a representation of MLN size. Even if we added one single variable to MDLN, we could observe the increase of Harrell’s C-index and iAUC. Multivariable analysis showed that the combination of MDLN, LNR, ENE, and DR (all as binary variables, namely M11) had the highest predictive accuracy based on Harrell’s C-index, iAUC, and AIC (0.932, 95% CI: 0.891–0.975; 0.937, 95% CI: 0.896–0.977; and 112, respectively). Taken together, our results indicated that MDLN is the most accurate indicator for predicting recurrence/progression among MLN characteristics, whereas the combination of MDLN with other MLN characteristics such as LNR, ENE, and DR may significantly improve the predictive accuracy for recurrence/progression in N1 PTC patients.
Predictive accuracy of structural recurrence/progression using the combination of detailed MLN characteristics.
| Model Name | Variables | Harrell’s C-index (95% CI) | Integrated AUC (95% CI) | AIC |
|---|---|---|---|---|
| Univariable analysis | ||||
| U1* | MDLN | 0.864 (0.775, 0.932) | 0.775 (0.701, 0.901) | 145 |
| U2** | MDLN ≥1.6 | 0.839 (0.772, 0.892) | 0.838 (0.766, 0.891) | 134 |
| U3* | MDMF | 0.814 (0.701, 0.913) | 0.764 (0.665, 0.863) | 149 |
| U4** | MDMF ≥1.1 | 0.782 (0.676, 0.870) | 0.781 (0.676, 0.870) | 146 |
| U5* | LNR | 0.782 (0.696, 0.863) | 0.687 (0.602, 0.778) | 159 |
| U6** | LNR ≥0.3 | 0.734 (0.659, 0.796) | 0.736 (0.658, 0.795) | 151 |
| U7** | ENE >40% | 0.709 (0.589, 0.823) | 0.706 (0.588, 0.823) | 152 |
| U8** | DR ≥50% | 0.612 (0.513, 0.726) | 0.610 (0.512, 0.727) | 162 |
| *Multivariable analysis | ||||
| M1 | U2 + U6 | 0.891 (0.848, 0.935) | 0.888 (0.840, 0.935) | 126 |
| M2 | U2 + U7 | 0.889 (0.813, 0.951) | 0.891 (0.820, 0.952) | 125 |
| M3 | U2 + U8 | 0.871 (0.793, 0.929) | 0.873 (0.802, 0.931) | 128 |
| M4 | U6 + U7 | 0.833 (0.759, 0.906) | 0.830 (0.753, 0.903) | 139 |
| M5 | U6 + U8 | 0.787 (0.723, 0.860) | 0.784 (0.719, 0.856) | 148 |
| M6 | U7 + U8 | 0.754 (0.637, 0.871) | 0.738 (0.632, 0.856) | 151 |
| M7 | U2 + U6 + U7 | 0.920 (0.878, 0.964) | 0.918 (0.874, 0.963) | 119 |
| M8 | U2 + U6 + U8 | 0.909 (0.867, 0.957) | 0.908 (0.860, 0.957) | 122 |
| M9 | U2 + U7 + U8 | 0.903 (0.820, 0.963) | 0.907 (0.845, 0.964) | 121 |
| M10 | U6 + U7 + U8 | 0.873 (0.801, 0.931) | 0.849 (0.789, 0.931) | 137 |
| M11 | U2 + U6 + U7 + U8 | 0.932 (0.891, 0.975) | 0.937 (0.896, 0.977) | 112 |
*Univariable analysis using each factor as a continuous variable. **Univariable analysis using each factor as a binary variable according to the indicated cut-off value.
AIC, Akaike information criterion; DR, desmoplastic reaction; ENE, extranodal extension; iAUC, incremental area under the curve; LNR, lymph node ratio; MDLN, maximal diameter of metastatic lymph node; MDMF, maximal diameter of metastatic ocus; MLN, metastatic lymph nod.
Predictive accuracy of the combination of 2015 ATA RSS or 1-year RTT reclassification with detailed MLN characteristics
The practical performance of the model was examined by adding M11 to the currently used 2015 ATA RSS and 1-year RTT reclassification, which increased HR with higher statistical significance (Supplementary Table 3). We therefore analyzed the predictive accuracy of the 2015 ATA RSS and 1-year RTT reclassification with or without the addition of M11. As shown in Table 4, Harrell’s C-index, iAUC, and AIC of the 2015 ATA RSS were 0.781 (95% CI: 0.668–0.877), 0.778 (95% CI: 0.683–0.876), and 150, respectively. After combination with the M11 model, Harrell’s C-index, iAUC, and AIC of the 2015 ATA RSS improved to 0.936 (95% CI: 0.668–0.877), 0.939 (95% CI: 0.908–0.979), and 115, respectively. Bootstrap comparison of Harrell’s C-index and iAUC showed the statistically significant superiority of combination models (0.16, 95% CI: 0.09–0.25 and 0.17, 95% CI: 0.09–0.25, respectively). Harrell’s C-index, iAUC, and AIC of 1-year RTT reclassification were 0.867 (95% CI: 0.790–0.942), 0.868 (95% CI: 0.789–0.942), and 131, respectively. Consistently, combination models improved the prediction accuracy of Harrell’s C-index, iAUC, and AIC to 0.960 (95% CI: 0.944–0.988), 0.958 (95% CI: 0.940–0.988), and 105, respectively. Consistent with the analysis of the 2015 ATA RSS, bootstrap comparison showed that prediction accuracy increased in combination models (comparison of Harrell’s C-index = 0.10, 95% CI: 0.04–0.17 and comparison of iAUC = 0.09, 95% CI: 0.04–0.17).
Comparison of prediction accuracy for structural recurrence/progression using 2015 ATA risk stratification or 1-year response to therapy with or without the best combination of MLN characteristics.
| Model | Harrell’s C-index (95% CI) | Comparison of Harrell’s C-index | Integrated AUC (95% CI) | Comparison of iAUC | AIC |
|---|---|---|---|---|---|
| A. 2015 ATA Risk Stratification | |||||
| ATA RSS | 0.781 (0.688, 0.877) | Ref | 0.778 (0.683, 0.876) | Ref | 150 |
| ATA RSS + M11 | 0.936 (0.906, 0.979) | 0.16 (0.09, 0.25) | 0.939 (0.908, 0.979) | 0.17 (0.09, 0.25) | 115 |
| B. 1-year Response to Therapy | |||||
| 1-year RTT | 0.867 (0.790, 0.942) | Ref | 0.868 (0.789, 0.942) | Ref | 131 |
| 1-year RTT + M11 | 0.960 (0.944, 0.988) | 0.10 (0.04, 0.17) | 0.958 (0.940, 0.988) | 0.09 (0.04, 0.17) | 105 |
1-year RTT, 1-year response to therapy; AIC, Akaike information criterion; ATA RSS, 2015 ATA risk stratification; iAUC, incremental area under the curve; MLN, metastatic lymph nod.
Discussion
Many studies have investigated prognostic predictors of recurrence and progression in patients with PTC (31, 32). Clinicians currently use the 2015 ATA RSS and 1-year RTT reclassification for risk assessment in patients (11, 33). However, considering the diverse clinical outcomes of PTC, these risk stratification systems are limited in providing precise risk assessment, especially in patients with N1 stage PTC (31, 34, 35). As demonstrated in this study, approximately 40 or more MLNs can be detected in one patient, and these MLNs from individual patients provide important quantitative and qualitative histological information (17). In line with this notion, we hypothesized that detailed investigation of the characteristics of MLNs could provide important information to improve the predictive accuracy of currently used risk assessment methods in N1 PTC patients.
First, we estimated the MDMF, as this measure is a more precise indicator of metastatic tumor burden than the measurement of MDLN, which might include cells in the tumor microenvironment (TME) as well as non-cancer-related cells such as normal lymphocytes. However, the data indicated that MDLN had the best prediction accuracy for size-related variables, although MDMF had considerable statistical power for predicting recurrence/progression. The largest MLN foci were measured as representative samples, which may not accurately reflect the exact tumor burden. However, recent studies have highlighted the importance of TME components, such as cancer-associated fibroblasts and tumor-infiltrating lymphocytes, on tumor progression (36, 37). Therefore, MDLN may be a better marker of tumor burden and tumor-associated cells than MDMF. Whether MDLN can be used as an indicator of the exact amounts of cancer cells and cancer-related cells constituting the TME of MLNs should be determined in future studies. Next, we analyzed the predictive ability of the LNR. To exclude incomplete removal of MLNs and reduce confounding effect of different surgical techniques, we recruited the study subjects based on the number of surgically resected LNs (more than six LNs harvested by CCND and/or more than 18 LNs harvested by MRND). However, such inclusion criteria may limit the ability to generate extensive evidence based on real-world data. In previous studies, we used different cut-off values to predict recurrence/progression according to the inclusion criteria (21, 30). However, if the appropriate cut-off values are established for each cohort, LNR can serve as a useful indicator, as demonstrated in this study.
N1 PTC patients were divided into three groups according to the fraction of MLNs with ENE; ENE was a strong predictor, as shown in univariable and multivariable Cox proportional hazard models (highest HR). Since the presence of ENE might be closely related to the actual number of MLNs, the inclusion of ENE in the 2015 ATA RSS may be difficult (24, 38). However, as demonstrated in this study, the use of appropriate statistical analyses to estimate the optimal number or proportion may enable the incorporation of ENE as a clinically useful predictor (24). Despite the presence of several MLNs with DR, we selected and analyzed the most severe MLNs, which proved to be a prognostic predictor. In contrast to previous reports in other solid tumors, the prognostic value of DR in the primary site of PTC has been underestimated as it is related to the encapsulated margin, which is a favorable histological feature (39, 40). However, primary PTC and metastatic lesions differ in terms of the factors that determine behavior, such as somatic alterations (41). In addition, TME with respect to MDLN may differ between the primary site and MLNs. Therefore, the role of DR in metastatic sites should be investigated separately from that in the primary site (42). CM and PB in MLNs did not show prognostic value; however, these features may improve the diagnostic sensitivity of thyroid ultrasound, as CM and PB were frequently associated with clinically detected MLNs in this study (43). These data suggested that detailed MLN characteristics are stronger predictors of recurrence/progression in N1 PTC patients than primary tumor characteristics. High ATA risk and biochemical incomplete response in 1-year RTT reclassification also remained important factors for increased HR.
The most important finding of this study was that prediction accuracy can be improved through the combined analysis of MLN characteristics. To achieve this goal, we used MDLN, LNR, ENE, and DR, which were converted to binary variables. Although the process of analyzing data was labor intensive due to the large number of MLNs and statistical variables, comprehensive analyses using Harrell’s C-index, iAUC, and AIC consistently showed that the combination of MLN characteristics improved the predictive accuracy. Furthermore, these features combined with the 2015 ATA RSS and 1-year RTT reclassification significantly improved the accuracy of currently used systems for predicting recurrence. Since our study aimed to investigate the predictive ability of detailed characterization of MLNs and enrolled only N1 patients while excluding N0 patients, our data cannot be generalized in every clinical setting. However, our detailed characterization of MLNs could improve the positive predictive value of parameters, such as ultrasound and serum Tg levels, that are commonly used in clinical practice.
In fact, patients with higher values of MDLN, MDMF, MDMFR, LNR, ENE, DR, CM, and PB might have received different doses of RAI compared to patients with lower values. However, all of the enrolled patients had N1 disease and received RAIT at 8–20 weeks postoperatively according to ATA guidelines. Due to the higher risk of persistent or recurrent disease, a large number of macroscopic or clinically LNs or presence of ENE could be major factors in using high dose of RAI (≥150 mCi). All of these features were frequently observed in patients with higher values of our detailed characteristics such as MDLN, MDMF, MDMFR, LNR, ENE, and DR. Based on this idea, we postulated that different doses of RAI were less likely to contribute to a positive conclusion. In addition, the RAI dose itself is not a risk factor in predicting the recurrence/progression in the 2015 ATA RSS and 1-year RTT reclassification. Since we aimed to develop new predictors of recurrence/progression under currently used treatment strategies to reflect the real-world situation, we focused on and followed these risk stratification systems. Even if they were treated with different doses, the therapeutic response would be reflected in 1-year RTT. Therefore, the different 131RAI doses would not be a problem if our detailed characteristics of MLN improved the predictive capacity of 1-year RTT. Finally, the dose of postoperative RAIT was not different according to the presence or absence of recurrence/progression in this study. Since RAI dose did not show statistical significance as a single variable, we excluded RAI dose from our further analyses.
The characterization of MLNs in detail might require high-intensive work, which can limit its clinical application. In the future, the introduction of artificial intelligence will facilitate the identification of MLN characteristics that are useful for the initial diagnostic characterization of thyroid nodules (44). The calculation of cut-off values using a large number of specimens and the techniques described may lead to the development of a system that offers higher predictive capability than that suggested in this study.
In conclusion, this study demonstrated that the analysis of detailed characteristics of MLNs provides important information for predicting the recurrence and progression in N1 PTC and that the combination of MLN characteristics improved the prediction accuracy of the currently used 2015 ATA RSS and 1-year RTT reclassification.
Supplementary materials
This is linked to the online version of the paper at https://doi.org/10.1530/EJE-20-0131.
Declaration of interest
The authors declare that there is no conflict of interest that could be perceived as prejudicing the impartiality of this study.
Funding
J L was supported by a National Research Foundation of Korea (NRF) grant funded by the Korean government (MSIT) (NRF-2020R1A2C1006047). S G J was supported by a National Research Foundation of Korea (NRF) grant funded by the Korean government (MSIT) (NRF-2018R1D1A1B07045227). Y S J was supported by a National Research Foundation of Korea (NRF) grant funded by the Korean government (MSIT) (NRF-2018R1A2B6004179).
Author contribution statement
J L, S G J, and Y S J designed the project and supervised the research. J L, C H K, I K M, and H J K performed histological review and statistical analyses. C H K, S J, H K, and M J C collected and provided data. J L, S G J, and Y S J wrote the manuscript. All authors contributed to discussions about the research and reviewed the manuscript.
Acknowledgements
The authors would like to thank Ji Young Kim (Severance Hospital), Hwanju Lee (Severance Hospital), Hee Chang Yu (Severance Hospital), and Hoyoung Kim (Severance Hospital) for technical support.
References
- 1↑
Chen AY, Jemal A, Ward EM. Increasing incidence of differentiated thyroid cancer in the United States, 1988–2005. Cancer 2009 115 3801–3807. (https://doi.org/10.1002/cncr.24416)
- 2↑
Jung KW, Won YJ, Kong HJ, Oh CM, Seo HG, Lee JS. Cancer statistics in Korea: incidence, mortality, survival and prevalence in 2010. Cancer Research and Treatment 2013 45 1–14. (https://doi.org/10.4143/crt.2013.45.1.1)
- 3↑
Davies L, Welch HG. Current thyroid cancer trends in the United States. JAMA Otolaryngology: Head and Neck Surgery 2014 140 317–322. (https://doi.org/10.1001/jamaoto.2014.1)
- 4↑
Oh CM, Won YJ, Jung KW, Kong HJ, Cho H, Lee JK, Lee DH, Lee KH & Community of Population-Based Regional Cancer Registries. Cancer statistics in Korea: incidence, mortality, survival, and prevalence in 2013. Cancer Research and Treatment 2016 48 436–450. (https://doi.org/10.4143/crt.2016.089)
- 5↑
Ito Y, Tomoda C, Uruno T, Takamura Y, Miya A, Kobayashi K, Matsuzuka F, Kuma K, Miyauchi A. Papillary microcarcinoma of the thyroid: how should it be treated? World Journal of Surgery 2004 28 1115–1121. (https://doi.org/10.1007/s00268-004-7644-5)
- 6↑
Ito Y, Miyauchi A, Oda H. Low-risk papillary microcarcinoma of the thyroid: a review of active surveillance trials. European Journal of Surgical Oncology 2018 44 307–315. (https://doi.org/10.1016/j.ejso.2017.03.004)
- 7↑
Lee J, Rhee Y, Lee S, Ahn CW, Cha BS, Kim KR, Lee HC, Kim SI, Park CS, Lim SK. Frequent, aggressive behaviors of thyroid microcarcinomas in Korean patients. Endocrine Journal 2006 53 627–632. (https://doi.org/10.1507/endocrj.k06-013)
- 8↑
Wada N, Nakayama H, Suganuma N, Masudo Y, Rino Y, Masuda M, Imada T. Prognostic value of the sixth edition AJCC/UICC TNM classification for differentiated thyroid carcinoma with extrathyroid extension. Journal of Clinical Endocrinology and Metabolism 2007 92 215–218. (https://doi.org/10.1210/jc.2006-1443)
- 9↑
Xing M, Alzahrani AS, Carson KA, Viola D, Elisei R, Bendlova B, Yip L, Mian C, Vianello F & Tuttle RM et al.Association between BRAF V600E mutation and mortality in patients with papillary thyroid cancer. JAMA 2013 309 1493–1501. (https://doi.org/10.1001/jama.2013.3190)
- 10↑
Liu X, Qu S, Liu R, Sheng C, Shi X, Zhu G, Murugan AK, Guan H, Yu H & Wang Y et al.Tert promoter mutations and their association with BRAF V600E mutation and aggressive clinicopathological characteristics of thyroid cancer. Journal of Clinical Endocrinology and Metabolism 2014 99 E1130–E1136. (https://doi.org/10.1210/jc.2013-4048)
- 11↑
Haugen BR, Alexander EK, Bible KC, Doherty GM, Mandel SJ, Nikiforov YE, Pacini F, Randolph GW, Sawka AM & Schlumberger M et al.2015 American Thyroid Association management guidelines for adult patients with thyroid nodules and differentiated thyroid cancer: the American Thyroid Association guidelines task force on thyroid nodules and differentiated thyroid cancer. Thyroid 2016 26 1–133. (https://doi.org/10.1089/thy.2015.0020)
- 12↑
Haugen BR. 2015 American Thyroid Association Management Guidelines for adult patients with thyroid nodules and differentiated thyroid cancer: what is new and what has changed? Cancer 2017 123 372–381. (https://doi.org/10.1002/cncr.30360)
- 13↑
Randolph GW, Duh QY, Heller KS, LiVolsi VA, Mandel SJ, Steward DL, Tufano RP, Tuttle RM & American Thyroid Association Surgical Affairs Committee’s Taskforce on Thyroid Cancer Nodal Surgery. The prognostic significance of nodal metastases from papillary thyroid carcinoma can be stratified based on the size and number of metastatic lymph nodes, as well as the presence of extranodal extension. Thyroid 2012 22 1144–1152. (https://doi.org/10.1089/thy.2012.0043)
- 14↑
Adam MA, Pura J, Goffredo P, Dinan MA, Reed SD, Scheri RP, Hyslop T, Roman SA, Sosa JA. Presence and number of lymph node metastases are associated with compromised survival for patients younger than age 45 years with papillary thyroid cancer. Journal of Clinical Oncology 2015 33 2370–2375. (https://doi.org/10.1200/JCO.2014.59.8391)
- 15↑
Urken ML, Mechanick JI, Sarlin J, Scherl S, Wenig BM. Pathologic reporting of lymph node metastases in differentiated thyroid cancer: a call to action for the College of American Pathologists. Endocrine Pathology 2014 25 214–218. (https://doi.org/10.1007/s12022-013-9282-7)
- 16↑
Seethala RR, Baker TP, Simpson JF. A response to the ‘call to action’ on pathologic reporting of lymph node metastases in differentiated thyroid cancer from the college of American pathologists. Endocrine Pathology 2014 25 441–442. (https://doi.org/10.1007/s12022-014-9324-9)
- 17↑
Jeon M-J, Shong Y-K. Metastatic lymph node characteristics predicting prognosis of papillary thyroid cancer patients. Annals of Thyroid 2018 3 15–15. (https://doi.org/10.21037/aot.2018.06.01)
- 18↑
Jeon MJ, Yoon JH, Han JM, Yim JH, Hong SJ, Song DE, Ryu JS, Kim TY, Shong YK, Kim WB. The prognostic value of the metastatic lymph node ratio and maximal metastatic tumor size in pathological N1a papillary thyroid carcinoma. European Journal of Endocrinology 2013 168 219–225. (https://doi.org/10.1530/EJE-12-0744)
- 19↑
Schneider DF, Chen H, Sippel RS. Impact of lymph node ratio on survival in papillary thyroid cancer. Annals of Surgical Oncology 2013 20 1906–1911. (https://doi.org/10.1245/s10434-012-2802-8)
- 20↑
Amit M, Tam S, Boonsripitayanon M, Cabanillas ME, Busaidy NL, Grubbs EG, Lai SY, Gross ND, Sturgis EM, Zafereo ME. Association of lymph node density with survival of patients with papillary thyroid cancer. JAMA Otolaryngology: Head and Neck Surgery 2018 144 108–114. (https://doi.org/10.1001/jamaoto.2017.2416)
- 21↑
Lee J, Lee SG, Kim K, Yim SH, Ryu H, Lee CR, Kang SW, Jeong JJ, Nam KH & Chung WY et al.Clinical value of lymph node ratio integration with the 8(th) edition of the UICC TNM classification and 2015 ATA risk stratification systems for recurrence prediction in papillary thyroid cancer. Scientific Reports 2019 9 13361. (https://doi.org/10.1038/s41598-019-50069-4)
- 22↑
Wang LY, Palmer FL, Nixon IJ, Thomas D, Shah JP, Patel SG, Tuttle RM, Shaha AR, Ganly I. Central lymph node characteristics predictive of outcome in patients with differentiated thyroid cancer. Thyroid 2014 24 1790–1795. (https://doi.org/10.1089/thy.2014.0256)
- 23↑
Wang LY, Palmer FL, Nixon IJ, Tuttle RM, Shah JP, Patel SG, Shaha AR, Ganly I. Lateral neck lymph node characteristics prognostic of outcome in patients with clinically evident N1b papillary thyroid cancer. Annals of Surgical Oncology 2015 22 3530–3536. (https://doi.org/10.1245/s10434-015-4398-2)
- 24↑
Kim HI, Hyeon J, Park SY, Ahn HS, Kim K, Han JM, Bae JC, Shin JH, Kim JS & Kim SW et al.Impact of extranodal extension on risk stratification in papillary thyroid carcinoma. Thyroid 2019 29 963–970. (https://doi.org/10.1089/thy.2018.0541)
- 25↑
Nakayama H, Ohuchida K, Yoshida M, Miyazaki T, Takesue S, Abe T, Endo S, Koikawa K, Okumura T & Moriyama T et al.Degree of desmoplasia in metastatic lymph node lesions is associated with lesion size and poor prognosis in pancreatic cancer patients. Oncology Letters 2017 14 3141–3147. (https://doi.org/10.3892/ol.2017.6549)
- 26↑
Ahuja A, Ng CF, King W, Metreweli C. Solitary cystic nodal metastasis from occult papillary carcinoma of the thyroid mimicking a branchial cyst: a potential pitfall. Clinical Radiology 1998 53 61–63. (https://doi.org/10.1016/s0009-9260(98)80037-8)
- 27↑
Sneed DC. Protocol for the examination of specimens from patients with malignant tumors of the thyroid gland, exclusive of lymphomas: a basis for checklists. Cancer Committee, College of American Pathologists. Archives of Pathology and Laboratory Medicine 1999 123 45–49. (https://doi.org/10.1043/0003-9985(1999)123<0045:PFTEOS>2.0.CO;2)
- 28↑
Wunderbaldinger P, Harisinghani MG, Hahn PF, Daniels GH, Turetschek K, Simeone J, O'Neill MJ, Mueller PR. Cystic lymph node metastases in papillary thyroid carcinoma. American Journal of Roentgenology 2002 178 693–697. (https://doi.org/10.2214/ajr.178.3.1780693)
- 29↑
Chernock RD, Lewis JS Jr. Classification of psammoma bodies in the revised College of American Pathologists thyroid cancer protocol. Archives of Pathology and Laboratory Medicine 2015 139 967. (https://doi.org/10.5858/arpa.2014-0483-LE)
- 30↑
Lee SG, Ho J, Choi JB, Kim TH, Kim MJ, Ban EJ, Lee CR, Kang SW, Jeong JJ & Nam KH et al.Optimal cut-off values of lymph node ratio predicting recurrence in papillary thyroid cancer. Medicine 2016 95 e2692. (https://doi.org/10.1097/MD.0000000000002692)
- 31↑
Tuttle RM, Haugen B, Perrier ND. Updated American Joint Committee on cancer/tumor-node-metastasis staging system for differentiated and anaplastic thyroid cancer (eighth edition): what changed and why?. Thyroid 2017 27 751–756. (https://doi.org/10.1089/thy.2017.0102)
- 32↑
Kim TH, Kim YN, Kim HI, Park SY, Choe JH, Kim JH, Kim JS, Oh YL, Hahn SY & Shin JH et al.Prognostic value of the eighth edition AJCC TNM classification for differentiated thyroid carcinoma. Oral Oncology 2017 71 81–86. (https://doi.org/10.1016/j.oraloncology.2017.06.004)
- 33↑
Lee SG, Lee WK, Lee HS, Moon J, Lee CR, Kang SW, Jeong JJ, Nam KH, Chung WY & Jo YS et al.Practical performance of the 2015 American Thyroid Association guidelines for predicting tumor recurrence in patients with papillary thyroid cancer in South Korea. Thyroid 2017 27 174–181. (https://doi.org/10.1089/thy.2016.0252)
- 34↑
Nixon IJ, Kuk D, Wreesmann V, Morris L, Palmer FL, Ganly I, Patel SG, Singh B, Tuttle RM & Shaha AR et al.Defining a valid age cutoff in staging of well-differentiated thyroid cancer. Annals of Surgical Oncology 2016 23 410–415. (https://doi.org/10.1245/s10434-015-4762-2)
- 35↑
Ito Y, Miyauchi A, Kihara M, Masuoka H, Higashiyama T, Miya A. Subclassification of tumor extension and nodal metastasis in papillary thyroid cancer to improve prognostic accuracy of the eighth edition of the tumor-node-metastasis classification. World Journal of Surgery 2020 44 336–345. (https://doi.org/10.1007/s00268-019-05120-w)
- 36↑
Mlecnik B, Bindea G, Kirilovsky A, Angell HK, Obenauf AC, Tosolini M, Church SE, Maby P, Vasaturo A & Angelova M et al.The tumor microenvironment and Immunoscore are critical determinants of dissemination to distant metastasis. Science Translational Medicine 2016 8 327ra26. (https://doi.org/10.1126/scitranslmed.aad6352)
- 37↑
Lim B, Woodward WA, Wang X, Reuben JM, Ueno NT. Inflammatory breast cancer biology: the tumour microenvironment is key. Nature Reviews: Cancer 2018 18 485–499. (https://doi.org/10.1038/s41568-018-0010-y)
- 38↑
Asanuma K, Kusama R, Maruyama M, Fujimori M, Amano J. Macroscopic extranodal invasion is a risk factor for tumor recurrence in papillary thyroid cancer. Cancer Letters 2001 164 85–89. (https://doi.org/10.1016/s0304-3835(00)00698-4)
- 39↑
Koperek O, Asari R, Niederle B, Kaserer K. Desmoplastic stromal reaction in papillary thyroid microcarcinoma. Histopathology 2011 58 919–924. (https://doi.org/10.1111/j.1365-2559.2011.03791.x)
- 40↑
Lee WK, Lee J, Kim H, Lee SG, Choi SH, Jeong S, Kwon HJ, Jung SG, Jo YS. Peripheral location and infiltrative margin predict invasive features of papillary thyroid microcarcinoma. European Journal of Endocrinology 2019 181 139–149. (https://doi.org/10.1530/EJE-18-1025)
- 41↑
Masoodi T, Siraj AK, Siraj S, Azam S, Qadri Z, Albalawy WN, Parvathareddy SK, Al-Sobhi SS, Al-Dayel F & Alkuraya FS et al.Whole-exome sequencing of matched primary and metastatic papillary thyroid cancer. Thyroid 2020 30 42–56. (https://doi.org/10.1089/thy.2019.0052)
- 42↑
Flier JS, Underhill LH, Dvorak HF. Tumors: wounds that do not heal. New England Journal of Medicine 1986 315 1650–1659. (https://doi.org/10.1056/NEJM198612253152606)
- 43↑
Triggiani V, Guastamacchia E, Licchelli B, Tafaro E. Microcalcifications and psammoma bodies in thyroid tumors. Thyroid 2008 18 1017–1018. (https://doi.org/10.1089/thy.2008.0082)
- 44↑
Choi YJ, Baek JH, Park HS, Shim WH, Kim TY, Shong YK, Lee JH. A computer-aided diagnosis system using artificial intelligence for the diagnosis and characterization of thyroid nodules on ultrasound: initial clinical assessment. Thyroid 2017 27 546–552. (https://doi.org/10.1089/thy.2016.0372)
