Abstract
Objective
Non-palpable testes remain a diagnostic challenge, often involving exploratory laparoscopy. We evaluated the diagnostic value of a wide range of reproductive hormones in order to distinguish between bilateral cryptorchidism and bilateral anorchia.
Design
In this retrospective study, we identified and included 36 boys with non-palpable testes (20 with cryptorchidism, 3 with congenital hypogonadotropic hypogonadism (CHH), and 13 with anorchia) at first examination during childhood.
Methods
Information on karyotype, phenotype, surgical results from laparoscopy, and biochemistry was retrieved from patient files. We compared serum concentrations of AMH, inhibin B, FSH, LH, testosterone, estradiol, and hCG stimulation testing in cryptorchid and anorchid boys to serum concentrations in a large, age-matched control group. Receiver-operating characteristic curves were used to determine the cut-off values of each reproductive hormone as a predictor of the presence of functional testicular tissue.
Results
Concentrations of AMH in 0–1 year olds: ≥155 pmol/L and >1–15 year olds: ≥19 pmol/L, inhibin B (≥22 pg/mL and ≥4 pg/mL), FSH (≤28.9 IU/L and ≤20.3 IU/L) and hCG-induced testosterone (>1-15 year olds: ≥2 nmol/L) were significantly sensitive and specific markers in predicting the presence of functional testicular tissue in boys with non-palpable testes. In infancy, anorchid infants had significantly elevated gonadotropin levels, while CHH had low levels.
Conclusion
Our findings suggest that laparoscopy may not be necessary in all boys with non-palpable testes if reproductive hormones unequivocally confirm the presence of functional testicular tissue. However, proving the absence may still be a diagnostic challenge.
Introduction
Differentiating between hidden testes (i.e. cryptorchidism) and anorchia is a diagnostic challenge (1). Approximately 30% of cryptorchid boys present with bilateral cryptorchidism, while 20% have completely non-palpable testes (1), and bilaterally non-palpable testes may be due to anorchia (2).
Anorchia can be defined as the absence of functional testicular tissue, with or without rudimentary epididymal and spermatic cord remnants, in a normally androgenized boy with karyotype 46, XY (3). Anorchia has an estimated incidence of one in 20,000 live births (4), and the condition can be either congenital or acquired (5). The etiology of congenital cryptorchidism is, like anorchia, still unclear, but studies have shown that intrauterine growth restriction, low birth weight, maternal environment as well as a moderate genetic risk are factors associated with risk of cryptorchidism (6).
In boys born with non-palpable testes, it is important to verify the presence of functional testicular tissue, as germ cells may be present and give rise to germ cell neoplasia in situ (GCNIS) which may result in testicular cancer after puberty (7, 8, 9). Anorchid boys on the other hand have no functional testicular tissue but may have atrophic testes consisting of fibrous tissue with no viable germ cells (3).
The clinical evaluation of newborns with non-palpable testes may include assessment of karyotype, reproductive hormones and abdominal imaging (ultrasound or magnetic resonance imaging, MRI) (10). A human chorionic gonadotropin (hCG) stimulation test is also often performed to establish the presence or absence of functional testis tissue (11). Ultimately, an exploratory laparoscopy may be needed to assess the presence of testicular tissue. However, recent studies have suggested that serum concentrations of anti-Müllerian hormone (AMH), inhibin B, follicle-stimulating hormone (FSH) and luteinizing hormone (LH) may differentiate between bilaterally cryptorchid and anorchid boys (5, 10, 12, 13). AMH and inhibin B are relatively new markers and have not both been analyzed in the same group of boys with non-palpable testes. Therefore, optimal diagnostic strategies remain to be determined in boys with non-palpable testes (14).
Thus, with the aim to distinguish between hidden testes and bilateral anorchia, the diagnostic values of (1) serum concentrations of AMH, inhibin B, FSH, LH, testosterone and estradiol (E2) and (2) hCG stimulation testing were evaluated in this study.
Subjects and methods
Study design
The study was a retrospective study conducted using data from January 1990 to April 2018 at the Department of Growth and Reproduction, Rigshospitalet.
Subjects
Bilaterally cryptorchid boys
In total, 277 boys diagnosed with bilateral testis retention (International Classification of Disease (ICD)-10, code DQ53.2) were identified. All subtypes of congenital cryptorchidism with non-palpable testes were included. Exclusion criteria included (1) unavailable electronic patient file to confirm the diagnosis, (2) diagnosis after the age of 15 years, (3) hormonal treatment prior to time of diagnosis, (4) sex chromosome disorders, (5) presence of retractile testis, and (6) boys with congenital hypogonadotropic hypogonadism, leaving 20 bilaterally cryptorchid boys with non-palpable testes at birth to be included in this study. Bilateral cryptorchidism was confirmed with methods such as hormonal workup, ultrasound, MRI, and exploratory laparoscopy.
Bilaterally cryptorchid boys with congenital hypogonadotropic hypogonadism (CHH)
Of the 277 boys diagnosed with bilateral testis retention as described earlier, three boys were diagnosed with congenital hypogonadotropic hypogonadism (CHH). These three were also included in this study to compare hormone results and clinical characteristics. However, due to the small number they were not included in statistical analyses.
Bilaterally anorchid boys
In total, 13 boys diagnosed with bilateral anorchia (ICD-10, code DQ55.0) were identified. Patient files were used to obtain data on karyotype (to ensure correct diagnosis), genital phenotype, and biochemistry. Anorchia was confirmed by exploratory laparoscopy (complete absence of testicular tissue at surgery or presence of a small nodule of residual fibrous tissue) and by hormonal workup.
In total, 36 boys with non-palpable testes at birth were included in this study.
Hormone analysis
Blood samples were obtained by venipuncture, centrifuged and stored at −20°C. All the samples were analysed in the same laboratory. Inhibin B concentrations were measured using a double antibody immunometric assay (from June 2010 and onwards: GenII, Beckman Coulter; before 2010: Serotec, Oxford, UK (previously named Oxford Bio-Innovation, Oxfordshire, UK)) with a limit of detection (LoD) of 20 pg/mL for the old assay and 3 pg/mL for the new assay. AMH concentrations were measured using a double antibody immunometric (Immunotech, Beckman Coulter, Inc.) with a LoD of 2 pmol/L. FSH and LH concentrations were determined using the time-resolved immunofluorometric assays (Delfia, PerkinElmer) with LoD’s of 0.05 IU/L for both analytes. Testosterone concentrations were measured using a RIA (DPC Coat-A-Count, Diagnostic Products Corp., L.A., CA) with a LoD of 0.23 nmol/L. From 2014 and onwards, testosterone concentrations were measured using the Access 2 Immunoassay System (Beckman Coulter) with a detection limit of 0.35 nmol/L. The two methods were compared and calibrated to each other. Furthermore, testosterone concentrations were measured during hCG stimulation test just before and 72 h after hCG injection. The dose of hCG administered was 100 IE/kg, maximum 5000 IE intramuscularly.
E2 concentrations were measured using RIA (from 1998 and onwards: Pantex, Santa Monica, CA; before 1998: Immunodiagnostic Systems, Bolton, UK) with a LoD of 18 pmol/L. All hormone concentrations have been given decimal points according to clinical use. The performances of the hormone assays are outlined in Table 1 via coefficients of variation (CVs) between (inter-assay CV) and within (intra-assay CV) analysis runs expressed at different concentration levels as means ± 2 s.d.
Performance of hormone assays shown as inter-assay variation (between) and intra-assay variation (within) analysis runs, expressed at different concentration levels as means ± 2 s.d. and coefficients of variation (CVs).
| Inhibin B | Inhibin B | AMH | FSH | LH | TS | T | E2 | |
|---|---|---|---|---|---|---|---|---|
| Years | 1996–2008: Oxford Bio-Innovation (OBI) 2008– 2010: Serotec |
2010 - | 2009–2015* | 1993 - | 1993 - | 1993–2014 | 2014 - | 1993–1998: Immunodiagnostic Systems (IDS) 1998–2019: Pantex |
| Assay (unit) | OBI, Serotec (pg/mL) | Inhibin B Gen II (pg/mL) | Immunotech (pmol/L) |
Delfia (IU/L) | Delfia (IU/L) | DPC Coat-A-Count (nmol/L) | Access 2 (nmol/L) | Immunodiagnostic Systems (pmol/L) |
| LoD | 20 | 3 | 2 | 0.05 | 0.05 | 0.23 | 0.35 | 18 |
| Inter-assay variation | ||||||||
| Low level | ||||||||
| Mean ± 2s.d. | 49 ± 18.4 | 34 ± 4.4 | 19 ± 4.5 | 4.94 ± 0.2 | 3.89 ± 0.2 | 2.42 ± 0.6 | 3.56 ± 0.3 | 72 ± 21.4 |
| CV (%) | 18.8 | 6.5 | 11.6 | 2.2 | 1.9 | 12.8 | 4.7 | 14.9 |
| Medium level | ||||||||
| Mean ± 2s.d. | 233 ± 40.0 | 106 ± 10.0 | 98 ± 20.9 | 16.2 ± 0.9 | 16.8 ± 0.6 | 15.2 ± 2.7 | 14.9 ± 1.0 | 288 ± 32.1 |
| CV (%) | 17.2 | 4.7 | 10.6 | 2.7 | 1.8 | 9.0 | 3.3 | 5.6 |
| High level | ||||||||
| Mean ± 2s.d. | 627 ± 106.0 | 424 ± 42.4 | 525 ± 105.0 | 48.9 ± 2.4 | 62.1 ± 2.1 | 29.1 ± 5.8 | 32.0 ± 3.3 | 730 ± 77.1 |
| CV (%) | 8.5 | 5.0 | 10.1 | 2.5 | 1.7 | 10.0 | 5.2 | 5.3 |
| Intra-assay variation | ||||||||
| Low level | ||||||||
| Mean ± 2s.d. | NA | 35 ± 4.7 | 13 ± 2.0 | 1.31 ± 0.05 | 0.33 ± 0.02 | 0.85 ± 0.3 | 2.81 ± 0.1 | 75 ± 11.3 |
| CV (%) | 6.8 | 7.8 | 2.1 | 3.0 | 17 | 2.6 | 7.5 | |
| Medium level | ||||||||
| Mean ± 2s.d. | 147 ± 13.2 | 106 ± 9.6 | 123 ± 13.3 | 4.84 ± 0.1 | 4.44 ± 0.2 | 6.73 ± 1.0 | 13.6 ± 1.1 | 394 ± 33.9 |
| CV (%) | 9.0 | 4.6 | 5.4 | 1.3 | 1.7 | 7.6 | 4.1 | 4.3 |
| High level | ||||||||
| Mean ± 2s.d. | NA | 425 ± 42.0 | 231 ± 29.6 | 51.4 ± 1.9 | 25.4 ± 0.8 | 40.4 ± 7.9 | 34.8 ± 2.1 | 668 ± 40.1 |
| CV (%) | 5.0 | 6.4 | 1.9 | 1.5 | 9.8 | 3.0 | 3.0 | |
*In 2015, the Access AMH assay (Beckman Coulter) replaced Immunotech but all patient samples included in this study were analyzed prior to 2015.
AMH, anti-Müllerian hormone; CV, coefficient of variance; E2, estradiol; FSH, follicle-stimulating hormone; LH, luteinizing hormone; LoD, limit of detection; NA, not available; TS, testosterone.
Karyotyping
Lymphocytes in peripheral blood were used for karyotyping in all boys, using routine G-banding and counting metaphases.
Clinical examination
All boys were scored retrospectively from case record notes using the external masculinization score (EMS; 0–12 points, 0 = completely feminized, 12 = completely masculinized) as described by Ahmed and co-workers in 2000 (19). Anorchid boys were also scored using the internal masculinization score (IMS; 0–15 points, 0 = completely feminized, 15 = completely masculinized) as also described by (19).
Statistical analysis
The Mann–Whitney U test was used for comparisons of age, EMS, and hormone measurements between cryptorchid and anorchid boys. A P value below 0.05 was considered statistically significant. Receiver-operating characteristic (ROC) curves were used to determine the areas under the curves (AUCs) and their corresponding 95%-CIs and to establish the cut-off values of reproductive hormones as predictors of functional testicular tissue. The cut-off values were assessed with equal weightings of sensitivity and specificity. The AUC was evaluated according to >0.90: excellent, >0.80: good, >0.70: fair, >0.60: poor, and <0.60: sub-standard (20, 21). The first available concentration (untreated) for each hormone for each patient was included in the ROC analyses and calculations.
Ethical considerations
Approvals were obtained from the Danish Data Protection Agency (no. 2012-58-0004, I-Suite no. 04204) and the Danish Patient Safety Authorities (no. 3-3013-1376/1/). The Copenhagen Puberty Study has been approved by the Danish Data Protection Agency (2015-41-4494) and by the regional ethics committee (KF 01 282214 and V200.1996/90).
Results
Distinguishing between hidden testes and bilateral anorchia
The median age at diagnosis was 2.6 years (0.0–9.1 years) in the cryptorchid group and slightly higher (4.0 (0.0–14.6) years) in the anorchid group (P = 0.08) (Tables 2 and 3).
Clinical characteristics of cryptorchid boys and boys with CHH.
| ID | At diagnosis | Biochemistry | Spont. puberty | Age at orchiopexy (years) | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Age (years) | Diagnosis | Karyotype | Total EMS | Location of right testicle/left testicle | Laparoscopy | Age at sampling (years) | TS before/after hCG (nmol/L) | FSH (IU/L) | LH (IU/L) | AMH (pmol/L) | Inhibin B (pg/mL) | ||||
| 0 h | 72 h | ||||||||||||||
| 1 | 0.0 | Cryptorc. | 46,XY | 8 | Inguinal/Inguinal | No | 0.1 | 0.50 | 0.27 | 4.54 | 0.24 | - | <LoD | - | 1.1 |
| 2 | 0.0 | Cryptorc. | 46,XY | 11 | Inguinal/Inguinal | No | 0.2 | 5.17 | - | 1.31 | 6.97 | 527 | 473 | <11 yr | 0.4 |
| 3 | 0.0 | Cryptorc. | 46,XY | 11 | Inguinal/Inguinal | No | 0.0 | 3.31 | - | 1.63 | 4.78 | - | 272 | Yes | 2.1 |
| 4 | 0.2 | Cryptorc. | 46,XY | 7 | Intraabd./Intraabd. | Yes | 0.2 | 8.14 | - | 4.01 | 8.26 | - | 185 | Yes | 1.4 |
| 5 | 0.3 | Cryptorc. | 46,XY | 10 | Intraabd./Intraabd. | Yes | 0.3 | 3.68 | - | 3.05 | 3.58 | 402 | 174 | <11 yr | 1.1 |
| 6 | 0.7 | Cryptorc. | 46,XY | 11 | Inguinal/Inguinal | - | 0.6 | <LoD | 11.6 | 1.56 | 0.30 | - | 42 | Yes | 0.9 |
| 7 | 1.2 | Cryptorc. | 46,XY | 11 | Inguinal/Inguinal | No | 1.2 | <LoD | - | <LoD | <LoD | - | 112 | Yes | 1.2 |
| 8 | 1.2 | Cryptorc. | 46,XY | 7.5 | Inguinal/Inguinal | No | 1.9 | <LoD | 12.2 | 0.54 | 1.65 | - | 76 | Yes | (spont. descent) |
| 9 | 1.4 | Cryptorc. | 46,XY | 11 | Inguinal/Inguinal | No | 0.8 | <LoD | 7.36 | - | -. | - | 136 | Yes | 1.5 |
| 10 | 1.5 | Cryptorc. | 46,XY | 10.5 | Intraabd./Inguinal | Yes | 0.0 | 1.93 | - | 1.44 | 2.36 | - | 207 | <11 yr | 1.5 |
| 11 | 1.7 | Cryptorc. | 46,XY | 10 | Inguinal/Inguinal | No | 1.7 | 9.82 | - | 2.46 | 3.39 | 60 | 428 | Yes | 2.6 |
| 12 | 1.7 | Cryptorc. | 46,XY | 7 | Intraabd./Intraabd. | Yes | 5.2 | 0.59 | 1.90 | 1.03 | 0.11 | - | 20 | Yes | 7.6 |
| 13 | 1.8 | Cryptorc. | 46,XYdel (16)(p11-2) | 11 | Inguinal/Inguinal | No | 1.8 | <LoD | 8.57 | 0.64 | <LoD | - | 112 | Yes | 2.5 |
| 14 | 3.6 | Cryptorc. | 46,XY | 11 | Inguinal/Inguinal | No | 0.2 | 4.35 | - | 4.82 | 3.31 | - | 180 | <11 yr | 3.6 |
| 15 | 3.9 | Cryptorc. | 46,XY | 11 | Inguinal/Inguinal | No | 3.9 | <LoD | 2.25 | 0.44 | <LoD | - | <LoD | - | - |
| 16 | 5.3 | Cryptorc. | 46,XY | 10 | Intraabd./Intraabd. | Yes | 16.6 | 12.0 | - | 13.3 | 4.85 | - | 63 | Yes | 5.3 |
| 17 | 6.1 | Cryptorc. | 46,XY | 11 | Inguinal/Inguinal | No | 6.0 | <LoD | - | 0.45 | 0.07 | - | 43 | Yes | 6.9 |
| 18 | 7.5 | Cryptorc. | 46,XY | 11 | Inguinal/Inguinal | No | 7.5 | <LoD | 3.74 | 0.46 | <LoD | - | 141 | Yes | 7.7 |
| 19 | 7.8 | Cryptorc. | 46,XY | 10 | Intraabd./Intraabd. | Yes | 7.8 | <LoD | 3.33 | 0.71 | 3.48 | - | 62 | - | 8.2 |
| 20 | - | Cryptorc. | 46,XY | 10 | Intraabd./Intraabd. | Yes | 15.1 | 16.2 | - | 19.8 | 4.80 | - | 117 | - | - |
| 1 | 0.48 | CHH | 46,XY | 8 | Inguinal/Inguinal | - | 0.48 | <LoD | 0.32 | 0.14 | <LoD | - | 41 | <11yr | - |
| 2 | 0.90 | CHH | 46,XY | 8 | Scrotal remnant/Intraabd. | Yes | 0.77 | <LoD | 2.70 | 0.07 | <LoD | 50 | <LoD | No | 2.1 |
| 3 | 3.93 | CHH | 46,XY | 8 | Inguinal/Inguinal | No | 4.25 | <LoD | 0.25 | 0.23 | <LoD | - | - | No | 5.7 |
-, missing data; AMH, anti-Müllerian hormone; CHH, congenital hypogonadotropic hypogonadism; Cryptorc., cryptorchidism; EMS, external masculinization score; FSH, follicle-stimulating hormone; hCG, human chorionic gonadotropin; Intraabd., intraabdominal; LH, luteinizing hormone; LoD, limit of detection; supp., suppository; TS, testosterone.
Clinical characteristics of anorchid boys.
| ID | At diagnosis | Biochemistry | Spont. puberty | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Age at diagnosis (years) | Karyotype | Total EMS | Total IMS | Age at laparoscopy (years) | Age at sampling (years) | TS before/after hCG (nmol/L) | FSH (IU/L) | LH (IU/L) | AMH (pmol/L) | Inhibin B (pg/mL) | |||
| 0 h | 72 h | ||||||||||||
| 1 | 0.0 | 46,XY | 9 | - | None | 0.9 | <LoD | - | 148 | 21.6 | <LoD | <LoD | No |
| 2 | 0.5 | 46,XY | 6 | 11 | 0.5 | 0.7 | <LoD | - | 87.9 | 7.10 | <LoD | <LoD | <11 years |
| 3 | 1.0 | 46,XY | 9 | 11 | 1.0 | 1.0 | <LoD | - | 52.9 | 4.00 | - | <LoD | <11 years |
| 4 | 1.4 | 46,XY | 9 | 11 | 1.4 | 1.4 | <LoD | <LoD | 134 | 28.9 | 3 | <LoD | <11 years |
| 5 | 1.7 | 46,XY | 9 | 11 | 1.7 | 1.3 | - | - | - | - | 4 | <LoD | <11 years |
| 6 | 3.8 | 46,XY | 9 | 11 | 3.8 | 3.9 | <LoD | - | 53.4 | 2.00 | <LoD | <LoD | No |
| 7 | 4.0 | 46,XY | 9 | 11 | 4.0 | 5.7 | <LoD | - | 1.60 | <LoD | <LoD | <LoD | No |
| 8 | 4.9 | 46,XY | 6 | 15 | 17.0 | 4.3 | 0.99 | 1.20 | 20.8 | 0.27 | <LoD | <LoD | No |
| 9 | 7.3 | 46,XY | 9 | 11 | 7.3 | 7.3 | <LoD | <LoD | 59.0 | 16.0 | - | <LoD | No |
| 10 | 9.3 | 46,XY | 6 | 11 | 9.3 | 10.0 | <LoD | 0.39 | 10.0 | <LoD | - | - | No |
| 11 | 10.0 | 46,XY | 9 | - | 10.0 | - | - | - | - | - | - | - | No |
| 12 | 10.0 | 46,XYish del(7)(q11.23) | 9 | 11 | 20.0 | 14.0 | 0.31 | 1.27 | 34.7 | 9.62 | <LoD | <LoD | No |
| 13 | 14.6 | 46,XY | 4 | 9 | 14.6 | 11.5 | 0.35 | 0.40 | 108 | 23.5 | - | <LoD | No |
-, missing data; AMH, anti-Müllerian hormone; EMS, external masculinization score; FSH, follicle-stimulating hormone; hCG, human chorionic gonadotropin; IMS, internal masculinization score; LH, luteinizing hormone; LoD, limit of detection; TS, testosterone.
The first available reproductive hormones during or after diagnosis in both groups are shown in Fig. 1 according to age and reference ranges, while testosterone concentrations before and after hCG stimulation are shown in Fig. 2. FSH concentrations were significantly higher in the anorchid group compared to the cryptorchid group at time of diagnosis from birth to 15 years of age (P < 0.001). As compared to boys with cryptorchidism at the time of diagnosis, LH concentrations were significantly higher (P < 0.05), whereas AMH and inhibin B concentrations were significantly lower in anorchid boys (all P values ≤0.05). During the first year of life, significantly lower testosterone concentrations were observed in the anorchid group compared to the cryptorchid group (P = 0.03), but not later in childhood (P = 0.58).
Reproductive hormone concentrations of follicle-stimulating hormone (FSH), luteinizing hormone (LH), testosterone (T), estradiol (E2), inhibin B and AMH as a function of age at first evaluation in 20 boys with cryptorchidism (green), three boys with congenital hypogonadotropic hypogonadism (blue), and 13 boys with anorchia (red). Lines represent reference ranges (± 2 s.d.) for healthy boys and adolescents.
Citation: European Journal of Endocrinology 183, 1; 10.1530/EJE-20-0041
Reproductive hormone concentrations of follicle-stimulating hormone (FSH), luteinizing hormone (LH), testosterone (T), estradiol (E2), inhibin B and AMH as a function of age at first evaluation in 20 boys with cryptorchidism (green), three boys with congenital hypogonadotropic hypogonadism (blue), and 13 boys with anorchia (red). Lines represent reference ranges (± 2 s.d.) for healthy boys and adolescents.
Citation: European Journal of Endocrinology 183, 1; 10.1530/EJE-20-0041
Reproductive hormone concentrations of follicle-stimulating hormone (FSH), luteinizing hormone (LH), testosterone (T), estradiol (E2), inhibin B and AMH as a function of age at first evaluation in 20 boys with cryptorchidism (green), three boys with congenital hypogonadotropic hypogonadism (blue), and 13 boys with anorchia (red). Lines represent reference ranges (± 2 s.d.) for healthy boys and adolescents.
Citation: European Journal of Endocrinology 183, 1; 10.1530/EJE-20-0041
Human chorionic gonadotropin (hCG) stimulation testing in nine out of 20 cryptorchid boys (green), three boys with congenital hypogonadotropic hypogonadism (blue), and seven out of 13 anorchid boys (red). Dots represent testosterone concentrations before (colored) and 72 h after (white) injection with hCG in cryptorchid and anorchid boys. The shaded area is the reference range (+2 s.d. to −2 s.d.) for basal testosterone concentrations according to age in healthy boys.
Citation: European Journal of Endocrinology 183, 1; 10.1530/EJE-20-0041
Human chorionic gonadotropin (hCG) stimulation testing in nine out of 20 cryptorchid boys (green), three boys with congenital hypogonadotropic hypogonadism (blue), and seven out of 13 anorchid boys (red). Dots represent testosterone concentrations before (colored) and 72 h after (white) injection with hCG in cryptorchid and anorchid boys. The shaded area is the reference range (+2 s.d. to −2 s.d.) for basal testosterone concentrations according to age in healthy boys.
Citation: European Journal of Endocrinology 183, 1; 10.1530/EJE-20-0041
Human chorionic gonadotropin (hCG) stimulation testing in nine out of 20 cryptorchid boys (green), three boys with congenital hypogonadotropic hypogonadism (blue), and seven out of 13 anorchid boys (red). Dots represent testosterone concentrations before (colored) and 72 h after (white) injection with hCG in cryptorchid and anorchid boys. The shaded area is the reference range (+2 s.d. to −2 s.d.) for basal testosterone concentrations according to age in healthy boys.
Citation: European Journal of Endocrinology 183, 1; 10.1530/EJE-20-0041
In the three CHH boys, the gonadotropins were very low, testosterone was immeasurable, and AMH and inhibin B were low, while testosterone 72 h after hCG injection showed varying responses (Figs 1 and 2).
Results from the ROC curve analyses are shown in Table 4. The presence of testicular tissue in boys with non-palpable testes separated the cryptorchid boys from the anorchid boys. Individual hormone cut-off concentrations for the presence of functional testicular tissue in the two age groups are shown in Table 4. Percentages of sensitivity ranged from 30–100% and specificity from 64–100%. Almost all reproductive hormone levels found in cryptorchid boys were statistically significantly higher (AMH, inhibin B, testosterone after hCG stimulation) and lower (FSH, LH) than in anorchid boys at time of diagnosis (all P values ≤0.05).
Results from the ROC curve analyses on 33 boys with non-palpable testes based on biochemistry at first evaluation in childhood (20 cryptorchid and 13 anorchid patients). Data are presented as percent (95 % CI).
| Age (years) | n | Cut-off value | AUC | Sensitivity | Specificity | PPV | NPV | Accuracy | Ability of test to predict the presence of testicular tissue | |
|---|---|---|---|---|---|---|---|---|---|---|
| FSH (IU/L) | 0–1 | 11 | ≤28.9 | 1.00 (1.00–1.00) | 100% (63–100) | 100% (29–100) | 100% (100–100) | 100% (100–100) | 100% (72–100) | Excellent |
| >1–15 | 18 | ≤20.3 | 0.95 (0.86–1.00) | 100% (69–100) | 75% (35–97) | 83% (60–94) | 100% (100–100) | 89% (65–99) | Excellent | |
| LH (IU/L) | 0–1 | 12 | ≤7.02 | 0.83 (0.57–1.00) | 89% (52–100) | 67% (9–99) | 89% (61–98) | 67% (21–94%) | 83% (52–98) | Good |
| >1–15 | 20 | ≤7.21 | 0.71 (0.45–0.98) | 100% (66–100) | 64% (31–89) | 69% (51–83) | 100% (100–100) | 80% (56–94) | Fair | |
| TS (nmol/L) | 0–1 | 13 | ≥0.31 | 0.86 (0.66–1.00) | 78% (40–97) | 100% (40–100) | 100% (100–100) | 67% (37–87) | 85% (55–98) | Good |
| >1–15 | 19 | ≥0.21 | 0.52 (0.26–0.79) | 30% (7–65) | 67% (30–93) | 50% (21–79) | 46% (32–61) | 47% (24–71) | Bad | |
| TS (hCG) (nmol/L) | 0–1 | 2 | - | - | - | - | - | - | - | - |
| >1–15 | 13 | ≥1.58 | 1.00 (1.00–1.00) | 100% (59–100) | 100% (54–100) | 100% (100–100) | 100% (100–100) | 100% (75–100) | Excellent | |
| E2 (pmol/L) | 0–1 | 9 | ≥14 | 0.79 (0.47–1.00) | 57% (18–90) | 100% (16–100) | 100% (100–100) | 40% (22–61) | 67% (29–93) | Fair |
| >1–15 | 9 | ≥19 | 0.83 (0.57–1.00) | 80% (28–99) | 100% (40–100) | 100% (100–100) | 80% (41–96) | 89% (52–100) | Good | |
| AMH (pmol/L) | 0–1 | 6 | ≥155 | 1.00 (1.00–1.00) | 100% (29–100) | 100% (29–100) | 100% (100–100) | 100% (100–100) | 100% (54–100) | Excellent |
| >1–15 | 11 | ≥19 | 1.00 (1.00–1.00) | 100% (40–100) | 100% (59–100) | 100% (100–100) | 100% (100–100) | 100% (72–100) | Excellent | |
| Inhibin B (pg/mL) | 0–1 | 13 | ≥22 | 0.94 (0.81–1.00) | 89% (52–100) | 100% (40–100) | 100% (100–100) | 80% (39–96) | 92% (64–100) | Excellent |
| >1–15 | 18 | ≥4 | 1.00 (1.00–1.00) | 100% (69–100) | 100% (63–100) | 100% (100–100) | 100% (100–100) | 100% (81–100) | Excellent |
≤ and ≥, indicate direction of presence of testicular tissue; AMH, anti-Müllerian hormone; Cut-off value, the hormone concentration that best separated the anorchid from cryptorchid boys, i.e. the presence of testicular tissue; FSH, follicle-stimulating hormone; LH, luteinizing hormone; NPV, negative predictive value; PPV, positive predictive value; ROC curve, receiver-operating characteristic curve; TS (hCG), testosterone 72 h after hCG injection; TS, testosterone.
Clinical characteristics
In the diagnostic process of boys with non-palpable testes, eight cryptorchid boys (38%), twelve anorchid boys (92%), and one (33%) with CHH underwent both biochemical workup, MRI, and laparoscopy, while one (8%), 12 (57%), and one (33%), respectively, only underwent biochemical workup and MRI. In one cryptorchid boy and one boy with CHH, data on whether a laparoscopy was done or not was missing (Table 2). One anorchid boy had bilateral testis torsion at birth (Table 3 and Supplementary Table 1, see section on supplementary materials given at the end of this article).
A normal 46,XY karyotype was reported in 19 of 20 (95%) cryptorchid boys, 12 of 13 (92%) anorchid boys, and all boys with CHH (100%). A deviant karyotype of 46,XYdel(16)(p11-2) was reported in one cryptorchid boy and a 46,XYish del(7)(q11.23) karyotype (compatible with Williams syndrome and not associated with anorchia) was reported in one anorchid boy. Furthermore, a single boy (ID no. 12 in Table 2) had a karyotype of 46,XY[49]/47,XYY [1] which due to the low metaphase count was deemed clinically insignificant and thus labeled as 46,XY.
At birth, a significantly lower median EMS was reported in the anorchid group (9 points, range: 4–9) than in the cryptorchid group (10 points, range 7–11), P = 0.01 (Tables 2 and 3).
Surgical findings at laparoscopy in the 12 anorchid boys who underwent exploratory laparoscopy (93%) are summarized in Supplementary Table 1. The median age at surgery was 4.5 years (range: 0.5–19.9 years).
Discussion
In this retrospective study comprising 34 boys with non-palpable testes at birth due to cryptorchidism (but not CHH) or anorchia, we found that 1) during the first year of life serum concentrations of AMH, inhibin B, and FSH appeared to be the most accurate markers to detect the presence of functional testicular tissue, 2) in the age from 1 year to 15 years, serum concentrations of AMH and inhibin B as well as testosterone 72 h after hCG injection appeared to be the most accurate markers to detect the presence of functional testis tissue, and (3) already in mini-puberty, anorchid infants showed a distinct exaggerated gonadotropin secretion as compared to cryptorchid and control boys. Conversely, boys with CHH had low concentrations of gonadotropins and testicular hormones. This is the first study to assess the diagnostic accuracy of AMH, inhibin B, E2, basal and hCG stimulated testosterone, and gonadotropins in a single cohort.
Our present findings support the notion that AMH and inhibin B from birth to 15 years of age and FSH during first year of life at certain cut-off levels appeared to be excellent diagnostic markers of functional testicular tissue. Furthermore, a testosterone concentration 72 h after hCG stimulation appeared to be an excellent diagnostic test to prove the presence of testicular tissue from 1 year onward to 15 years of age, when assessing a male infant with non-palpable testes, although the usefulness of the hCG stimulation test has previously been questioned (11, 22, 23, 24). Serum concentrations of LH and total testosterone were considered fair and bad diagnostic markers, respectively, of detecting testicular tissue from 1 to 15 years old but considered good diagnostic markers from 0 to 1 years old.
In a recent study, it was reported that while serum concentrations of FSH <2 IU/L ruled out anorchia, LH concentrations >5 IU/L confirmed anorchia until the age of 6 years (5). In accordance with these findings, others have suggested supra-normal gonadotropin concentrations to support the diagnosis of anorchia (22). Furthermore, the testicular hormones AMH and inhibin B have also been found to be markers of functional testicular tissue (10, 12), and particularly inhibin B has been reported to differentiate between cryptorchidism and anorchia (10, 25).
In general, the application of ROC analyses to assess cut-off values and their corresponding sensitivities, specificities, and predictive values is an excellent way to separate two groups – in this instance patients with viable testicular tissue (cryptorchidism) and those without (anorchia). Thus, in the present study, the cut-offs indicated the presence of testicular tissue, that is, an AMH concentration above 19 pmol/L in a boy during childhood strongly indicates that testicular tissue is present. In the case of anorchia, however, it is meaningless that a patient with an AMH concentration of 18 pmol/L (and thus below the cut-off) is grouped as anorchia. In males, only the testes produce AMH and thus any detectable value of AMH above the detection limit should prompt exploratory surgery to find testicular tissue due to the risk of GCNIS. It is also important to note that the rather small sample size of this study – as clearly depicted by the large 95% CIs of the sensitivities, specificities and predictive values – limits the sturdiness of the conclusions and cut-offs found in this study. Lastly, it is also important to note that many boys with non-palpable testes will have altered testicular function and thereby lower concentrations of the testicular hormones. Assay performance for all reproductive hormones varies according to concentration level and intra- and inter-assay variations are larger for lower concentrations. This again limits the strength of the cut-off values found in this study. Nonetheless, this is the first study to evaluate a full biochemical workup in the same cohort of patients with non-palpable gonads and namely the testicular markers of AMH and inhibin B from birth until 15 years of age along with FSH during the first year of life appear to be excellent markers of testicular tissue, particularly when either high or undetectable.
As expected, due to the lack of feedback from gonadal hormones, LH and FSH concentrations were markedly elevated during mini-puberty and puberty in boys with anorchia, and significantly so compared to cryptorchid boys and controls from infancy into childhood. This highlights that the hypothalamic-pituitary-gonadal axis is functionally intact in anorchid boys. Similarly, an exaggerated biphasic secretion pattern of gonadotropins has also been reported in hypo- and agonadal girls with Turner syndrome (26, 27). The gonadotropin levels in CHH boys were close to undetectable as a consequence of neuroendocrine impairment of GnRH secretion. This has also previously been described in CHH patients (28). The diagnostic dilemma in patients with hidden testes lies more in proving the absence than the presence (which is easier by testicular hormones) of testicular tissue. In this study, the CHH boys were cryptorchid with low to undetectable levels of testicular hormones and would thus falsely be classified as anorchid if only testicular hormones (particularly AMH and inhibin B) were considered. Thus, the gonadotropins add essential information in separating CHH from anorchia. Our study was, however, limited by the CHH sample size and thus gonadotropin cut-off concentration values to distinguish between CHH and anorchia were not produced. It would be highly relevant to further investigate this in a future study.
Median age of diagnosis was slightly higher in the anorchid than cryptorchid group (i.e. 4.0 years of age). This rather high median age is explained by the conservative method of watchful waiting that was often chosen in the management of a child with bilateral non-palpable testes in the 1990s. All cryptorchid boys with intra-abdominal gonads (38%) and most anorchid boys (93% – i.e. all but one boy with perinatal bilateral testicular torsion) underwent exploratory laparoscopy. Importantly, however, one (5%) cryptorchid boy and four (31%) anorchid boys had re-laparoscopies performed due to uncertainty of previous laparoscopic findings. In contrast with this, laparoscopy has been reported as a safe way to diagnose a boy with non-palpable testes in a previous study (29). Nevertheless, availability of non-surgical diagnostic markers like AMH and inhibin B appear to be attractive alternatives to surgical procedures in some cases of boys with non-palpable testes.
To confirm the presence (or absence) of intra-abdominal testes, surgery had been performed in the cryptorchid boys with intra-abdominally located testes and in all anorchid boys. However, the need for surgery due to the possibility of GCNIS is also debated. Nataraja et al. examined excised testicular remnants from 140 boys and found that one in every tenth boy had germ cells present in the remnant, and one in every fourth boy had seminiferous tubules in the remnant. However, endocrine testing was not reported, and it is thus unknown whether these remnants were capable of producing AMH, inhibin B, and T (30). If there is no sign of viable germ cells in a testicular remnant containing functional Sertoli or Leydig cells, germ cell tumors cannot develop (31). In a review it was concluded that 5–15% of testicular remnants contained seminiferous material but only few had viable germ cells; thus, the risk of malignant degeneration was considered minimal (9). Moreover, another study of 11 anorchid boys found significant concordance between the serum concentration of AMH and the results from laparoscopic and histological findings (13). Finally, another study showed that in a cohort of 31 boys with non-palpable testes, an hCG stimulation test and imaging was sufficient in predicting testicular tissue prior to orchidopexy (23).
Thus, laparoscopy seems unnecessary in patients with bilateral non-palpable testes if the biochemical workup unequivocally confirms the absence of functional testicular tissue.
The strengths of this study include that (1) the diagnosis of congenital bilateral cryptorchidism and anorchia was proved by a broad biochemical work-up including hCG stimulation testing as well as early imaging, laparoscopy or inguinal exploration, (2) the design was based on a single-center study, meaning that all biochemical investigations were made using the same assays, and all patients were clinically assessed by the same specialists, and (3) the hormonal cut-off values were based on highly sensitive assays. Limitations include that (1) the study size was relatively small, (2) the cryptorchid boys were diagnosed slightly earlier than the anorchid boys although not at statistically significantly different ages, and (3) the cut-off values found in this study may vary with other assays.
In conclusion, each of the hormones AMH, inhibin B, FSH, and testosterone after hCG stimulation test appeared to be sensitive and specific diagnostic markers in detecting the presence of functional testes in pre-pubertal boys. Our findings suggest that an exploratory laparoscopy may not be needed in a boy with bilaterally non-palpable testes if the serum concentrations of AMH and inhibin B unequivocally confirm the absence of functional testicular tissue. However, at low – but measurable – concentrations of AMH and inhibin B, an exploratory laparoscopy would be indicated to ensure that functional testicular tissue that could develop germ cell neoplasia in situ is not present. Moreover, gonadotropins (high in anorchia, low in CHH) appear to be important in separating boys with CHH and boys with anorchia. Thus, the results of this study point to an unharmful and minimally invasive approach by measuring gonadotropins and testicular hormones in all patients with non-palpable testes.
Supplementary materials
This is linked to the online version of the paper at https://doi.org/10.1530/EJE-20-0041.
Declaration of interest
The authors declare that there is no conflict of interest that could be perceived as prejudicing the impartiality of the research reported.
Funding
M L L was supported by the Copenhagen University Hospital’s Research Foundation (Rigshospitalets Forskningsudvalg) through a 3-year stipend and the Absalon Foundation. Clinical trial registry: COPENHAGEN Puberty Study ClinicalTrials.gov no. NCT01411527 (registered 3 August 2011).
References
- 1↑
Radmayr C, Dogan HS, Hoebeke P, Kocvara R, Nijman R, Silay S, Stein R, Undre S, Tekgul S. Management of undescended testes: European Association of Urology/European Society for Paediatric Urology Guidelines. Journal of Pediatric Urology 2016 12 335–343. (https://doi.org/10.1016/j.jpurol.2016.07.014)
- 2↑
Ahmed SF, Dobbie R, Finlayson AR, Gilbert J, Youngson G, Chalmers J, Stone D. Prevalence of hypospadias and other genital anomalies among singleton births, 1988–1997, in Scotland. Archives of Disease in Childhood: Fetal and Neonatal Edition 2004 89 F149–F151. (https://doi.org/10.1136/adc.2002.024034)
- 3↑
Baker GHW. Male endocrinological disorders and male factor infertility. In Oxford Textbook of Endocrinology and Diabetes, 2nd ed., ch 9.4, pp. 1385–1393. Eds Wass JAH, Stewart PM & Amiel S. Oxford, UK: Oxford University Press, 2011. (https://doi.org/10.1093/med/9780199235292.001.1)
- 4↑
Bobrow M, Gough MH. Bilateral absence of testes. Lancet 1970 295 366. (https://doi.org/10.1016/S0140-6736(70)90753-1)
- 5↑
Grinspon RP, Ropelato MG, Bedecarrás P, Loreti N, Ballerini MG, Gottlieb S, Campo SM, Rey RA. Gonadotrophin secretion pattern in anorchid boys from birth to pubertal age: pathophysiological aspects and diagnostic usefulness. Clinical Endocrinology 2012 76 698–705. (https://doi.org/10.1111/j.1365-2265.2011.04297.x)
- 6↑
Barthold JS, Reinhardt S, Thorup J. Genetic, maternal, and environmental risk factors for cryptorchidism: an update. European Journal of Pediatric Surgery 2016 26 399–408. (https://doi.org/10.1055/s-0036-1592416)
- 7↑
Hutson JM, Li R, Southwell BR, Petersen BL, Thorup J, Cortes D. Germ cell development in the postnatal testis: the key to prevent malignancy in cryptorchidism? Frontiers in Endocrinology 2013 3 1–11. (https://doi.org/10.3389/fendo.2012.00176)
- 8↑
Nef S, Parada LF. Hormones in male sexual development. Genes and Development 2000 14 3075–3086. (https://doi.org/10.1101/gad.843800)
- 9↑
Wood HM, Elder JS. Cryptorchidism and testicular cancer: separating fact from fiction. Journal of Urology 2009 181 452–461. (https://doi.org/10.1016/j.juro.2008.10.074)
- 10↑
Thorup J, Petersen BL, Kvist K, Cortes D. Bilateral vanished testes diagnosed with a single blood sample showing very high gonadotropins (follicle-stimulating hormone and luteinizing hormone) and very low inhibin B. Scandinavian Journal of Urology and Nephrology 2011 45 425–431. (https://doi.org/10.3109/00365599.2011.609833)
- 11↑
Ahmed SF, Keir L, McNeilly J, Galloway P, O’Toole S, Wallace AM. The concordance between serum anti-Mullerian hormone and testosterone concentrations depends on duration of hCG stimulation in boys undergoing investigation of gonadal function. Clinical Endocrinology 2010 72 814–819. (https://doi.org/10.1111/j.1365-2265.2009.03724.x)
- 12↑
Lee MM, Donahoe PK, Silverman BL, Hasegawa T, Hasegawa Y, Gustafson ML, Chang YC, McLaughlin DT. Measurements of serum Müllerian inhibiting substance in the evaluation of children with nonpalpable gonads. New England Journal of Medicine 1997 336 1480–1486. (https://doi.org/10.1056/NEJM199705223362102)
- 13↑
Teo AQA, Khan AR, Williams MPL, Carroll D, Hughes IA. Is surgical exploration necessary in bilateral anorchia? Journal of Pediatric Urology 2013 9 e78–e81. (https://doi.org/10.1016/j.jpurol.2012.09.006)
- 14↑
Ng KL, Ahmed SF, Hughes IA. Pituitary–gonadal axis in male undermasculinisation. Archives of Disease in Childhood 2000 82 54–58. (https://doi.org/10.1136/adc.82.1.54)
- 15↑
Juul A, Bang P, Hertel NT, Main K, Dalgaard P, Jørgensen K, Müller J, Hall K, Skakkebaek NE. Serum insulin-like growth factor-I in 1030 healthy children, adolescents, and adults: relation to age, sex, stage of puberty, testicular size, and body mass index. Journal of Clinical Endocrinology and Metabolism 1994 78 744–752. (https://doi.org/10.1210/jcem.78.3.8126152)
- 16↑
Sørensen K, Aksglaede L, Petersen JH, Juul A. Recent changes in pubertal timing in healthy Danish boys: associations with body mass index. Journal of Clinical Endocrinology and Metabolism 2010 95 263–270. (https://doi.org/10.1210/jc.2009-1478)
- 17↑
Andersson AM, Juul A, Petersen JH, Müller J, Groome NP, Skakkebaek NE. Serum inhibin B in healthy pubertal and adolescent boys: relation to age, stage of puberty, and follicle-stimulating hormone, luteinizing hormone, testosterone, and estradiol levels. Journal of Clinical Endocrinology and Metabolism 1997 82 3976–3981. (https://doi.org/10.1210/jcem.82.12.4449)
- 18↑
Lindhardt Johansen M, Hagen CP, Rajpert-De Meyts E, Kjærgaard S, Petersen BL, Skakkebaek NE, Main KM, Juul A. 45,X/46,XY mosaicism: phenotypic characteristics, growth, and reproductive function – a retrospective longitudinal study. Journal of Clinical Endocrinology and Metabolism 2012 97 1540–1549. (https://doi.org/10.1210/jc.2012-1388)
- 19↑
Ahmed SF, Khwaja O, Hughes IA. The role of a clinical score in the assessment of ambiguous genitalia. BJU International 2000 85 120–124. (https://doi.org/10.1046/j.1464-410x.2000.00354.x)
- 20↑
Hajian-Tilaki K. Receiver operating characteristic (ROC) curve analysis for medical diagnostic test evaluation. Caspian Journal of Internal Medicine 2013 4 627–635.
- 21↑
Safari S, Baratloo A, Elfil M, Negida A. Evidence based emergency medicine; part 5 receiver operating curve and area under the curve. Emergency 2016 4 111–113.
- 22↑
Brauner R, Neve M, Allali S, Trivin C, Lottmann H, Bashamboo A, McElreavey K. Clinical, biological and genetic analysis of anorchia in 26 boys. PLoS One 2011 6 4–8. (https://doi.org/10.1371/journal.pone.0023292)
- 23↑
Davenport M, Brain C, Vandenberg C, Zappala S, Duffy P, Ransley PG, Grant D. The use of the hCG stimulation test in the endocrine evaluation of cryptorchidism. British Journal of Urology 1995 76 790–794. (https://doi.org/10.1111/j.1464-410x.1995.tb00776.x)
- 24↑
Bartone FF, Huseman CA, Maizels M, Firlit CF. Pitfalls in using human chorionic gonadotropin stimulation test to diagnose anorchia. Journal of Urology 1984 132 563–567. (https://doi.org/10.1016/s0022-5347(17)49741-2)
- 25↑
Kubini K, Zachmann M, Albers N, Hiort O, Bettendorf M, Wölfle J, Bidlingmaier F, Klingmüller D. Basal inhibin B and the testosterone response to human chorionic gonadotropin correlate in prepubertal boys. Journal of Clinical Endocrinology and Metabolism 2000 85 134–138. (https://doi.org/10.1210/jcem.85.1.6285)
- 26↑
Hagen CP, Main KM, Kjaergaard S, Juul A. FSH, LH, inhibin B and estradiol levels in Turner syndrome depend on age and karyotype: longitudinal study of 70 Turner girls with or without spontaneous puberty. Human Reproduction 2010 25 3134–3141. (https://doi.org/10.1093/humrep/deq291)
- 27↑
Johannsen TH, Main KM, Ljubicic ML, Jensen TK, Andersen HR, Andersen MS, Petersen JH, Andersson AM, Juul A. Sex-differences in reproductive hormones during mini-puberty in infants with normal and disordered sex development. Journal of Clinical Endocrinology and Metabolism 2018 103 3028–3037. (https://doi.org/10.1210/jc.2018-00482)
- 28↑
Bougnères P, François M, Pantalone L, Rodrigue D, Bouvattier C, Demesteere E, Roger D, Lahlou N. Effects of an early postnatal treatment of hypogonadotropic hypogonadism with a continuous subcutaneous infusion of recombinant follicle-stimulating hormone and luteinizing hormone. Journal of Clinical Endocrinology and Metabolism 2008 93 2202–2205. (https://doi.org/10.1210/jc.2008-0121)
- 29↑
Cortes D, Thorup JM, Lenz K, Beck BL, Nielsen OH. Laparoscopy in 100 consecutive patients with 128 impalpable testes. British Journal of Urology 1995 75 281–287. (https://doi.org/10.1111/j.1464-410x.1995.tb07338.x)
- 30↑
Nataraja RM, Asher CM, Nash R, Murphy FL. Is routine excision of testicular remnants in testicular regression syndrome indicated? Journal of Pediatric Urology 2015 11 151.e1–151.e5. (https://doi.org/10.1016/j.jpurol.2015.01.018)
- 31↑
Looijenga LHJ, Hersmus R, de Leeuw BHCGM, Stoop H, Cools M, Oosterhuis JW, Drop SLS, Wolffenbuttel KP. Gonadal tumors and DSD. Best Practice and Research: Clinical Endocrinology and Metabolism 2010 24 291–310. (https://doi.org/10.1016/j.beem.2009.10.002)
