An overview of clinical activities in Endo-ERN: the need for alignment of future network criteria

Objective: Given that volumes of patients and interventions are important criteria to qualify as a reference centre (RC) for the European Reference Network on Rare Endocrine Conditions (Endo-ERN), the present study aimed to evaluate the data that were reported in the original application against subsequent assessments of activity and review the criteria that may define RCs using two main thematic groups (MTGs): Pituitary and Thyroid, as examples. Methods: Review


Introduction
In March 2017, 24 European Reference Networks (ERNs) were officially installed under the directive 2011/24/EU on patient's rights in cross-border health care. The mission of ERNs is to reduce health care inequalities for all patients with rare and/or complex conditions across the European Union (EU). This is to be achieved through cross-border expert consultation and guideline conformity, enabling the highest standard of care. The European Reference Network on Rare Endocrine Conditions (Endo-ERN), at present, is the largest ERN and comprises 71 reference centres (RCs) representing 19 countries. Endo-ERN has divided the European landscape for rare endocrine conditions in eight main thematic disease areas, the Main Thematic Groups (MTGs), and covers both paediatric and adult expertise. The MTGs, which are organ and physiology based, are: Adrenal, Disorders of calcium & phosphate homeostasis, Genetic disorders of glucose & insulin homeostasis, Genetic endocrine tumour syndromes, Growth & genetic obesity syndromes, Pituitary, Sex development & maturation, and Thyroid. Each MTG includes subthemes of several specific conditions based on hormonal overproduction, -insufficiency, -resistance and/ or tumour growth. Endo-ERN's mission is to reduce and ultimately abolish inequalities in care for patients with rare endocrine disorders in Europe, through facilitating knowledge sharing and related health care and research. The mission of Endo-ERN is defined in five work packages The 2016 application to the first call for ERN membership was eligible only for centres that had obtained national endorsement for specific, rare disease expertise. This advice to the member states was to base the endorsement on the EUCERD criteria (1), that, among others, include participation in a regional or national assessment program, patient-centred organization of care, and on highly specific interventions covered by the health care provider. These requirements were very differently handled depending on national health care system organization and legal context. A generic EU application form for all ERNs had to be completed, that included numbers of new patients, patients under chronic care, and procedures (e.g. operations, radiotherapy, or other complex interventions, but in this application, it was not specified in detail) performed over the last 3 years. Centres were evaluated according to Endo-ERN-specific network criteria, which were proposed after discussion by the (future) steering committee, and included an expert based minimal volume of new and chronic patients and surgical volumes. If they were available, these were based on international guidelines. At this stage, this process was a novelty both for the EU and potential RCs. No endocrine disease network was in place and there were tight deadlines. Therefore, not a condition specific but a generic application form was provided by the EU. No clear instructions were provided on how to define a new or chronic patient (i.e. first-time diagnosis or first-time visitor in that specific hospital for example), and no guidance was provided on how to retrieve these numbers from the hospital systems. Consequently, RCs were allowed to report according to their discretion (use of own strategy), using their local systems and practices to retrieve patient and interventions volumes. As a consequence, definitions of intervention were interpreted by centres very variably. Interpretations ranged from surgical procedures only to a diverse spectrum of procedures, including all dynamic hormone tests.
The aim of the present study was to evaluate how self-reported volume data in the original applications were obtained. Next, using a survey, we tried to obtain insight into which data were considered retrievable and which data set is best to use for future centre evaluations.
In particular, what is the best definition of a new patient, a patient under chronic care, and what interventions should be registered to define a reference centre. Because of the large size of the network and the heterogeneity of subgroups of conditions, both between and within MTGs, we decided to focus on two example MTGs: Pituitary (43 RCs) and Thyroid (35 RCs).

Analysis of original application forms (submitted in 2016, data from 2013, 2014, and 2015)
We retrieved and analysed the data from the original network applications of the individual RCs participating in the two specific MTGs of Endo-ERN. The following data were extracted: total number of new patients, total number of patients under chronic care, and total number of procedures per year of the years 2013-2015, and per sub-theme. Subsequently, we calculated the means for the reported 3 years. Additionally, a distinction was made between RCs covering adult expertise, paediatric expertise, or both. The application form can be found in the supplementary files.

Survey on RCs interpretation on definitions (2018-2019)
We aimed to obtain more clarity on the strategies used by the participating RCs to define new patients and procedures in the original application. Furthermore, we wanted to assess future possibilities to extract more reliable, uniform data and to identify barriers in extracting patient numbers and interventions. A survey was developed under the guidance of the adult chairs of both MTGs (NRB and RPP). The survey captured the number of new patients, the number of patients under chronic care, and the number of procedures performed following uniform definitions. The definition of a new patient was 'any treatment naive patient, and/or any new patient for the healthcare provider'. Definition of a patient under chronic care was 'a patient under chronic treatment at the RC, not including remote consultations or referrals for specific testing only'. A list of the key procedures was composed which were considered to be relevant for the specific conditions covered by MTGs Pituitary and Thyroid. Centres were also asked to provide numbers on key procedures and availability of other specialized care. For MTG Pituitary, the following key procedures were defined: (1.) transsphenoidal surgery, (2.) conventional radiotherapy, (3.) radiosurgery, and (4.) genetic testing for hypopituitarism. Furthermore, the availability of medical treatment, bilateral inferior petrosal sinus sampling, dynamic hormone testing, and multidisciplinary care pathways were surveyed. For MTG Thyroid, the key procedures were (1.) thyroid surgery, (2.) radioiodine therapy, and (3.) molecular analysis for radioiodine-sensitive and non-metastatic medullary thyroid carcinoma. Availability was captured for novel drugs and personalised therapy for rare genetic thyroid disease and emerging genetic studies and new-born screening in congenital hypo-and hyperthyroidism. The survey was sent via e-mail to all RCs participating in MTGs Pituitary, Thyroid, or both. In case of no response, centres that did not respond prior to the given deadline were contacted up to a maximum of three times. The full survey can be found in the supplementary files. The survey was distributed to 51 RCs. Twenty-seven RCs participated in both of the MTGs Pituitary and Thyroid.

Comparison with data obtained for the continuous monitoring program of ERNs (data from 2017 to 2018)
Recently, the EU introduced a continuous monitoring program for all ERNs. This program includes mandatory periodic reporting on 18 general key performance indicators. These are not condition-or ERN-specific. Among these performance indicators are the number of new patients seen per year (to start with 2017 and 2018). Currently, this is a feature to compare numbers between ERNs. In the future, this will likely be used to monitor the performance of the specific ERN as well as individual RCs. These numbers were extracted from the monitoring forms provided to the EC. As in the original network applications, no clear definitions were provided.

Statistical analysis
Descriptive statistics were used to present data. Population data from the EU countries were extracted from the Eurostat website (https://ec.europa.eu/eurostat/ databrowser/view/tps00001/default/table?lang=en, access date 05-02-2020). Statistical analysis was performed using IBM SPSS statistics 25.0 (IBM Corp. Released 2017).

Description of both MTGs
Pituitary is the largest MTG of Endo-ERN and includes 43 RCs from 15 countries, with the highest representation in Italy and the United Kingdom (8 RCs) (Fig. 1A). Thyroid includes 35 RCs from 14 countries, with Germany being the best represented with 7 RCs (Fig. 1B).

Definitions of patients (new patient and patient under chronic care)
Twenty-three unique centres answered the question in our survey on how they defined a new patient in the application forms. One centre (4%) only defined treatment naive patients as new patients. Seventeen centres (74%) defined a new patient only as any patient not previously seen by the RC. Four centres (17%) defined any treatment naive patient as a new patient. One RC (4%) defined a new patient as any patient that was administratively considered a new patient because the interval between the last and present consultation exceeded a period of 1 year, in addition to any patient not previously seen by the RC.
Twenty-four unique centres answered the question of how they defined a patient under chronic care in the application forms. Any chronic patient being a patient under long-term treatment at the reference centre was reported unanimously as a patient under chronic care. In addition, three (13%) centres included patients being referred to their centre for a single consultation only, three (13%) for diagnostic tests only, six (25%) for a specific procedure, and four (17%) as patients with previous treatment in the RC, but referred back to the affiliated centres as chronic patients.
To the question in which way the numbers were obtained, a variety of answers were received. Some centres retrieved numbers from a national database on rare endocrine conditions, some via diagnosis based institutional IT-systems, while others used systems intended for care declarations, and finally personal disease-specific databases were used.  Table 1).

MTG Pituitary
Procedures: The total number of key procedures by centres ranged from 18 to 150/year, with a median of 66 procedures/year. Subtheme pituitary adenoma: All RCs provided medical treatment and multidisciplinary care, 93% of the RCs provide transsphenoidal surgery, and 89% also conventional radiotherapy, while radiosurgery is available in only 36% of centres. Inferior petrosal sinus sampling is offered by 85% of the RCs (Fig. 2). Transsphenoidal surgery is the most frequently performed procedure in patients with pituitary adenoma, with a median number of surgeries performed per centre of 48, 46, and 50 in 2013, 2014, and 2015, respectively (range 9-120/year). Conventional radiotherapy was performed in a median of five, four, and four patients in 2013, 2014, and 2015, respectively (range 0-15/year). Radiosurgery was performed in a median number of three (range 0-24) patients per year in 2013, 2014, and 2015, respectively (Table 1).

MTG Thyroid
Patients: Twenty-four of 35 RCs (69%) completed the survey. The median number of new patients was 50, 56,   Table 1).
Procedures: The total number of key procedures by centres ranged from 20-1376/year, with a median of 150 procedures/year. Subtheme rare genetic thyroid disease: Treatment with novel drugs for rare genetic thyroid disease and personalised therapy was available in 46% and 59% of the RCs, respectively (Fig. 2).
Subtheme congenital hypo-and hyperthyroidism: Neonatal screening was provided by 77% of the RCs, and 55% of the RCs participated in emerging genetic studies with regards to treatment (Fig. 2).
Subtheme thyroid carcinoma: All RCs provided thyroid surgery. Therapy with radioactive iodine was available in 91% of the RCs for those patients with radioactive iodinesensitive thyroid carcinoma. Molecular analysis of the tumour was available in 87% of the RCs (Table 1) (Table 1).

MTG Pituitary
The median number of new patients was 80 and 88 (range 3-919/year) for 2017 and 2018, respectively. The median number of new patients per year was somewhat lower than those reported for 2013-2015. However, the maximum number of new patients reported per year increased threefold in only 3 years' time. In four centres, the reported number of new patients per year had at least doubled since the first application, whereas two centres now reported more than 50 new patients for the first time.

Discussion
This paper outlines the progress within Endo-ERN to establish valuable and reliable measures on the performance of this rare disease network (Fig. 3). This report provides valuable insights into activities and its reporting in the landscape of rare endocrine conditions at the launch of Endo-ERN, in particular for the two MTGs described: Pituitary and Thyroid. Endo-ERN can provide unique, valuable information on clinical activities in the care of rare endocrine conditions throughout Europe. However, there is an urgent need to clearly define key performance indicators, such as new and chronic patients and to have an upfront discussion on which procedures and interventions best indicate an expert centre, to evaluate the performance of, and characterize RCs. In the original, generic application, no definitions on the requested numbers were provided. The survey indicated that a proportion of the variation in performed procedures was among other reasons due to a lack of clear definitions, and consequently, this resulted in the use of different definitions by the RCs. For example, one centre very strictly defined a 'new patient' only as a patient diagnosed in that centre and did not report patients or disease-specific key procedures from its collaborative centre in the same city.  of the second call at least partly overlapped with our study period. Consequently, the documents used in this call do not yet incorporate our findings. Therefore, the second application round was started using the same process as in the first application round. Nevertheless, the reviewers are aware of the identified heterogeneity in numbers and, although not yet incorporated in the formal criteria, the reviewers will request clarification in case of unclearly defined numbers. A mission of Endo-ERN is to capture all rare endocrine conditions throughout the life span in all EU countries. However, this is also an important limitation. A large variety of diseases also means a large variety in needs, complexity, and in mechanisms of care provision. These not only depend on disease group, but also on countryspecific organisation of care. Within a European network, heterogeneity in care organisation will always occur. This prohibits full alignment in care provision between all Endo-ERN centres.
In this process, we have learned that there are local and national variations with respect to diagnosis registration and challenges with extracting patient numbers from hospital files. These considerations have to be taken into account when developing new measures. As a first step, the centres within Endo-ERN need to be aligned on the definition of these measures. Through the European Registries for Rare Endocrine Conditions (EuRRECa) project (https://eurreca.net/, access date 05-02-2020) and in collaboration with the European Society of Endocrinology and the European Society for Paediatric Endocrinology, Endo-ERN has developed an e-reporting platform which can now be used for collecting monthly reports on clinical encounters (https://eurreca. net/e-rec/, access date 05-02-2020). e-REC provides a unique identifier for each new case that has a rare condition covered within Endo-ERN and will provide clarity about new and chronic patients, even in shared care situations. The periodical electronic reporting of cases using e-REC is based on the number of new patients seen per MTG per month and currently is implemented in the RCs. Preliminary data generated from e-REC shows clear variation in reported cases per MTG and per RC (2). Once e-REC is fully implemented, not only will it become an invaluable data for epidemiological surveillance, but this simple activity may also facilitate endorsements of centres as well as targeted condition-specific activities aimed at quality improvement including peer-review programs, such as that introduced in the field of diabetes and endocrinology (https://www.endocrinology.org/ clinical-practice/interdepartmental-peer-review/ and https://www.sweet-project.org/peer-review-program.php, access date 05-02-2020). At present, reference centres are already encouraged to provide numbers for the continuous monitoring program via e-REC. Moreover, the core endocrine registry in EuRRECa complements the e-REC platform by having the functionality that allows collection of generic and condition-specific core outcomes that can be directly reported by patients and health care professionals and used as clinical benchmarks in the long-term. We observed a large range of reported patient numbers by the RCs, that first of all and most likely is influenced by differences in the population size these RCs provide care for. These catchment areas differ tremendously both between RCs and countries. For example, the collaborating centres in the Paris metropolitan area have a coverage of over 12 million inhabitants, which is about nine times the population of Estonia. Moreover, differences in the organisation of the care (for rare diseases) between countries and regions clearly also affect the numbers. For instance, French RCs report on a higher number of patients and procedures as compared with other countries that participate in MTGs Pituitary and Thyroid. A likely explanation for these observations is the specific care structure for rare diseases in France. In 2006, a network for rare endocrine diseases, named Firendo, was installed (http://www.firendo.fr/de/home-firendo-the-french-rareendocrine-disease-network/, access date 05-02-2020). This national '3 layer' network consists of one coordinating reference centre, three constitutive centres, and 29 competence centres, with a structured and pre-defined way in which the referral of patients is organised. The structure of Endo-ERN was partly based on this model. The presence of this national and effective referral network in France can explain, at least in part, the high number of patients seen and procedures performed by the specialized (pituitary) care providers. It is expected that in countries with a similar care structure the reported numbers will be higher because of these intrinsic features of centralisation of care. On the contrary, The Netherlands, for example, has established regional referral systems, but a nationwide system has not yet been developed.
When comparing the number of new patients reported for the 2013-2015 period to the numbers obtained for the continuous monitoring program for ERNs (2017-2018), the median numbers of reported new patients are comparable. However, when looking at the level of individual RCs, significant increases and decreases are reported by several RCs. These could be the consequence of a new referral system or use of adjusted definitions, but also merely reflect real changes in patient numbers and/or procedures. However, without knowledge of the care provided to these patients, it is hard to draw any conclusions on centre performance. If an RC provides complex care only within a decentralised system, this does not necessarily have to result in or reflect on lower quality of care. On the contrary, centralizing all the care for a specific condition will result in higher numbers of new patients, but may not necessarily result in the provision of more complex care by the RC. While a certain surgical volume is important for improved outcomes (3,4), a direct relationship between the volume of patients per hospital and patient outcomes is not clear in all fields (5). Therefore, we advocate the use of specific key procedures in measuring provided care. To acquire clinically meaningful data, the procedures should be prioritized per specific field of expertise, of which the availability and frequency are most important for the evaluation of the performance of an RC. We acknowledge that a higher number of performed procedures will not reflect the quality of this procedure. Therefore, expert discussion is necessary to develop core outcome sets that can be measured. These outcome sets can include clinician-reported outcome measures as well as patient-reported outcome measures, such as health-related quality of life.
In conclusion, we describe the current pathway towards reliably quantifying performance measures within Endo-ERN. The first steps towards this goal have been taken. However, we advocate that definitions should be provided with the questionnaires when evaluating numbers of new and chronic patients. Furthermore, objective benchmarks of clinical practice should be developed for the subthemes of each MTG or MTG-subgroup separately. The number of performed 'core procedures' should be recorded as well as the availability of highly specialized care. This will result in numbers that are suitable for comparison, thereby increasing the ability to evaluate the performance and coverage of Endo-ERN over time.

Declaration of interest
The authors declare that there is no conflict of interest that could be perceived as prejudicing the impartiality of this study.

Funding
Endo-ERN is a European Reference Network co-funded by the European Union's 3rd Health Programme (CHAFEA FPA grant No 739527).