Background: Randomized controlled trials (RCTs) are the gold standard in evaluating the usefulness of potential treatments. For all studies, specific eligibility criteria are relevant to define the enrolled population, but highly selected populations are usually unrepresentative of clinical practice. Thus, external validity (i.e. generalizability or applicability) may be compromised.
Objectives: To determine the generalizability of RCTs in the treatment of systemic sclerosis (SSc) using the Canadian Scleroderma Research Group (CSRG) database.
Methods: Articles related to SSc that were published between 1958 and 2006 (inclusive) were identified using the PubMed database available at http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?DB=pubmed. Protocols from unpublished RCTs were also included. Articles were eliminated on the basis of the following exclusion criteria: not a RCT, mixed population (not enough of subjects having SSc), and not available in English.
Trial quality was scored and key points on trial design, including inclusion and exclusion criteria, were recorded. The inclusion and exclusion criteria were used in conjunction with the CSRG database to determine the number of registered patients who would theoretically be eligible for these trials. In addition, articles were classified into subcategories according to the target system. The CSRG database contains 438 SSc patients from 14 Canadian centers. Inclusion criteria were excluded from the analysis if they were not available or if they had more than 50% of the data missing (except for the pulmonary arterial hypertension criteria, such as right heart catheterization, echocardiogram) from the CSRG database. Results are in median (%) and mean (%) with 95% confidence interval [95% CI].
Results: In total, 210 articles were evaluated and 73 were ultimately selected for inclusion in this paper. The mean % of patients eligible were 35% [95%CI 17, 53] for SSc associated with Raynaud's phenomenon (n=13 trials), 24% [95% CI 1, 47] for digital ulcers (n=6 trials), 48% [95% CI 27, 68] for gastrointestinal (GI)(n=6 trials), 32%[95% CI 20, 43] for overall disease modification (n=25 trials), 6% [95% CI 4,8] for pulmonary arterial hypertension (n=13 trials), 2% [95% CI 0, 4] for interstitial lung disease (n=4 trials) and 38% [95% CI 12, 64] for other categories (n=6 trials). Results in median % and mean % were similar.
We assessed trial quality with a 5 point scale which was combined into 3 subsets. Generalizable trials had less trial quality, confirming the trade-offs between unmet needs (generalizability) and highly selected inclusion/exclusion criteria (p=0.0022).

Conclusion: Except for GI trials, <38% of SSc patients were allowed into trials. SSc lung disease studies had the worst external validity. We need RCT design that includes many patients with SSc as there is a large burden of disease and trials typically do not include the majority of patients.