External inspection of compliance with standards for improved healthcare outcomes: Cochrane systematic review
Assessed as up to date: 2016/02/11
Inspection systems are used in healthcare to promote quality improvements (i.e. to achieve changes in organisational structures or processes, healthcare provider behaviour and patient outcomes). These systems are based on the assumption that externally promoted adherence to evidence-based standards (through inspection/assessment) will result in higher quality of healthcare. However, the benefits of external inspection in terms of organisational-, provider- and patient-level outcomes are not clear. This is the first update of the original Cochrane review, published in 2011.Objectives
To evaluate the effectiveness of external inspection of compliance with standards in improving healthcare organisation behaviour, healthcare professional behaviour and patient outcomes.Search methods
We searched the following electronic databases for studies up to 1 June 2015: the Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE, Embase, Database of Abstracts of Reviews of Effectiveness, HMIC, ClinicalTrials.gov and the World Health Organization International Clinical Trials Registry Platform. There was no language restriction and we included studies regardless of publication status. We also searched the reference lists of included studies and contacted authors of relevant papers, accreditation bodies and the International Organization for Standardization (ISO), regarding any further published or unpublished work. We also searched an online database of systematic reviews (PDQ-evidence.org).Selection criteria
We included randomised controlled trials (RCTs), non-randomised trials (NRCTs), interrupted time series (ITSs) and controlled before-after studies (CBAs) evaluating the effect of external inspection against external standards on healthcare organisation change, healthcare professional behaviour or patient outcomes in hospitals, primary healthcare organisations and other community-based healthcare organisations.Data collection and analysis
Two review authors independently applied eligibility criteria, extracted data and assessed the risk of bias of each included study. Since meta-analysis was not possible, we produced a narrative results summary. We used the GRADE tool to assess the certainty of the evidence.Main results
We did not identify any new eligible studies in this update. One cluster RCT involving 20 South African public hospitals and one ITS involving all acute hospital trusts in England, met the inclusion criteria. A trust is a National Health Service hospital which has opted to withdraw from local authority control and be managed by a trust instead.
The cluster RCT reported mixed effects of external inspection on compliance with COHSASA (Council for Health Services Accreditation for South Africa) accreditation standards and eight indicators of hospital quality. Improved total compliance score with COHSASA accreditation standards was reported for 21/28 service elements: mean intervention effect was 30% (95% confidence interval (CI) 23% to 37%) (P < 0.001). The score increased from 48% to 78% in intervention hospitals, while remaining the same in control hospitals (43%). The median intervention effect for the indicators of hospital quality of care was 2.4% (range -1.9% to +11.8%).
The ITS study evaluated compliance with policies to address healthcare-acquired infections and reported a mean reduction in MRSA (methicillin-resistant Staphylococcus aureus) infection rates of 100 cases per quarter (95% CI -221.0 to 21.5, P = 0.096) at three months' follow-up and an increase of 70 cases per quarter (95% CI -250.5 to 391.0; P = 0.632) at 24 months' follow-up. Regression analysis showed similar MRSA rates before and after the external inspection (difference in slope 24.27, 95% CI -10.4 to 58.9; P = 0.147).
Neither included study reported data on unanticipated/adverse consequences or economic outcomes. The cluster RCT reported mainly outcomes related to healthcare organisation change, and no patient reported outcomes other than patient satisfaction.
The certainty of the included evidence from both studies was very low. It is uncertain whether external inspection accreditation programmes lead to improved compliance with accreditation standards. It is also uncertain if external inspection infection programmes lead to improved compliance with standards, and if this in turn influences healthcare-acquired MRSA infection rates.Authors' conclusions
The review highlights the paucity of high-quality controlled evaluations of the effectiveness and the cost-effectiveness of external inspection systems. If policy makers wish to understand the effectiveness of this type of intervention better, there needs to be further studies across a range of settings and contexts and studies reporting outcomes important to patients.
Flodgren Gerd, Gonçalves-Bradley Daniela C, Pomey Marie-Pascale
Can third-party inspections of whether healthcare organisations are fulfilling mandatory standards improve healthcare outcomes?
What is the aim of this review?
The aim of this Cochrane Review was to find out if external inspection of compliance with standards can improve improving healthcare organisation behaviour, healthcare professional behaviour and patient outcomes. Cochrane researchers collected and analysed all relevant studies to answer this question and found two studies.
It is unclear whether third-party inspection programmes designed to measure a healthcare organisation's compliance with standards of care can improve professional practice and healthcare outcomes. There was little information on patient outcomes. This review highlights the lack of high-quality evaluations.
What was studied in the review?
Third-party (external) inspection programmes are used within healthcare settings (e.g. clinics and hospitals) to increase the compliance with evidence-based standards of care, but very little is known of their benefits in terms of organisational performance (e.g.,waiting list times, inpatient length of stay),the performance of healthcare professionals (e.g. referral rate, prescribing rate) and patient outcomes (e.g. mortality, and condition specific outcomes like blood glucose for diabetic patients, or weight loss for overweight or obese patients), or their cost-effectiveness. Accreditation programmes are one example of an external review system. Accreditation is a process of review that healthcare organisations participate in to demonstrate the ability to meet predetermined criteria and standards of accreditation established by a professional accrediting agency. Infection control inspection programmes are also examples. Designed to reduce infection, standards based on evidence and best practice, are used to improve care quality and safety to decrease healthcare-acquired infection rates (also called a hospital acquired infection or nosocomial infection, is an infection that is acquired at a hospital or another healthcare setting). If the standards are not adhered to, the external inspection body can take actions to reinforce them.
What are the main results of the review?
The review authors searched the literature for studies evaluating the effects of external inspection of compliance with standards and found two relevant studies: one study involving 20 hospitals in the Republic of South Africa and one study providing time series data (a sequence of outcome measurements taken at successive equally spaced points in time) involving all acute hospital trusts in England (a trust is a National Health Service hospital which has opted to withdraw from local authority control and be managed by a trust instead). The comparison was no inspection.
One study reported improved compliance scores with hospital accreditation standards. However, it is uncertain whether external inspection leads to improved compliance with standards because the certainty of the evidence was very low. Only one of the nine intervention hospitals achieved accreditation status during the study period.
Another study reported on the effects of an infection control inspection programme. This programme was commissioned in the UK to reduce infection rates of MRSA (methicillin-resistant Staphylococcus aureus, which is a form of bacterial infection that is resistant to many antibiotics). However, the inspection programme was only one element of a wider range of methods being applied to infection control in the UK National Health Service at that time. Even before the introduction of the inspection programme, the infection rates appeared to be decreasing - but the introduction of the inspection programme did not accelerate this decrease. It is also uncertain whether the Healthcare Commission's Healthcare Associated Infections Inspection Programme may lead to lower MRSA infection rates or not because the certainty of the evidence was very low.
How up-to-date is this review?
The review authors searched for studies that had been published up to June 2015.
Implications for practice
In terms of considering quality of care delivered across a whole healthcare system, external inspection (as defined for this review) as opposed to voluntary inspection, has the advantage of incorporating all organisations rather than only volunteer organisations. The trend today is towards more mandatory government-mandated accreditation systems (Accreditation Canada 2015). For those running a healthcare system this is a very attractive advantage and it is likely that external inspection will continue to be used. Situations where this occurs offer a useful opportunity to better define the effects of such processes, the optimal configuration of inspection processes and their value for money. Results of a recent survey shows that a sustainable organisation typically complement regulation mechanisms, funding or governmental commitment to quality and health-care improvement that offer a supportive environment (Shaw 2013).
Implications for research
This review update identified no new eligible studies for inclusion in addition to the two studies previously included. If policy makers wish to understand the effectiveness of this type of intervention better, then there needs to be further studies across a range of settings and contexts. There does not seem to be any prima facie reason for not conducting a trial; however, if it is felt that an experimental design cannot be used then other non-randomised designs, such as ITS designs, could be used as such designs offer a useful way of interpreting the data.
Whatever design is used, including an appropriate follow-up period is important to examine whether any improvements observed after the external inspection endure. Any studies should endeavour to include outcomes important to patients and preferable also an economic evaluation.
One of the studies experienced some problems during data collection, and had to drop some important outcomes (Salmon 2003). Researchers and inspecting bodies should ensure that inspection and data collection are conducted using standardised and validated instruments (Tuijn 2011).Get full text at The Cochrane Library
Evidence Central is an integrated web and mobile solution that helps clinicians quickly answer etiology, diagnosis, treatment, and prognosis questions using the latest evidence-based research. Learn more.