Dr Stephen Bolsin had only just started working at Bristol Royal Infirmary when he began to suspect something was wrong.
It started in 1989, when he noticed that heart operations on children were taking much longer than normal. A decade later – after a full-blown audit and the biggest public inquiry ever into the “workings of the NHS” – it turned out that more than 170 babies could have been spared if they had had surgery at a different hospital.
The Bristol Heart Scandal forever altered the way mortality rates in hospitals are monitored, and spurred the government to publish UK heart surgeon’s individual mortality rates publicly, in an attempt to make doctors more accountable.
Since cardiac surgeons started to publish their mortality data, there has been an ongoing debate about whether or not this is a good thing.
While some evidence has shown that this initiative has lowered mortality rates, other experts argue that there are unintended negative consequences. In a recent letter, a group of heart surgeons warned that publishing data was:
“…having a damaging effect on individual surgeons, with destruction of confidence, disruption of functional teams and inappropriate suspensions, with unfair media attention.”
Thankfully, there has been no evidence to suggest anything similar has happened with cancer patients, but it begs the question: should other surgical areas be made to publish their mortality data too? And if so, would it lead to any improvements?
Cardiac to cancer
Professor Arnie Purushotham, a breast surgeon and Cancer Research UK’s senior clinical adviser, thinks they should – particularly when it comes to cancer surgery.
“Patients are beginning to expect this information – but it’s not available,” he argues. “We need to look ahead, because there are variations in outcomes and this might point to a bigger systematic problem.”
England is still lagging behind the best performing countries in patient survival. And while this is partly due to our problems diagnosing cancer early, large variations in treatment also play a role.
For example, the National Cancer Intelligence Network (NCIN) looked at mortality rates in the 30 days after bowel cancer patients had surgery and found worrying variation between hospitals across England. This so-called ‘30-day post-operative mortality’ is thought to be in indicator of the quality of an operation.
And often the elderly, patients with advanced staged disease, or the socio-economically deprived patients fare worse because they miss out on surgery. And it’s not just bowel cancer patients who see an imbalance in care.
So it’s clear we need to do something to rectify the inequalities in care and help patients live longer. Although we’ve suggested before that publishing data could improve this, we first need to determine the best measure(s) to publish.
Singled out or grouped together
Some mortality rates are already published for cancer surgery but, there are a number of associated factors that influence the patient’s outcome, so publishing just the mortality rates won’t paint the full picture.
Prof Purushotham explains: “You can’t attribute mortality rates to the surgeon alone.”
So many other things play a role in determining the patient’s outcome and their care after surgery – the type of tumour, the cancer stage, the complexity of the surgery itself, the prescription of treatments like chemotherapy and radiotherapy and any other medical issues the patient may have– known as co-morbidity.
On top of this, the quality of the multi-disciplinary team (MDT) that delivers this complex cancer care plays a crucial part in how a patientfares. .
Prof Purushotham suggests that comparing MDT or hospital-level data, instead of data on individual surgeons, might be a better indicator of cancer surgery quality.
But he cautions: “The analysis needs to be sophisticated, otherwise mediocrity becomes the better outcome measure”.
In other words, top-tier hospitals that take on complex cases because they’re more experienced might, due to the difficulty of the care their patients’ need, seem less ‘good’ than hospitals that take on less complex cases, where patients generally do better. So the latter reap the rewards of better-looking outcomes, while the better hospitals seem worse.
But getting the data needed for that level of sophistication is no easy feat.
Crude measures
Thanks to the modernisation of the National Cancer Registration Service (NCRS), data collection in the UK has improved greatly in the last few years.
England, in particular, now has one of the most complete cancer data collections in the world. But it still lacks information on several areas that would be crucial to analysis of cancer outcomes.
“We have good robust data about the diagnosis, the surgical intervention and mortality outcomes,” says Professor Henrik Møller, cancer epidemiologist from King’s College London.
“The things that are less well-recorded are the characteristics about the subtype of disease, the stage, the grade and the aggressiveness of the tumour. But we’ve become better at recording that in recent times.”
And then there are the other things that influence how well a patient fares, like overall their fitness, where the available data is more of a “rough estimate”.
“The idea that you can somehow, with statistical adjustment, provide a fair comparison of different hospitals or different surgeons, is actually an illusion,” – Prof Henrik Møller
And while the independent Cancer Taskforce has called for better data collection to be a priority, even if better quality data emerges Møller still doesn’t think an accurate or fair comparison between hospitals, MDTs or doctors is easily achievable.
“The idea that you can somehow, with statistical adjustment, provide a fair comparison of different hospitals or different surgeons, is actually an illusion,” he says.
Møller and other experts argue that it’s idealistic to assume that one could adequately isolate the varitions in the quality of care by adjusting comparisons of patients’ risks and other factors.
This, he explains, is because of how simplistic the measurement of the data is and how it fails to take into account any of the nunances, such as patient preference, needed to make a proper comparison.
He also highlights that the nature of the patient pathway means patients often aren’t randomly assigned to a hospital or clinician, but rather deliberately assigned to one that suits that patient’s individual needs. And this, Møller says, can bias the data, leading to potentially incorrect conclusions about which hospital or MDT is preforming better.
He also thinks there could be negative consequences to publishing the outcome data, which we need to be careful to avoid.
“Having surgeons’ ‘league tables’ could drive down quality,” he says. “In the case of lung cancer it could be that fewer patients will be offered an operation because surgeons will be more selective about whom they operate on – and we don’t want that. We want there to be more lung cancer operations because, right now, we are operating on too few.”
But Prof Purushotham thinks it could equally have the opposite effect.
“Publishing high quality, accurate data could not only lead to better practice but overall raise the quality of standards of surgery,” he says, explaining that the data could highlight under-performing hospitals and identify ways in which they could improve.
Next Steps
Despite these problems with creating accurate and fair outcome measures for cancer surgery, we think such a measures would be very useful: the people who plan and decide where the funding goes for health care services, otherwise known as the NHS commissioners, increasingly have to base their decisions on outcome data.
Because of this, we’re leading a project to take forward a recommendation made last year in the Cancer Strategy for England, which tasked Cancer Research UK to work with the Royal College of Surgeons and government organisations to figure out what these outcome measures should be.
As part of this project, we’re holding a workshop with patients and clinicians later this week to discuss and agree the measures needed to determine quality in surgery for breast, bowel, oesophageal, stomach and lung cancer.
We’ll be thinking about how best to collect and analyse these measures in England – but also how to use this data responsibily.
We know this won’t be easy but we think it’s worth trying to overcome the obstacles for the benefit of the patients, if nothing else.
By making this information available, we hope hospitals will be held to account for the quality of surgery services they provide. Ultimately, we hope this will lead to improved care of cancer patients, and reduced variation in care between hospitals.
So while the ‘Goldilocks’ measure – not too simplistic as to mislead, not too complex as to be impossible – may be difficult to find, we hope the meeting will begin get us closer to a measure that, for patients and doctors alike, is ‘just right’.
– Misha
from Cancer Research UK - Science blog http://ift.tt/1JF3QZl
Dr Stephen Bolsin had only just started working at Bristol Royal Infirmary when he began to suspect something was wrong.
It started in 1989, when he noticed that heart operations on children were taking much longer than normal. A decade later – after a full-blown audit and the biggest public inquiry ever into the “workings of the NHS” – it turned out that more than 170 babies could have been spared if they had had surgery at a different hospital.
The Bristol Heart Scandal forever altered the way mortality rates in hospitals are monitored, and spurred the government to publish UK heart surgeon’s individual mortality rates publicly, in an attempt to make doctors more accountable.
Since cardiac surgeons started to publish their mortality data, there has been an ongoing debate about whether or not this is a good thing.
While some evidence has shown that this initiative has lowered mortality rates, other experts argue that there are unintended negative consequences. In a recent letter, a group of heart surgeons warned that publishing data was:
“…having a damaging effect on individual surgeons, with destruction of confidence, disruption of functional teams and inappropriate suspensions, with unfair media attention.”
Thankfully, there has been no evidence to suggest anything similar has happened with cancer patients, but it begs the question: should other surgical areas be made to publish their mortality data too? And if so, would it lead to any improvements?
Cardiac to cancer
Professor Arnie Purushotham, a breast surgeon and Cancer Research UK’s senior clinical adviser, thinks they should – particularly when it comes to cancer surgery.
“Patients are beginning to expect this information – but it’s not available,” he argues. “We need to look ahead, because there are variations in outcomes and this might point to a bigger systematic problem.”
England is still lagging behind the best performing countries in patient survival. And while this is partly due to our problems diagnosing cancer early, large variations in treatment also play a role.
For example, the National Cancer Intelligence Network (NCIN) looked at mortality rates in the 30 days after bowel cancer patients had surgery and found worrying variation between hospitals across England. This so-called ‘30-day post-operative mortality’ is thought to be in indicator of the quality of an operation.
And often the elderly, patients with advanced staged disease, or the socio-economically deprived patients fare worse because they miss out on surgery. And it’s not just bowel cancer patients who see an imbalance in care.
So it’s clear we need to do something to rectify the inequalities in care and help patients live longer. Although we’ve suggested before that publishing data could improve this, we first need to determine the best measure(s) to publish.
Singled out or grouped together
Some mortality rates are already published for cancer surgery but, there are a number of associated factors that influence the patient’s outcome, so publishing just the mortality rates won’t paint the full picture.
Prof Purushotham explains: “You can’t attribute mortality rates to the surgeon alone.”
So many other things play a role in determining the patient’s outcome and their care after surgery – the type of tumour, the cancer stage, the complexity of the surgery itself, the prescription of treatments like chemotherapy and radiotherapy and any other medical issues the patient may have– known as co-morbidity.
On top of this, the quality of the multi-disciplinary team (MDT) that delivers this complex cancer care plays a crucial part in how a patientfares. .
Prof Purushotham suggests that comparing MDT or hospital-level data, instead of data on individual surgeons, might be a better indicator of cancer surgery quality.
But he cautions: “The analysis needs to be sophisticated, otherwise mediocrity becomes the better outcome measure”.
In other words, top-tier hospitals that take on complex cases because they’re more experienced might, due to the difficulty of the care their patients’ need, seem less ‘good’ than hospitals that take on less complex cases, where patients generally do better. So the latter reap the rewards of better-looking outcomes, while the better hospitals seem worse.
But getting the data needed for that level of sophistication is no easy feat.
Crude measures
Thanks to the modernisation of the National Cancer Registration Service (NCRS), data collection in the UK has improved greatly in the last few years.
England, in particular, now has one of the most complete cancer data collections in the world. But it still lacks information on several areas that would be crucial to analysis of cancer outcomes.
“We have good robust data about the diagnosis, the surgical intervention and mortality outcomes,” says Professor Henrik Møller, cancer epidemiologist from King’s College London.
“The things that are less well-recorded are the characteristics about the subtype of disease, the stage, the grade and the aggressiveness of the tumour. But we’ve become better at recording that in recent times.”
And then there are the other things that influence how well a patient fares, like overall their fitness, where the available data is more of a “rough estimate”.
“The idea that you can somehow, with statistical adjustment, provide a fair comparison of different hospitals or different surgeons, is actually an illusion,” – Prof Henrik Møller
And while the independent Cancer Taskforce has called for better data collection to be a priority, even if better quality data emerges Møller still doesn’t think an accurate or fair comparison between hospitals, MDTs or doctors is easily achievable.
“The idea that you can somehow, with statistical adjustment, provide a fair comparison of different hospitals or different surgeons, is actually an illusion,” he says.
Møller and other experts argue that it’s idealistic to assume that one could adequately isolate the varitions in the quality of care by adjusting comparisons of patients’ risks and other factors.
This, he explains, is because of how simplistic the measurement of the data is and how it fails to take into account any of the nunances, such as patient preference, needed to make a proper comparison.
He also highlights that the nature of the patient pathway means patients often aren’t randomly assigned to a hospital or clinician, but rather deliberately assigned to one that suits that patient’s individual needs. And this, Møller says, can bias the data, leading to potentially incorrect conclusions about which hospital or MDT is preforming better.
He also thinks there could be negative consequences to publishing the outcome data, which we need to be careful to avoid.
“Having surgeons’ ‘league tables’ could drive down quality,” he says. “In the case of lung cancer it could be that fewer patients will be offered an operation because surgeons will be more selective about whom they operate on – and we don’t want that. We want there to be more lung cancer operations because, right now, we are operating on too few.”
But Prof Purushotham thinks it could equally have the opposite effect.
“Publishing high quality, accurate data could not only lead to better practice but overall raise the quality of standards of surgery,” he says, explaining that the data could highlight under-performing hospitals and identify ways in which they could improve.
Next Steps
Despite these problems with creating accurate and fair outcome measures for cancer surgery, we think such a measures would be very useful: the people who plan and decide where the funding goes for health care services, otherwise known as the NHS commissioners, increasingly have to base their decisions on outcome data.
Because of this, we’re leading a project to take forward a recommendation made last year in the Cancer Strategy for England, which tasked Cancer Research UK to work with the Royal College of Surgeons and government organisations to figure out what these outcome measures should be.
As part of this project, we’re holding a workshop with patients and clinicians later this week to discuss and agree the measures needed to determine quality in surgery for breast, bowel, oesophageal, stomach and lung cancer.
We’ll be thinking about how best to collect and analyse these measures in England – but also how to use this data responsibily.
We know this won’t be easy but we think it’s worth trying to overcome the obstacles for the benefit of the patients, if nothing else.
By making this information available, we hope hospitals will be held to account for the quality of surgery services they provide. Ultimately, we hope this will lead to improved care of cancer patients, and reduced variation in care between hospitals.
So while the ‘Goldilocks’ measure – not too simplistic as to mislead, not too complex as to be impossible – may be difficult to find, we hope the meeting will begin get us closer to a measure that, for patients and doctors alike, is ‘just right’.
– Misha
from Cancer Research UK - Science blog http://ift.tt/1JF3QZl
