Pulse oximeter accuracy in different ethnic groups: Respiratory Futures talked to Professor Dominick Shaw and Dr Andrew Fogarty, clinical academics at the University of Nottingham about concerns around racial bias in clinical vital signs monitoring.
Can you tell us about vital sign monitoring and its relation to early warning scores?
Measuring physiological processes (vital sign monitoring) is the foundation upon which clinical care is based. Measures of heart rate, blood pressure, temperature and blood oxygen saturation (normally evaluated using pulse oximetry) are used to assess disease severity, monitor for deterioration, guide therapy and allocate medical resources globally.
In the last 10 years the development and implementation of early warning scores such as the National Early Warning Score (NEWS) [1], which combine vital sign observations into summary scores has become commonplace in hospitals. Early warning scores are now used as a patient safety net, resulting in the common refrain that the patient “is scoring”, i.e. their early warning score is high suggesting an intervention or senior review is urgently required. In the typical patient journey through a health care system, vital sign monitoring occurs in primary care, at paramedic assessment, on admission to hospital, and throughout the hospital stay, and into virtual wards after discharge.
What are your concerns about vital sign monitoring?
Given vital sign monitoring is the bedrock of care, clinicians may assume it is precise and validated. This is not the case. Real-world data suggest that pulse oximetry is inaccurate in patients with darker skin [2,3], particularly in the clinically important range of 85% to 89% where decisions on treatment escalation are often made. The effect size is not insignificant, with pulse oximetry overreading by approximately 5%, compared to arterial blood gas-derived saturations in individuals with a recorded Black, Asian or mixed ethnicity [4].
Concerns about pulse oximeter accuracy in different ethnic groups have been known about for decades, but in the context of the Covid pandemic, differential measurement error is not simply an academic concern and is likely to be clinically detrimental. Given that the United Nations estimate 1.3 billion people live in Africa, and 4.7 billion people in Asia, the potential deleterious effect of pulse oximetry inaccuracy on clinical outcomes is extremely concerning.
What is causing the error in measurement?
The cause of the inaccuracy related to skin tone is thought to be melanin interfering with the light signals that differentiate between oxygenated and deoxygenated haemoglobin, but the approvals methodology used to certify pulse oximetry should have identified this effect. We know that skin colour affects pulse oximetry accuracy but the standards for approving pulse oximetry devices do not require evidence of accuracy from diverse skin tones.
For example The International Organization for Standardization states that to approve a pulse oximetry device (ISO 80601-2-61), pulse oximetry accuracy (SpO2) should be obtained from controlled desaturation studies down to between 70% and 100% arterial saturation (SaO2) using simultaneously drawn arterial blood and have an accuracy within 3% [5].
However, clinicians will be surprised to learn that the desaturation study only requires healthy volunteers, with 5 samples at 4 incremental saturations and enough subjects to achieve statistical significance, which can be only 10 subjects in total (i.e. 20 blood samples each). Given the specification mentions other factors affecting accuracy including finger motion and low perfusion, does not require samples to be from representative populations and only small sample sizes for standards to be met, one can see how extrapolation from a small healthy control study bears little resemblance to real-life clinical medicine. Data from pulse oximeters using multiple light wavelengths, (rather than two) is more reassuring [6] but these oximeters are no longer widely used.
Are there any other contributing factors to the issue?
The problem of racial bias and patient monitoring may not just be limited to measurements of oxygen saturation. Recent data suggest that temperature measurement may also be confounded by skin tone. Bhavani et al. looked at temporal artery (forehead) thermometer measurements compared to oral temperature measurements. They found that temporal measurement was associated with a lower odds of identifying fever in Black patients, while there was no significant difference in White patients [7].
If unchecked assumptions about the accuracy of vital sign monitoring are allowed, market forces will continue to outstrip approvals processes. Patients now check their SpO2 at home using cheap pulse oximeters (available online for £12), and the move to smartwatches and wearable technologies threatens to reinforce the potentially erroneous notion that technology designed for lifestyle choices can be used to diagnose and guide medical therapy. Whilst there are examples of the potential for this approach to identify important abnormalities, the underlying accuracy of wearable and vital sign technologies needs more robust approvals. Unfortunately to the best of our knowledge, no physiological simulators have been proven adequate for pulse oximetry calibration.
What can be done to help solve this issue?
The issue of racial differences in vital sign monitoring has been hidden in plain sight for many years. The question now is not is there a problem, but how do we address it? An independent review of the evidence has been called for [8] but a regulatory reset and better testing of vital sign technologies performance in real-world studies, across a range of populations, is needed, along with wider dissemination of information about the issue.
The consequences of differential pulse oximetry error are likely to be that those with darker skin tones had higher apparent oxygen saturations than in reality. During the Covid-19 pandemic, this may have delayed admission to hospital, delayed the initiation of oxygen, and delayed the prescription of dexamethasone and tocilizumab, which reduce mortality. Pulse oximetry inaccuracy is likely to have contributed to the fact that patients with darker skin tones were sicker at time of transfer to intensive care [9] and had higher mortality rates than patients with White skin [10].
This feature was also informed by contributions from Dr Dan Clark, Nottingham University NHS Hospitals Trust.
References
1 Forster S, McKeever TM, Churpek M, et al. Predicting outcome in acute respiratory admissions using patterns of National Early Warning Scores. Clin Med 2022;22:409–15.
2 Sjoding MW, Dickson RP, Iwashyna TJ, et al. Racial Bias in Pulse Oximetry Measurement. N Engl J Med 2020;383:2477–8.
3 Valbuena VSM, Seelye S, Sjoding MW, et al. Racial bias and reproducibility in pulse oximetry among medical and surgical inpatients in general care in the Veterans Health Administration 2013-19: multicenter, retrospective cohort study. BMJ 2022;378:e069775.
4 Crooks CJ, West J, Morling JR, et al. Pulse oximeter measurements vary across ethnic groups: an observational study in patients with COVID-19. Eur Respir J 2022;59. doi:10.1183/13993003.03246-2021
5 BSI Standards Publication. Part 2-61: Particular requirements for basic safety and essential performance of pulse oximeter equipment (ISO 80601-2-61:2011). BS EN ISO 80601-2-61:2011
6 Bothma PA, Joynt GM, Lipman J, et al. Accuracy of pulse oximetry in pigmented patients. S Afr Med J 1996;86:594–6.
7 Bhavani SV, Wiley Z, Verhoef PA, et al. Racial Differences in Detection of Fever Using Temporal vs Oral Temperature Measurements in Hospitalized Patients. JAMA 2022;328:885–6.
8 Equity in medical devices: independent review call for evidence. GOV.UK. 2022.https://www.gov.uk/government/consultations/equity-in-medical-devices-independent-review-call-for-evidence (accessed 29 Nov 2022).
9 Crooks CJ, West J, Morling JR, et al. Differential pulse oximetry readings between ethnic groups & delayed transfer to intensive care units. QJM Published Online First: 6 September 2022. doi:10.1093/qjmed/hcac218
10 Wong A-KI, Charpignon M, Kim H, et al. Analysis of Discrepancies Between Pulse Oximetry and Arterial Oxygen Saturation Measurements by Race and Ethnicity and Association With Organ Dysfunction and Mortality. JAMA Netw Open 2021;4:e2131674.
Further resources
The practical uses of pulse oximetry in primary care – Primary Care Respiratory Society