December's WHO calculations of excess deaths during the pandemic highlight a big, but largely unseen, problem in global health: Lack of proper mortality surveillance in half the world's countries.

In December, the World Health Organization (WHO) published its most extensive report yet on excess deaths over the first two years of the pandemic, estimating that some 14.8 million more people died than normal between 2020 and 2021.

This figure indicates Covid surpassed coronary heart disease as the world's biggest killer in 2021. The total number of excess deaths in 2020 and 2021is almost triple the organization's calculation of deaths directly attributed to Covid — around 5.42 million — over the two-year period.

Other entities tracking excess deaths over the course of the pandemic have made their own calculations, and the WHO's estimate marks the most conservative of them all. In a March 2021 article published in the Lancet, for example, researchers at the University of Washington School of Medicine's Institute for Health Metrics and Evaluation (IHME) reported excess death rates over the course of the pandemic close to 18 million.

The Economist, which has also been tracking global deaths from the near start of the pandemic, reported around 16 million deaths.

The discrepancies between the models has a lot to do with how they handle the 50% of countries with no data to report.

But first, what is excess death and why is it being calculated?

Early on in the pandemic, statisticians looking to measure the number of people who'd died of the SARS-CoV-2 virus quickly found that merely trying to calculate deaths attributed to the virus barely scratched the surface at demonstrating the pandemic's impact on a global scale.

Although some countries, like the United Kingdom, did fairly well at accurately reporting Covid-related deaths, many didn't, and for others, reporting deaths due to Covid was impossible.

That's because especially near the start of the pandemic, many countries didn't have adequate testing supplies, making it impossible to say whether a death could be attributed to Covid or not. And even the countries with testing supplies may not have reported Covid as the cause of death if a deceased person already had another illness, like cancer.

Further, even if Covid deaths were perfectly reported and calculated for every country in the world, they wouldn't take into account the entire burden of the disease — which includes knock-on effects like its impact on mental health and health system overload.

The WHO made its excess death calculations using monthly all-cause mortality (ACM) data. This is data reported by doctors to local death registers. Using algorithms calculating how many deaths would have occurred in a "normal" year based on past mortality statistics, researchers were able to get a rough idea of how many deaths that occurred in 2020 and 2021 were abnormal, or excess.

What the calculations showed and what they didn't

Excess death is an especially useful metric for calculating pandemic burden in high-income countries with adequate health surveillance systems, like the US, most European countries and Japan.

But only around half the countries in the world report monthly ACM data usable for the WHO researchers. Other countries only report annual mortality, subnational data or none at all.

This is due to a number of factors. Some countries, for example, collect ACM data, but do not use digitized systems, making the transfer of information difficult for the WHO to collect in a timely fashion.

And other countries simply don't track ACM data at all. The mere existence of this demographic information implies the existence of a central office or authority that tracks individual deaths. In many countries, these offices don't exist, and even if they do, many deaths will not be reported to them.

For example: Monthly ACM data was only available for 13% of countries on the African continent, 18% in southeast Asia and 33% of countries in the Western Pacific region.

Near the start of the pandemic, some of the global mortality models made it appear like was very little Covid in these data-poor countries. But this was partly due to the fact that these countries simply didn't have the means to collect the data. So it appeared that Covid simply hadn't hit countries like Zambia, where ACM data is not collected.

But a study conducted by researchers in Zambia inspecting morgues in Lusaka showed that more than 32% of the deceased bodies in the sample tested positive for Covid antibodies, indicating the disease was widespread despite the low numbers in international models.

Prediction frameworks

To avoid conveying the false impression that COVID had spared data-poor countries, the researchers made predictions.

For countries with some data for specific cities, they extrapolated the available numbers to the entire country.

For countries with no data, they estimated the number of excess deaths based on trends from (mostly high-income) countries with ACM data. Using algorithms, the researchers included country-specific factors such as socioeconomic situation, COVID susceptibility and how government authorities responded to the pandemic.

This offered a general baseline estimate of the country's number of excess deaths.

These predictive inferences were "problematic" but necessary to get a general picture of the global havoc wreaked by the pandemic, Enrique Acosta, a researcher at the Centre for Demographic Studies in Spain and the Max Planck Institute for Demographic Research in Germany, wrote in a response to the WHO report.

"However speculative these estimates, most are surely closer to the truth than are officially reported numbers of deaths from Covid-19," he wrote. "To rely on confirmed deaths would imply that the pandemic spared low-income and lower-middle-income countries — vulnerable populations that have limited capacity for testing and response. This assumption is highly implausible, and even irresponsible."

How is death from other global illnesses calculated then?

Just because many countries in the world don't collect ACM data doesn't necessarily mean health researchers are completely in the dark about mortality in these places, Jonas Schoeley, a researcher at the Max Planck Institute for Demographic Research, told DW.

Health surveys can also offer insight into mortality statistics in countries without a death registry.

"A representative sample of households in a given region is selected and household members are interviewed with questions covering a wide spectrum, including fertility, mortality and disease histories," he said. "From these surveys, measures like child mortality, AIDS incidence or fertility rates can be estimated."

Still, he said, unlike registry data, these surveys only cover samples of a country's population, making the results more uncertain and potentially biased.

The best metric is still what will perhaps remain the hardest to procure: Real-time monitoring of global mortality through centralized systems in low and middle income countries worldwide, according to Acosta. "The construction of such systems will require considerable global efforts," Acosta wrote. "But, once built, they will serve as an essential early warning for future pandemics and health crises."