For all these reasons, the COVID Tracking Project stopped reporting a national summary of recovery figures in November and decided to remove state-level recovery figures from our website in January. Instead of providing figures for recoveries, we began to track and display hospital discharges for the eight states providing those data, which had a clearer, more standardized meaning across states. As we wrote about state recovery metrics, our recommendation is that state health officials carefully consider how they discuss and quantify this information, choosing metrics such as “released from isolation” or “inactive cases” over labels that imply full recovery.
What we have learned, and what we hope happens next
Over the past two months, a small crew at the COVID Tracking Project has been working to document our year of data collection, reflecting on how best to organize our project’s history so that journalists, policy makers, advocates, and the public might continue to find relevance in our work.
As we pored over our research on state reporting, we congealed our findings into a set of common reporting problems that made COVID-19 data especially difficult to aggregate on a national level. States tended to differ on how they defined data, what data they made available, and how they presented what data they did publish, making it difficult to compare data across states. All of those themes come through in the reporting arcs of these five COVID-19 metrics.
Some of these problems could have been avoided with clearer reporting guidance from the federal government; others were inevitable, given the constraints of the United States’ underfunded public-health infrastructure. But all of them tended to be poorly documented, meaning it took a great deal of excavation to uncover the sources of these problems—or even the existence of the problems themselves.
These data challenges may have been readily apparent to or expected by those familiar with the contours of public-health informatics. But pandemics affect us all, and the infrastructure that responds to them is meant to protect us all, so we all deserve to understand how capable the infrastructure is. Frankly, we need to understand its limitations to navigate through a pandemic.
Above and beyond any individual reporting practice, we believe that it was the lack of explanations from state governments and, most crucially, the CDC that led to misuse of data and wounded public trust. We tried our best to provide explanations where possible, and we saw transformation when we were able to get the message across to the public. Data users who were frustrated or even doubtful came to trust the numbers. Journalists reported more accurately. Hospitals could better anticipate surges.
If we could make just one change to the way state and federal COVID-19 data were reported, it would be to make an open acknowledgment of the limitations of public-health-data infrastructure whenever the data is presented. And if we could make one plea for what comes next, it’s that these systems receive the investment they deserve.
This article has been adapted from its original version, which can be read in full at The COVID Tracking Project.