aster.cloud aster.cloud
  • /
  • Platforms
    • Public Cloud
    • On-Premise
    • Hybrid Cloud
    • Data
  • Architecture
    • Design
    • Solutions
    • Enterprise
  • Engineering
    • Automation
    • Software Engineering
    • Project Management
    • DevOps
  • Programming
    • Learning
  • Tools
  • About
  • /
  • Platforms
    • Public Cloud
    • On-Premise
    • Hybrid Cloud
    • Data
  • Architecture
    • Design
    • Solutions
    • Enterprise
  • Engineering
    • Automation
    • Software Engineering
    • Project Management
    • DevOps
  • Programming
    • Learning
  • Tools
  • About
aster.cloud aster.cloud
  • /
  • Platforms
    • Public Cloud
    • On-Premise
    • Hybrid Cloud
    • Data
  • Architecture
    • Design
    • Solutions
    • Enterprise
  • Engineering
    • Automation
    • Software Engineering
    • Project Management
    • DevOps
  • Programming
    • Learning
  • Tools
  • About
  • Big Data
  • People

Why You Should Never Use Microsoft Excel To Count Coronavirus Cases

  • root
  • October 15, 2020
  • 4 minute read

Public Health England has admitted that 16,000 confirmed coronavirus cases in the UK were missed from daily figures being reported between September 25 and October 2. The missing figures were subsequently added to the daily totals, but given the importance of these numbers for monitoring the outbreak and making key decisions, the results of the error are far-reaching.

Not only does it lead to underestimating the scale of coronavirus in the UK, but perhaps more important is the subsequent delay in entering the details of positive cases into the NHS Test and Trace system which is used by a team of contact tracers. Although all those who tested positive had been informed of their results, other people in close contact with them and potentially at risk of exposure were not immediately followed up (ideally within 48 hours). This was a serious error. What could have caused it?


Partner with aster.cloud
for your next big idea.
Let us know here.



From our partners:

CITI.IO :: Business. Institutions. Society. Global Political Economy.
CYBERPOGO.COM :: For the Arts, Sciences, and Technology.
DADAHACKS.COM :: Parenting For The Rest Of Us.
ZEDISTA.COM :: Entertainment. Sports. Culture. Escape.
TAKUMAKU.COM :: For The Hearth And Home.
ASTER.CLOUD :: From The Cloud And Beyond.
LIWAIWAI.COM :: Intelligence, Inside and Outside.
GLOBALCLOUDPLATFORMS.COM :: For The World's Computing Needs.
FIREGULAMAN.COM :: For The Fire In The Belly Of The Coder.
ASTERCASTER.COM :: Supra Astra. Beyond The Stars.
BARTDAY.COM :: Prosperity For Everyone.

It emerged later that, that day a “technical glitch” was to blame.

To be more specific, the lab test results were being transferred to Excel templates. The templates hit a limit in the number of rows they could handle and then failed to update with more cases added. The issue was resolved with all new cases added to the totals reported over the weekend by breaking the data down across smaller spreadsheets.

The issue may have been fixed, but people’s confidence in the testing system in place in England will undoubtedly take a knock. It’s also likely that politicians and media will use this as political ammunition to argue the incompetence of government and Public Health England. Is this the right response? What should we take away from this mistake?

Read More  Oracle Chosen As TikTok’s Secure Cloud Provider

 

An avoidable mistake

We should not forget that the government and public health workers are doing an incredibly challenging and demanding job dealing with a pandemic. But this kind of mistake was avoidable. We live in a world of big data, with artificial intelligence and machine learning permeating all aspects of our lives. We have smart factories and smart cities; we have self-driving cars and machines trained to exhibit human intelligence. And yet Public Health England used Microsoft Excel as an intermediary to manage a large volume of sensitive data. And herein lies the problem.

Although Excel is popular and commonly used for analysis, it has several limitations that make it unsuitable for large amounts of data and more sophisticated analyses.

The companies that analysed the swab tests to identify who had the virus submitted their results as comma-separated text files to PHE. These were then ingested into Excel templates to be uploaded to a central system to be made available to the Test and Trace team and government. Although today’s Excel spreadsheets can handle 1,048,576 rows and 16,384 columns, developers at PHE used an older Excel file format (XLS instead of XLSX) resulting in each template being able to store only around 65,000 rows of data (or around 1,400 cases). When the limit was reached, any further cases were left off the template and therefore positive cases of coronavirus were missed in the daily reporting.

The bigger issue is that, in light of the data-driven and technologically advanced age in which we live, that a system based on shipping around Excel templates was even deemed suitable in the first place. Data engineers have for a long time been supporting businesses with managing, transforming and serving up data, and developing methods for building efficient, robust and accurate data pipelines. Data professionals have also developed approaches to information governance, including assessing data quality and developing appropriate security protocols.

Read More  Deliveroo’s Exit From Australia Shows Why Gig Workers Need More Protection 

For this kind of custom application there are plenty of data management technologies that could have been used, ranging from on-site to cloud-based solutions that can scale and provide managed data storage for subsequent reporting and analysis. The Public Health England developers no doubt had some reason to transform the text files into Excel templates, presumably to fit with legacy IT systems. But avoiding Excel together and shipping the data from source (with appropriate cleaning and checks) into the system would have been better and reduced the number of steps in the pipeline.

 

The blame game

Despite the benefits and widespread use of using Excel, it is not always the right tool for the job, especially for a data-driven system with such an important function. You can’t accurately report, model or make decisions on inaccurate or poor quality data.

During this pandemic we are all on a journey of discovery. Rather than point the finger and play the blame game, we need to reflect and learn from our mistakes. From this incident, we need to work on getting the basics right – and that includes robust data management. Perhaps rather concerning are reports that Public Health England is now breaking the lab data into smaller batches to create a larger number of Excel templates. This seems a poor fix and doesn’t really get to the root of the problem – the need for a robust data management infrastructure.

It is also remarkable how quickly technology or the algorithm is blamed (especially by politicians), but herein lies another fundamental issue – accountability and taking responsibility. In the face of a pandemic we need to work together, take responsibility, and handle data appropriately.The Conversation

Read More  God Save The King! – In Defence of King Charles III, The Coronation & The Monarchy

 

Paul Clough, Professor in Search & Analytics, University of Sheffield

This article is republished from The Conversation under a Creative Commons license.


For enquiries, product placements, sponsorships, and collaborations, connect with us at [email protected]. We'd love to hear from you!

Our humans need coffee too! Your support is highly appreciated, thank you!

root

Related Topics
  • Algorithm
  • Algorithms
  • Coronavirus
  • Coronavirus insights
  • Coronavirus UK
  • COVID-19
  • Data Management
  • Health Data
  • Machine Learning
  • Microsoft Excel
  • Public Health England
You May Also Like
View Post
  • Featured
  • People

Conclave: How A New Pope Is Chosen

  • April 25, 2025
View Post
  • People
  • Technology

AI is automating our jobs – but values need to change if we are to be liberated by it

  • April 17, 2025
Cloud
View Post
  • Engineering
  • People
  • Public Cloud

Why We Need Both Cloud Engineers And Cloud Architects

  • March 11, 2024
View Post
  • People
  • Technology
  • Work & Jobs

Usher’s New Look Embarks on AI-Focused Collaboration with IBM to Set Students up for Career Success

  • March 7, 2024
goswifties-taylor-swift-travis-kelce-fathers-daughters-super-bowl-bonding
View Post
  • Featured
  • People

How Taylor Swift Unexpectedly Brought Fathers and Daughters Together Through Football

  • February 11, 2024
Cloud
View Post
  • People
  • Technology

Clean Air Is A Valuable Economic Asset. Here Are 4 Steps To Achieve It

  • January 31, 2024
Earth
View Post
  • People

Why Standards And Controls Are Essential To The Future Of Digital Financial Markets

  • January 31, 2024
Netherlands
View Post
  • People

How We Can Identify Climate-Vulnerable Neighbourhoods And Protect Inhabitants

  • January 26, 2024

Stay Connected!
LATEST
  • 1
    AI security and compliance concerns are driving a private cloud boom
    • June 9, 2025
  • 2
    It’s time to stop debating whether AI is genuinely intelligent and focus on making it work for society
    • June 8, 2025
  • cookies-food-photographer-jennifer-pallian-OfdDiqx8Cz8-unsplash 3
    What is a cookie?
    • June 6, 2025
  • person-working-html-computer 4
    8 benefits of AI as a service
    • June 6, 2025
  • 5
    Where is the cloud headed?
    • June 6, 2025
  • 6
    Cloud breaches are surging, but enterprises aren’t quick enough to react
    • June 6, 2025
  • 7
    Enterprises are keen on cloud repatriation – but not for all workloads
    • June 4, 2025
  • 8
    The Summer Adventures : Hiking and Nature Walks Essentials
    • June 2, 2025
  • 9
    Just make it scale: An Aurora DSQL story
    • May 29, 2025
  • 10
    Reliance on US tech providers is making IT leaders skittish
    • May 28, 2025
about
Hello World!

We are aster.cloud. We’re created by programmers for programmers.

Our site aims to provide guides, programming tips, reviews, and interesting materials for tech people and those who want to learn in general.

We would like to hear from you.

If you have any feedback, enquiries, or sponsorship request, kindly reach out to us at:

[email protected]
Most Popular
  • Understand how Windows Server 2025 PAYG licensing works
    • May 20, 2025
  • By the numbers: How upskilling fills the IT skills gap
    • May 21, 2025
  • Global communication equipment in futuristic server room generated by artificial intelligence 3
    Weigh these 6 enterprise advantages of storage as a service
    • May 28, 2025
  • AI and private cloud: 2 lessons from Dell Tech World 2025
    • May 28, 2025
  • Examine the 4 types of edge computing, with examples
    • May 28, 2025
  • /
  • Technology
  • Tools
  • About
  • Contact Us

Input your search keywords and press Enter.