Test and Trace Data upload error highlights complexity of Large Volume Data Management

20-11-2020 | 4 min read |

The UK’s Test and Trace app for monitoring local cases of Covid-19 continues to be dogged with problems and earlier this autumn, another issue surfaced. It highlights the many complexities of handling large data sets.

As reported by the BBCthousands of UK people infected with Covid were missed off a routine report, simply because a large data file containing over 80,000 records was loaded into Excel. What’s wrong with that? It sounds logical enough, but actually, there were quite a few issues.

COVID19 mobile alert | TJC Group
London UK October 14 2020, Close Up Of The UK NHS Track and Trace Mobile Phone App To Alert People To COVID-19

Firstly, the version of Excel they used was out of date. It didn’t support the volume of data to be uploaded and the person responsible for completing the transaction evidently lacked an understanding of what tools were appropriate. That’s why tens of thousands of records were missed and these people were never notified about their need to isolate. Secondly, there wasn’t a quality check in place to ensure that the data did all get copied across accurately. No one actually realised there was a problem until much later, when the mistake was detected.

This example illustrates how a complex task like large volume data management is frequently regarded as straightforward for a layman when actually there are many pitfalls. While this particular mistake impacted Coronavirus tracking, it shows how easily other organisations can make similar errors that could have an adverse impact on audit or tax compliance.

Whether you are dealing with health screening or financial transactions, any data being handed over to a regulator needs to be professionally managed following a set process using appropriate software. 

Often, as was the case here, errors aren’t discovered until it’s too late and in financial circles, this can lead to costly penalties or processes having to be rethought at short notice with a knock-on impact to business efficiency.

Free Demo Banner

When dealing with financial regulators, HMRC being a good example, the onus is always on the taxpayer – an entity or an individual – to take all necessary steps to ensure they adhere to their legal obligations. There is no excuse for ignorance and being unaware of the rules. It’s the personal responsibility of the individual to keep themselves up to date with the requirements of their profession. When operating across multiple jurisdictions and complex data reporting requirements, working with a specialist in data management and archiving helps to prevent these data problems from occurring, especially when complex systems like SAP are involved.

Preventing data errors in Tax Management

Avoiding data errors is important because once regulators detect an error, your organisation will become an easy target – they will continue to be targeted for future mistakes.

A tax lawyer will know exactly how best to manage your company’s legal obligations. Your business users will know the local impact of new laws. A data specialist like TJC knows how to handle large volumes of data, spot errors and anomalies and ensure your compliance reporting is 100% accurate.

When it comes to managing large data volumes for financial regulators, TJC has been working with the French postal service, to extract billions of data records required by the French Ministry of Economy and Finance for tax compliance purposes.   

Using experts to optimise data management processes from the outset will ensure that all your audit and B2G (Business to Government) processes are fit for use across multiple countries.

For instance, audit requirements in the UK differ from France, where data needs to be handled in a specific way. In France, a seemingly simple tax declaration such as the Fichier des Écritures Comptables requires 18 fields of financial data to be provided, but complex rules and large data volumes can mean doing this is anything but simple.

When an organisation is using  a SAP system, additional special cases exist relating to the original system set up and users cannot be expected to appreciate all the finer details. Perhaps the user is actually a US company with a subsidiary in France – they might unwittingly follow US conventions when they submit their data to the French authorities and not appreciate the local knowledge required. Or maybe the dataset itself is wrong.

This illustrates how easily mistakes can crop into an extraction process for audit and tax reporting.

Combine this with the additional issue of a user’s IT systems being capable of dealing with very large data volumes and the problem soon magnifies.

Key challenges in tax digitalization era | TJC Group
Image source: TJC Group. 

Many organisations encounter problems because having initially established good audit processes, they later find out that their internal data management processes lacked enough rigor. Before undertaking a large-scale data extraction, audit your own capabilities by asking the following questions: 

  • Do your data management tools meet the latest fiscal requirements?
  • Are you able to extract the data you need from multiple sources, including outside SAP?
  • Are you likely to extract too much data and waste time adding filtering to interrogate it?
  • Do you have a detailed understanding of different, country-specific financial compliance laws, e.g., regional variations in VAT?
  • Will you be managing your data manually or do you have the means to automate?
  • Have you inadvertently created security issues with data access and storage?
  • Do you have an active process for ILM (Information Lifecycle Management) and GDPR compliance?

Ultimately, it comes down to efficiency and getting access to the right information easily, with minimum effort. By outsourcing large data management support and analysis to specialists allows your subject specialists to remain fully focused on their own areas of expertise.

How can we help?

TJC’s dedicated Business 2 Government team can help prepare your data for regulators by optimising the entire data management process from A to Z. We can also advise on how to automate data extraction and validation in the future, to save your business users, auditors and lawyers.

We can help you with targeted data extraction, reducing the volume of data to be interrogated and making the auditing process more efficient. We can advise you on the best software to employ – whether standard SAP is suitable for your requirements or if you will need a 3rd party tool or custom development. Get in touch with TJC if you want to know more.