22nd April 2024
Why the right time to improve data quality is right now
By James Holliss, Data & Analytics Lead, Civica
Civica’s recent Lunch & Learn focused on assessing and maximising data quality to unlock effective decision making. I explored the benefits of having good data quality, as well as the steps you can take to improve it.
But, for some organisations, data quality never comes up in conversation. They might never realise there’s an issue until it’s too late. In this article, I’m exploring those early questions every organisation should ask themselves, whether they know they have a data quality problem or they’re blissfully unaware of how their data is collected, stored and shared.
Big or small, public or private… there's no logic to who gets this right and who doesn’t. It all comes down to the culture of the organisation and the desire to improve.
One factor that’s driving a lot of organisations to work on data quality as a matter of urgency is the dramatic rise of AI use. If your data is being used to train AI models and then shared across the sector, you need to be confident that the right information is going in. Data quality is quickly becoming a bigger topic of conversation.
Of course, thanks to the National Data Strategy, government data should be improving across the board. With better data quality comes better decision making and an improved experience for staff and citizens.
When should I begin improving data quality?
It's never too early to think about data quality.
Whether you’re an early-stage startup setting up operations or a long-established organisation with a complicated legacy, good quality data will always benefit you.
The best way to begin the process is to start internal exploration (more on this later).
You don’t need to do lots of research or become an expert in data science. If you uncover something that needs to be fixed, or you run out of time, or reach the limit of your ability, the next stage is to bring on external help.
The external partner will begin with an assessment, picking up where you left off with your own internal review. The results will help you understand the full extent of the problems so you can put plans in place to resolve them.
Who should lead the project?
A lot depends on the company structure. But usually, a good person to champion the efforts is whoever found that there is a data quality issue. A data quality champion can coordinate the efforts, but they’ll need input from a lot of different people to understand who owns which data.
You don’t need to be a data expert to begin improving data quality. There are some relatively easy first steps you can take without committing to a multi-billion-pound system and a multi-year transformation programme.
How do I get started?
How do I get started? The first question you need to ask yourself: what data do we have as an organisation?
That's probably not going to be a question that one person can answer. You might have an HR system, a finance system, a reporting system and so on. At a very high level, you’ll need a catalogue of those systems and what kind of data they hold. If you can get information such as how often it's updated or where it gets its data from, even better.
As you have those conversations with various teams, you’ll come across different examples of problems. Keep a record of them! If the finance team are always having to manually fix a certain report that’s always missing a specific piece of data, that’s important to know. Sometimes these problems will be easy fixes that make a significant difference to the way people work. Early wins will give you lots of goodwill to keep up the momentum throughout the project.
Once you’ve gathered the initial information, you can start to plan the work. Some datasets might be OK, while some might be in a bad state and need a more urgent fix. That basic overview will enable a data partner like Civica to come in and get started straight away.
How could I benefit from external support?
The way you work with a data partner depends entirely on the level of expertise you have internally. With some clients, Civica supports with filling in the gaps, providing an external holistic view. And for other clients, Civica runs the whole project.
Often, organisations have found an issue and need support to explore it in more depth. External input can help build a case for support around it and help to get buy in from the decision makers.
What sort of problems might I come across?
When you get a full data quality dashboard, it shows a score for each data point.
A basic example of a data point might be the number of people that you employ. If you employ tens of thousands of people, that number can be out by hundreds of people sometimes. The organisation might be relying on that number for decisions on resourcing, investment, office sizing – but now you realise that the number can’t be relied on.
How big a commitment will this be?
How you approach the problem will depend on the scale of your organisation. Large organisations (say over 1000 employees) with complex interconnected systems often face the biggest challenges. Systems could be legacies from years ago. The data might be inherited from different department mergers. Employees might be spread across different countries with different processes.
In these cases, organisations might decide to tackle one department or business area at a time to make the project more manageable. Or they might go all in and overhaul everything at once. The approach depends on the scale of the problem and the internal appetite for resolving it quickly. A partner like Civica can help map out various approaches to help the decision makers choose what works for their organisation.
For smaller organisations, data quality problems are often quicker to fix and sometimes just a nudge in the right direction is all it takes.
Why aren’t more people talking about data quality?
If you’ve seen the Lunch & Learn, you’ll know that good data quality can be an enormous benefit to your organisation. With so many reasons to improve the quality of your data, it seems like an obvious next step in the data maturity journey. So why aren’t more organisations working to improve data quality?
I believe it comes down to three reasons:
1. What problem?
Firstly, people often don’t realise there’s a problem. They’re used to certain ways of working. They never stop to think that having to fix a data issue before sending something on shouldn’t be something they need to do. Manual admin tasks are just part of their working life and they’re unaware that there might be a better way of doing things.
2. Too big a problem
Secondly, once people realise there’s a problem, it might feel too difficult to fix. Many organisations lack the digital skills or maturity to know how to tackle a major data quality issue. People may be working with inherited legacy systems that don’t work well with the organisation’s other systems and the whole problem just seems impossible to untangle.
3. Not my problem
And thirdly, even with a clear way to resolve the data quality issues, the right people need convincing before a project goes ahead. The budget holders and the system managers need to see that there’s an issue and need to understand the potential benefits the organisation can realise if these are resolved.
No matter how many barriers you face in your organisation, getting started doesn’t have to be the daunting task it seems at first. Get in touch to learn more about how we’re helping to deliver the public sector services of the future.
Enjoying the Lunch & Learn series? Take a look at our schedule to discover what’s coming up next.