Missing Data Is A Known Unknown

March 16, 2021
Procurement

There is a famous quote about the fragility of knowledge by Donald Rumsfeld, the hawkish US Secretary of Defence during the Iraq war:

“as we know, there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns—the ones we don’t know we don’t know.”

Following the legal judgement against the UK Government for delaying publication of their contract notices, the question of how many contracting records are yet to be published is effectively a known unknown for the UK.

In relative terms, the UK publishes more data on procurement than almost any other country, but our inability to know how much contracting data is missing indicates how far the UK and other countries have to travel if we’re to achieve true transparency.

In an attempt to answer this question we’ve taken the last four years of contracting data for central government in the UK and compared it to the spending data for the same period. If we count how many suppliers have been awarded contracts each year between 2017 and 2020 we find an average of 6,401 suppliers have been listed each year. However, when we look at the spend data, the average number of suppliers being paid by central government entities during that period is over 31,852.

So an 80% difference.

When we reconcile this data to supplier records, something that takes significant amounts of work, including manual and automatic matching, we still end up with a 78% difference.

Does this mean that around 80% of award notices are missing? Not quite. This analysis can only indicate that there’s an awful lot of data missing, but it can’t say precisely how much data is missing.

The sheer scale of the differential indicates that there is a problem here. For instance a search on Contracts Finder, England’s procurement transparency database, for a prominent London Borough returns just two contract award notices. It simply isn’t credible that a council in London has let only two contracts in four years. The only reasonable explanation is that they have failed to publish the data required.

We know that the team at the Cabinet Office has been working hard to improve this data, but the responsibility for compliance lies with the publishers. It is not unreasonable to suggest that the Cabinet Office has better things to do, the publishers knows what they should be doing, they have the tools to do so, why can’t they publish this information?

Rumsfeld was roundly ridiculed for his quote, somewhat unfairly to my mind, trying find new ways to describe our collective ignorance seems like a useful endeavour, despite what else you might be getting up to.

Covid-19 has put a spotlight on all of our governments, their integrity is very much at stake and transparency is a chance to turn known unknowns into known knowns. Governments should think earnestly about the benefit of choosing collective enlightenment over collective ignorance.

If you’d like to discuss our government data or research capabilities, please get in touch.

September 8, 2022

UK Risks Its Place On Anti-Corruption Body

The UK has been placed ‘under review’ by the 77-country-strong Open Government Partnership (OGP) due to its failure to meet mandatory criteria...
September 8, 2022

Thurrock Exposes Transparency Blind Spot

An investigation by The Bureau of Investigative Journalism (TBIJ) into investments by Thurrock Borough Council has led to the resignation of the...
September 6, 2022

New Zealand Government Reviewing Procurement System.

It’s always encouraging when we see governments around the world looking to improve their procurement transparency and efficiency. The New Zealand Government...
September 1, 2022

Collecting Data For Sustainable Procurement In Construction

Over the last few weeks we have been looking at setting a sustainable procurement framework in the construction industry, and what kind...
August 30, 2022

Selecting Data For Sustainable Procurement In Construction

It is estimated that around 40-50% of natural resources are transformed into construction material, and that as much as 30% of all...
August 25, 2022

Setting A Sustainable Procurement Framework For Construction

When procuring construction projects, it can be useful to underpin sustainability criteria on existing policy and regulation. When assessing the enabling framework,...
August 18, 2022

Big Net Zero Contract Win For Small Cornish Business

A small Cornish company has purportedly won a £70bn contract to help deliver the country's transition to Net Zero. The Penzance based...
August 16, 2022

Supporting Sustainable Procurement In ICT

One of the key challenges of sustainably procuring ICT lies in the lack of transparency in supply chains. To overcome this challenge,...
August 11, 2022

Why Is Sustainable Procurement Important For The ICT Sector?

The extraction of raw materials, manufacturing, transportation, use, and disposal of ICT products is associated with a number of environmental, social, and...
August 10, 2022

Shifts Towards Sustainable Sourcing

A while ago,  we shared an article on findings by the Boston Consulting Group and the World Economic Forum that showed procurement is responsible...
August 9, 2022

Supporting Sustainable Procurement In The Construction Industry

Construction projects are usually long and complex, involving the participation of different stakeholders throughout the different project stages. There are certain factors...
August 4, 2022

Why Is Open SPP Important In The Construction Sector?

The construction industry is estimated to account for 6% of global GDP, with Africa's construction market valued at around USD 5.4 billion...
August 2, 2022

US Advances Equity And Economic Opportunity Through Procurement.

With the implementation of the Bipartisan Infrastructure Law, the Biden-Harris Administration set an objective to ensure federal investments built wealth and opportunity...
July 28, 2022

5 Reasons To Export To Government

If you want to grow exports, you should start with the largest and most transparent industry in the world, and here are...
July 26, 2022

Sustainable Public Procurement and Economic Development

What is it? For many industries contracting has become a global affair. Governments are rightly keen to gain economic advantages from a...

Newsletter

Compelling research, insights and data directly into your inbox.

Recent media stories

The Times
May 30, 2022
CIPFA
August 3, 2021

Search

Scroll to Top