Procurement / By /

Missing Data Is A Known Unknown

March 16, 2021
Procurement

There is a famous quote about the fragility of knowledge by Donald Rumsfeld, the hawkish US Secretary of Defence during the Iraq war:

“as we know, there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns—the ones we don’t know we don’t know.”

Following the legal judgement against the UK Government for delaying publication of their contract notices, the question of how many contracting records are yet to be published is effectively a known unknown for the UK.

In relative terms, the UK publishes more data on procurement than almost any other country, but our inability to know how much contracting data is missing indicates how far the UK and other countries have to travel if we’re to achieve true transparency.

In an attempt to answer this question we’ve taken the last four years of contracting data for central government in the UK and compared it to the spending data for the same period. If we count how many suppliers have been awarded contracts each year between 2017 and 2020 we find an average of 6,401 suppliers have been listed each year. However, when we look at the spend data, the average number of suppliers being paid by central government entities during that period is over 31,852.

So an 80% difference.

When we reconcile this data to supplier records, something that takes significant amounts of work, including manual and automatic matching, we still end up with a 78% difference.

Does this mean that around 80% of award notices are missing? Not quite. This analysis can only indicate that there’s an awful lot of data missing, but it can’t say precisely how much data is missing.

The sheer scale of the differential indicates that there is a problem here. For instance a search on Contracts Finder, England’s procurement transparency database, for a prominent London Borough returns just two contract award notices. It simply isn’t credible that a council in London has let only two contracts in four years. The only reasonable explanation is that they have failed to publish the data required.

We know that the team at the Cabinet Office has been working hard to improve this data, but the responsibility for compliance lies with the publishers. It is not unreasonable to suggest that the Cabinet Office has better things to do, the publishers knows what they should be doing, they have the tools to do so, why can’t they publish this information?

Rumsfeld was roundly ridiculed for his quote, somewhat unfairly to my mind, trying find new ways to describe our collective ignorance seems like a useful endeavour, despite what else you might be getting up to.

Covid-19 has put a spotlight on all of our governments, their integrity is very much at stake and transparency is a chance to turn known unknowns into known knowns. Governments should think earnestly about the benefit of choosing collective enlightenment over collective ignorance.

If you’d like to discuss our government data or research capabilities, please get in touch.

May 30, 2023

Space To Innovate- New UK Gov Opportunity

The UK Government's, Defence Science and Technology Laboratory (Dstl) Space Systems Programme has announced it is seeking proposals to aid UK Defence...
Read more...
May 25, 2023

EU Commission Move To Electronic Procurement

The European Commissions Directorate Generate for Informatics (DIGIT), has launched a new dynamic purchasing system for software licences and associated services (SIDE...
Read more...
May 22, 2023

Biosecure Procurement Trial

The UK Government yesterday announced new measures for domestic tree seed production and biosecurity procurement requirements. The next round of the Domestic...
Read more...
May 18, 2023

New Industry Body White Paper Review of Covid Procurement

Since early on in the pandemic, there has been significant focus on the procurement of Covid related PPE, due to unprecedented nature...
Read more...
May 11, 2023

EU Council Joint Procurement Under European Peace Facility

The European Union Council has adopted an assistance measure worth €1 billion under the European Peace Facility (EPF) that will further contribute...
Read more...
May 10, 2023

Measuring Sustainable Public Procurement

One of the projects we are most proud of is working with The Open Contracting Partnership to create a Sustainable Public Procurement...
Read more...
May 3, 2023

From Hero to Zero – UK Digital Marketplace Shuttered

Within living memory, the UK’s Crown Commercial Service (CCS) had planned to reshape a large swathe of public procurement in the image...
Read more...
May 2, 2023

We Sell Data, Not Seats.

Why we sell data not seats? We're data analysts. We really understand the data we gather. We've been doing it for 15...
Read more...
May 2, 2023

New Tech Disrupts Traditional Government Procurement

A Silicon Valley startup, GLASS, has created a revolutionary new tech, Glass Commerce, an e-commerce marketplace enabling compliant transactions between government buyers...
Read more...
April 25, 2023

Green Public Procurement: New Report

At Spend Network, we are always on the look out for governments, think tanks, research bodies, and other organisations who are working...
Read more...
April 25, 2023

EU Agrees on Anti-Coercion Instrument

The European Commission and European Parliament’s Committee on International Trade have agreed on an anti-coercion instrument (ACI) which will enable public procurement...
Read more...
April 13, 2023

£256 Million Contract for Cyprus Awarded by UK Government

The UK Ministry of Defence  has awarded a £256 million contract, for work, repairs, servicing, and facilities management for the UK defence...
Read more...
April 5, 2023

Endorsements For Export Agencies

Endorsements are a win-win for export agencies. Exporters face new challenges, both in terms of increased barriers to trade following the pandemic,...
Read more...
April 4, 2023

Looking For Reasons To Export to Government?

If you are looking to grow exports, you should start with the largest and most transparent industry in the world, and here...
Read more...
April 5, 2023

Lockheed Martin Selected As Preferred Bidder.

US defence giant Lockheed Martin has won a contract to deliver Australia’s first sovereign-controlled military satellite communication constellation. Lockheed Martin was recently announced...
Read more...

Newsletter

Compelling research, insights and data directly into your inbox.

Recent media stories

Search

Recent articles