Procurement / By /

New Year, New Tool. Introducing Our New Classifier.

January 31, 2021
Procurement

Our New Solution To Classification

In 2020, we developed an advanced classifier. This tool adds multiple labels to procurement notices based on the Common Procurement Vocabulary (CPV), with an accuracy of 84%. This classifier gives notices five scored, Level 3 CPV codes based on their text and description, that work for the 90 languages in our database. This classifier can process millions of recommendations within hours. As of writing, we are classifying all our new records each day, with full predictions appearing in our API in early in 2021.

This classifier is a series of algorithms that were trained on millions of rows of procurement data in our database. Our model has an F1 score of at least 84%, a precision of 83% and a recall of 88%.

What Does F1, Precision and Frequency Mean?

These terminologies are industry standard terms within the practise of demonstrating algorithmic accuracy.  F1 score is the ‘harmonic’ mean or average of precision and recall. Precision analyses correct and incorrect predictions based on the rows that went into the model. Recall looks at the wider picture: which rows the model got right versus the ones it completely missed.

The Challenges With The Classification

Developing code to categorise spending and tenders has proven technically challenging, due to a number of complicating factors in the underlying published data and the scale of the problem to be addressed.

Multi-Label Data Challenge. All of the documents published in TED and many of the public sources around Europe publish documents with multiple classifications, with no primary classification identified. Many records have more than ten classification references and some buyers deploy a technique of adding diverse requirements to a single tender in different lots. This requires buyers to provide diverse categorisations for the same tender, making it impossible to identify a primary classification for a tender.

So a document with the following codes 72541000, 72510000, 72590000, 30210000 would be classified to 72000000 : IT Services: consulting, software development, Internet and support.

Inaccurate Classification Challenge. We’ve found that a high number of documents have been incorrectly classified. Mostly this is down to confusion around the correct class to apply, but occasionally there is a clear misapplication of codes, usually around services. We often see consulting described as training or research. This problem is exacerbated by the inconsistent nature of CPV codes which are poorly optimised for the description of services. ‘Consultancy’ is mentioned 6 times in the top two layers of the classification and in four different divisions of the vocabulary.

Poor Narrative Data. With some documents having just five words in their combined titles and descriptions, this is barely enough to classify the documents and makes the task of classifying data much harder. This issue is particularly prevalent in Spain where narratives are, on average, half as long as narratives in the UK.

Scale of Classification Challenge. Scale increases the complexity of the analysis we need to do significantly, with thousands of classification options open to the algorithm. The risk of inaccurate classification increases exponentially with the number of classes to be used.

Addressing These Challenges

With our classifier tool, we tackle each of these problems. Our tool is detailed, accurate, and can be deployed at scale. We can use this on private and public sector tenders, contracts and spend. This tool classifies to CPV but we can adapt this tool to other major classifications.

Using The Classification

→ In our api / source data

We can provide data with enhanced classification through our off-the-shelf API or through a custom API. This can be done for either all or a curated part of our data. For instance, if you are a haulage company, we can give you a customised feed of tender, contracts and spend data for logistics, infrastructure, construction.

→ We can classify your data, send and return

We can take your dataset, apply classification labels and send it back to you. Whether you want more details of corporate costs or a B2C analytics firm looking at better segmentation of consumer spend, we can help.

→ Building a custom classification for you

We can work with existing classifications such as CPV or Proclass or even make one for you, tailored to your needs. We can create a new classification that both fully reflects modern buying trends such as consultancy, and improves on existing, outdated taxonomies such as digital and software services.

Examples

Where We Improve The CPV Level

TITLE ORIGINAL CPV PREDICTED CPV
GP IT Systems IT services: consulting, software development, Internet and … Software-related services
Provision of Summative Assessment … Business and management consultancy and related … Business and management consultancy services
Re-Roofing of Admin Building, Police HQ Portishead BS20 8QJ Construction work Roof works and other special trade construction works
MPV Passenger Transport without Passenger Assistance Transport services (excl. Waste transport) Taxi services

 

Adding CPVs Where None Existed

TITLE PREDICTED CPV
New build mixed-use development of 17 apartments and business hub Building construction work
PHE – ICT – Application Service Support and Maintenance for contract tracing solution Software-related services
Citizen & Consumer Protection Officer x4 Supply services of personnel inc. temporary staff
Provision of Youth and Young Carers Services Social work services
Evaluation of the Small Business Leadership Programme (SBLP) Business and management consultancy services

Different CPV For Better Accuracy

TITLE ORIGINAL CPV PREDICTED CPV
The Provision and Maintenance of NHS Health Check Software Health and social work services 85000000 Software-related services 72260000
C-19 Corporate Finance and Market Intelligence Support… Business and management consultancy … 79400000 Financial consultancy… 66170000
CA7688 – Roofing Services Tender Building services 71315000 Roof works and other works 45260000
Housing Solutions Officer x11 Housing services 70333000 Supply services of personnel inc temporary staff 79620000

Multilingual Example

TITLE ORIGINAL CPV PREDICTED CPV
Puhastusteenused Cleaning services Cleaning services
Services d’assurance Insurance services Insurance services
Villamos energia beszerzése Electricity Electricity
Furnizare alimente Chickens Meat
Rahmenvereinbarung Lieferung und Montage von Schulmöbeln School furniture School furniture
Servicios de reparación y mantenimiento … Repairs & Maintenance  Repairs and Maintenance

if you’d like:

  • classified procurement data;
  • to have your own data classified;
  • or to build your own classifier.
June 6, 2023

Net Zero Needs Further Action, Says NAO

A new National Audit Office (NAO) report highlights that further action is needed by the UK Government to strengthen governance and delivery...
Read more...
June 1, 2023

Critical Audit on Australia’s Frigate Procurement

The Australian National Audit Office (ANAO) has recently released its findings into a review of the Department of Defence (Defence) 2018 procurement...
Read more...
May 30, 2023

Space To Innovate- New UK Gov Opportunity

The UK Government's, Defence Science and Technology Laboratory (Dstl) Space Systems Programme has announced it is seeking proposals to aid UK Defence...
Read more...
May 25, 2023

EU Commission Move To Electronic Procurement

The European Commissions Directorate Generate for Informatics (DIGIT), has launched a new dynamic purchasing system for software licences and associated services (SIDE...
Read more...
May 22, 2023

Biosecure Procurement Trial

The UK Government yesterday announced new measures for domestic tree seed production and biosecurity procurement requirements. The next round of the Domestic...
Read more...
May 18, 2023

New Industry Body White Paper Review of Covid Procurement

Since early on in the pandemic, there has been significant focus on the procurement of Covid related PPE, due to unprecedented nature...
Read more...
May 11, 2023

EU Council Joint Procurement Under European Peace Facility

The European Union Council has adopted an assistance measure worth €1 billion under the European Peace Facility (EPF) that will further contribute...
Read more...
May 10, 2023

Measuring Sustainable Public Procurement

One of the projects we are most proud of is working with The Open Contracting Partnership to create a Sustainable Public Procurement...
Read more...
May 3, 2023

From Hero to Zero – UK Digital Marketplace Shuttered

Within living memory, the UK’s Crown Commercial Service (CCS) had planned to reshape a large swathe of public procurement in the image...
Read more...
May 2, 2023

We Sell Data, Not Seats.

Why we sell data not seats? We're data analysts. We really understand the data we gather. We've been doing it for 15...
Read more...
May 2, 2023

New Tech Disrupts Traditional Government Procurement

A Silicon Valley startup, GLASS, has created a revolutionary new tech, Glass Commerce, an e-commerce marketplace enabling compliant transactions between government buyers...
Read more...
April 25, 2023

Green Public Procurement: New Report

At Spend Network, we are always on the look out for governments, think tanks, research bodies, and other organisations who are working...
Read more...
April 25, 2023

EU Agrees on Anti-Coercion Instrument

The European Commission and European Parliament’s Committee on International Trade have agreed on an anti-coercion instrument (ACI) which will enable public procurement...
Read more...
April 13, 2023

£256 Million Contract for Cyprus Awarded by UK Government

The UK Ministry of Defence  has awarded a £256 million contract, for work, repairs, servicing, and facilities management for the UK defence...
Read more...
April 5, 2023

Endorsements For Export Agencies

Endorsements are a win-win for export agencies. Exporters face new challenges, both in terms of increased barriers to trade following the pandemic,...
Read more...

Newsletter

Compelling research, insights and data directly into your inbox.

Recent media stories

Search

Recent articles