Keith Tucker, Phil Lewis, Olga Taylor, Paul Dennis and Annette Jackson discuss ODEP’s recent developments, helping to benchmark the manufacturing of orthopaedic implants
Introduction
Since ODEP was introduced in 2002, the system has been regularly reviewed and upgraded. Three years ago, NICE reviewed its original directive, which resulted in changes to its benchmarks. During a similar time, knees were successfully introduced into the system. ODEP has always made every effort to match the aspirations of everyone involved, while at the same time having patients and their safety at the forefront of our thoughts.
ODEP has become an international brand and the panel are very conscious of the responsibility they carry when making their decisions. We are aware that manufacturers are not able to sell their joint replacement products in many parts of the world without an ODEP rating. The world map below illustrates the number and location of website hits we have received since 2015. In light of the ever-increasing international nature of our benchmarking, we have established an international collaboration group to which representatives from registries and organisations, such as ISAR, NORE, NJR, EPRD, LROI, MedTech Europe and AdvaMed, are invited. The first meeting was held at EFORT 2017 in Vienna and we intend to meet in Barcelona this summer, where we will also have a booth in the exhibition hall.
ODEP and the orthopaedic implant benchmarking system in the Netherlands are closely linked. Members of the Dutch Knee and Hip Societies and the Netherlands Orthopaedic Association (NOV) attend our meetings and are members of ODEP (www.odep.org.uk).
Developments
Following a long consultation process with all of our stakeholders between 2015 and 2017, ODEP developed a new methodology for hips and knees, which utilises real world data within the benchmarking methodology. Careful testing was carried out with four manufacturers as a pilot last autumn and the feedback was extremely positive. The new system subsequently went live in January 2018 and further pilots have been extended to include unicondylar knees and the recently introduced 13-year benchmarks.
ODEP for Shoulders is now established with the first output for Reverse Shoulders published on our website earlier this year. Further work is ongoing with Glenoids and we will be inviting test submissions for Humeral Stems for a trial run in April 2018
We are nearly at the stage of inviting submissions for a dummy run for femoral components for revision hips.
The new methodology explained
The new methodology is outlined below. It is now based on real world data. We reviewed the results for hips and knees in several of the registries that publish such data and, in fact, the observed revision rate, plus a 2 per cent non-inferiority margin, give virtually the same benchmark revision rate as our best guess from 2002, which was very reassuring.
Key changes:
- The acceptable revision rates, at each time point, will depend on the type of implant involved.
- Generally, they will be 2 per cent above the NJR (National Joint Registry for England, Wales and Northern Ireland) and other registry revision rates for that type of device at a particular benchmark.
- The “B” rating criteria have been modified specifically for benchmarking of niche products.
- Significant changes in the cohort size at each benchmark.
- A more standard use of Kaplan Meier with regard to censoring.
- The adoption of the 13-year benchmark.
- The increased availability of data has allowed us to review our thresholds.
There is no change in the other parts of the form. We still need to know the number of centres outside the originating centre(s), revisions and their cause, etc. The section “number past the benchmark” has been dropped. We maintain our strict policy about data sources and where a manufacturer collects their own data it must be validated in some way for it to be acceptable.
Below are the grids for hips and knees:
As always, ODEP adjudicates twice a year for hips, knees and shoulders, with the appeal meeting usually being conducted about three weeks after the main meetings. Submissions must arrive at least two weeks before each meeting so they can be shared with the panel.
The future
Over the next year or so the new EU regulations for the introduction and monitoring of new joint replacements will come into force. At present, they are being developed and we are very much involved in the discussions.
Beyond Compliance is well placed to help monitor pre-CE mark clinical evaluations and ODEP in helping with post-market surveillance.
ODEP relies heavily on the members of the panel freely giving up their time. They are not paid a fee but their travel and other similar expenses are paid. At present, we are hosted by NHS Supply Chain but by the time this article is published, with the re-organisation of procurement in the NHS, that will have changed.
ODEP also recognises the effort put in by members of Northgate Public Services, particularly Richard Armstrong and Robert Scott.
ODEP strives for transparency in all its business and is always welcoming of comments, criticism and new ideas.
