MCERTS and low-cost sensors: does certification guarantee real world performance?

In a previous blog, we talked about the development of standards for testing sensor performance and the European Standardisation Committee (CEN) Technical Specification (TS 17660-1:2021) for gaseous pollutants and the burgeoning development of a TS for particulate matter (PM10 and PM2.5). In this blog, we consider the impact of certification on sensor selection/purchasing decisions and the links between certification and real-world performance.

The TS for gaseous pollutants published in late 2021 provides a detailed testing and classification regime so that low-cost sensor (LCS ) manufacturers can begin to get their systems tested through national testing houses. (The term ‘low-cost sensor’ or ‘LCS’ in this blog is the term that has been widely adopted in promotion of, and scientific literature to describe sensor devices. Note, this term isn’t necessarily reflective of the true-life costs associated with running a LCS system or network – but that's a topic for another time.) 

The British Standards Institution (BSI) are also in the process of developing a Publicly Available Specification (PAS 4023 Air quality monitors – selection, deployment, and quality control of mountable, static air quality monitors in ambient air – Code of practice). PAS is currently out for consultation and provides guidance on how to select, deploy and quality assure LCS. 

But back to the main topic of this blog; with the relatively recent publication of the TS, no gaseous LCS have gone through formal testing yet and it may take a year or two before we start seeing certified instruments on the market. As a result, as an end user it is recommended that you carry out your own due diligence when procuring these instruments. But what about PM? Well, although TS 17660-2 for PM is still under development, there is an existing alternative – MCERTS (Monitoring Certification Scheme). MCERTS for PM certification process is a testing regime, which is run by the Environment Agency to ensure the quality and performance of particulate monitoring equipment used in the UK is of a particular standard.

The certification process assesses the performance and accuracy of the instrument, to ensure that it meets the requirements under BS EN 16450:2017 - Ambient air - Automated measuring systems for the measurement of the concentration of particulate matter (PM10; PM2.5). As part of the process, the instrument is subjected to various tests to verify its performance and accuracy through field tests. To achieve MCERTS for PM certification, equipment manufacturers must demonstrate that their equipment is designed and manufactured to meet the requirements of the MCERTS scheme, and that it has been tested and validated to ensure that it performs to the required standards.

Once the certification process has been completed, the equipment is granted a MCERTS for PM certificate if it meets the performance requirements, which will either certify the instrument as: 

A) reference equivalent e.g. similar to the automatic methods used within the Automatic Urban and Rural Network (AURN) for compliance monitoring (uncertainty of 25%); or 

B) as an indicative method (uncertainty of 50%). 

Importantly, even when certified, the equipment must still be maintained and calibrated on an ongoing basis to ensure that it continues to perform to the required standards, and that the accuracy of its measurements is not compromised over time. This means that even after industry certification, reference-equivalent and indicative PM instruments are still not simply plug and play. Ongoing quality assurance/quality control (QA/QC) remains vital to maintain confidence in sensor measurements.

MCERTS is well established and there are a number of LCS that have now been certified through this process. So, what should you be thinking about when purchasing an MCERTS PM LCS? 

Well, to produce valid measurements, the instrument must be used in exactly the same way as they were configured for MCERTS testing. For example, if an LCS was tested when it was configured to sample PM continuously, this is how it must be used by the end user to guarantee MCERTS and that the measurement data remain valid. Crucially, if for argument's sake, the configuration is changed to 15-minute spot values, to conserve power to make use of solar panels – this is not how the sensor was tested and the MCERTS status becomes invalid. This would also have implications for the quality of the data produced by the sensor. 

So, as an end-user how can you be sure the configuration of your LCS matches the method of testing?

We would recommend that the end user insist that the manufacturer provide certification evidence prior to purchase clearly stating the configuration of the LCS when certified.

Things to consider when investing in an indicative PM LCS system include:

• Is the firmware/processing algorithm version the same as were used for testing?
• What maintenance requirements are required to maintain MCERTS e.g. do you need to check the sample inlets every six months or change the optical particle counter every two years?
• How long does the MCERTS certification last? PM LCS need to be re-certified every five years, for example.

For LCS that have been certified through MCERTS, this provides confidence that they are performing well enough for indicative monitoring. However, that’s not the end of the story… 

To date, indicative monitoring is the best level of performance one might expect from an LCS. If you want to use your LCS for wider applications, indicative monitoring is not necessarily appropriate or sufficient.

Can sensor performance be improved beyond the level of MCERTS certification for wider application of your LCS?

In our experience, the answer is yes. In addition to the requirements outlined above, further QA/QC can enhance the performance of your existing LCS extending its value beyond basic measurements. Ricardo has extensive experience in implementing such QA/QC regimes. QA/QC for performance enhancement could typically involve a combination of co-locations and remote calibration of the indicative PM LCS. Co-locations provide a direct comparison of the LCS to more accurate reference measurements enabling further correction of the data with the application of correction factors. This process would not only improve the accuracy of the measurements, but also help build evidence on how well the LCS are performing in your network, which would in turn inform how you can use the data. The same benefits can be applied to gaseous LCS tested through TS 17660-1:2021.    

In summary, certification provides confidence to you as an end user that the instruments have the capability to perform as indicative methods. However, there are still key considerations end users should take into account to make sure that their LCS are performing as they should i.e. build up of evidence on ongoing performance through some kind of QA/QC regime, this will in turn make the data more defensible, and potentially improve the accuracy of the measurements. We would recommend doing the following when investing in an LCS:

• Request confirmation of testing from the supplier to align with real world use
• Service system according to maintenance requirements
• For demonstration of performance and improved accuracy, consider further QA/QC

For more information on further QA/QC or if you would like advice on how to approach procuring and operating sensors for a particular project or application, get in touch.