CMA improves water market data quality
The Central Market Agency (CMA) is the organisation that administers the market for water and wastewater retail services in Scotland. The CMA is at the hub of the new competitive arrangements, operating the computer systems that run the water market.
Following the opening of the water market in Scotland, the CMA has taken the initiative to monitor and assess data quality levels by way of its annual Market Audit. They are also involved in assisting water market participants with their data cleansing efforts in order to improve the data quality within the Central Systems.
The 2012-2013 Market Audit report highlighted that the required levels of data quality were not being met and further action must be taken to ensure data quality within the Central Systems improved.
In order to improve the data, the CMA selected Bridgeall to develop and deliver a custom data quality platform solution, known as the CUP project (Core UARN Project).
The solution involved combining the data from the Central Systems with data from the Scottish Assessors Association (SAA) dataset. Data for the whole of Scotland from these two sources were provisionally matched by 3rd party matching companies. The proposed matches were returned to the CMA and then uploaded into the CUP system. The CUP system allows the CMA team to accept, reject and create new matches, so that as many supply points as possible have an associated Unique Assessor Reference Number (UARN) from the SAA.
The solution enables the CUP team to view further Central System data and SAA data along with maps, information and photos from Site Surveys allowing them to classify the data into one of a number of categories and subcategories.
Clear understanding of the complex requirements
Key to the success of this system is the calculation of cardinality of each of the proposed matches. This metric shows the number of potential links to other SAA UARNs and Central Systems Supply Point Cores. This is re-calculated whenever a match is made or an action carried out against relevant data. The cardinality metric is essential to letting the CUP team hone in on areas of data that are ambiguous and need further analysis.
“This is a complex solution that requires inputs from multiple sources while keeping a single version of the truth at the core of the solution with the added complexity of calculating the cardinality in real time across the entire dataset” says Graeme McLean, Bridgeall’s Solution Architect for the CUP project.
The successful development and delivery of the CUP project into the deregulated water market has ensured that the CMA have the facility to match data from every postcode area in Scotland.
With the capacity to match over 300,000 Supply Points and add an external reference to the data, the solution developed by Bridgeall has proved invaluable to the CMA in terms of improving accuracy and quality of the data.
The user friendly solution has functionality embedded into it which enables the CUP team to view data in each of the categories together, or as separate datasets, allowing easier in-depth analysis and the ability to validate property matches with a high degree of certainty.
Jeremy Atkinson, Chief Executive of the Central Market Agency says “The Bridgeall solution was an important component in delivering our strategy for achieving a high level of data matching”.
Having developed numerous complex solutions for utilities companies, Bridgeall are in the unique position where they can combine intelligent IT with their in-depth knowledge of the water sector, to develop solutions that drive efficiency and add value.