Part 4: Understanding the quality and value of the work being delivered
4.1
In this Part, we set out our findings about how NZTA:
- gathers information to determine how well its consultants and contractors are performing;
- uses that performance information; and
- learns from experience.
Gathering performance information
NZTA gathers a wide range of information relevant to determining how well consultants and contractors are delivering quality and cost-effective maintenance work.
4.2
At a regional level, the information NZTA gathers focuses on those quality and cost-effective requirements within contract deliverables and key performance measures outlined in monthly and accrual reports prepared by NZTA's consultants and contractors. These reports include information about traffic and maintenance operations, asset management, safety incidents, finances, and the status of the contract against the overall programme. Consultant and contractor performance information is gathered and maintained in the PACE system and as part of specific contract management and "lessons learnt" processes.
4.3
At a national level, the information NZTA gathers generally focuses on the wider performance of the network. This includes information about road condition from an annual survey that assesses a range of condition measures and annual RAMM rating surveys that test a sample of the network. It also includes information about vehicle crashes, safety, and road user experience of the network through annual surveys.
Using performance information
NZTA regularly assesses how well its consultants and contractors are delivering against their contract deliverables and key performance measures. Although NZTA has recently done some benchmarking, information about the performance of each area in the network could be better used to inform how well NZTA's service delivery approaches and models are performing.
4.4
At a regional level, NZTA regularly assesses and evaluates information focused on those quality and cost-effective requirements outlined in contract deliverables or key performance measures of its consultants and contractors through a range of different activities and measures. As previously outlined, these include monthly and accrual reporting by consultants and contractors, performance evaluation reviews through the PACE system, and contract management and "lessons learnt" reviews.
4.5
NZTA has recently done some national-level trend analysis of road pavement surfacing and structure condition in each area in the network. The analysis showed that there was a consistent steadying or improving trend in all areas against most condition measures, except for those relating to rutting, which showed signs of deterioration.
4.6
In general, NZTA does not systematically or consistently assess information at a national level about the wider quality and performance of maintenance and renewal work in areas in the network, to understand differences, trends, and the reasons for them. NZTA has recently begun some initial work to compare maintenance and renewal costs between areas with similar traffic volume characteristics.
4.7
We analysed the available NZTA data for the areas we visited to see if there was any identifiable relationship between the service delivery model used and indicators of the value delivered through maintenance and renewal work, such as road condition indicators. Our limited analysis did not identify any obvious relationships between the value delivered and the delivery model used.
4.8
In our view, NZTA should regularly assess information throughout the network, including comparative costs and broader indicators of value for money such as road safety and traffic congestion trends, to inform its understanding about how the delivery approaches and models are performing in each area and region in terms of quality and value for money.
4.9
We consider that this information would be essential to inform any assessment at a national level of the quality and value for money of the maintenance and renewal work that the range of models is delivering (see recommendation 2) and formal reviews toward the end of a contract (see recommendation 4).
Learning from experience
NZTA uses performance evaluations to assess tenderers, and internal value added teams are in place to promote best practice and innovation.
4.10
NZTA's performance evaluations of its consultants and contractors are used in future tender processes for maintenance and renewal work. An objective of NZTA's performance evaluations is to provide an historical database to assist in assessing track records in tender evaluations. In this way, the performance evaluations maintained in the PACE system are specifically used by NZTA's tender evaluation teams to evaluate a tenderer's performance, as part of their non-price attribution evaluation.
4.11
NZTA's internal value added teams (VATs) promote best practice throughout the business. VATs within the HNO group are made up of expert staff in key functional areas. One of the key activities of the VATs is to identify best practice and embed it into the business. There are VATs in the areas of safety, property, asset management, customer service, project development and delivery, Roads of National Significance, and value for money.
4.12
The asset management VAT is specifically responsible for considering a range of performance reviews, and promulgating and promoting best practice in the HNO group. NZTA staff we spoke to as part of the audit indicated that a new Value for Money VAT had recently been established to more systematically identify and promote cost-effective innovations in the HNO group.
4.13
Internal teams focused on promoting best practice and lessons learnt in the HNO group are an important mechanism for ongoing improvement throughout the business. Although elements of good practice occur across the range of different areas and regional offices, the Auckland Motorway Alliance case study (see case study 2) identified a number of important operational innovations that, where relevant, could be promoted and applied throughout the network by these teams.