Geraldine Barker and Bridget Jackson, NAO: Myth Busting
APM Project Management Conference 2015, sponsored by 20|20 Business Insight.
In their presentation to the APM Conference Geraldine and Bridget, from the National Audit Office sought to dispel two myths namely; that NAO only reports when something has gone wrong and lessons aren’t learned.
They talked about the experience of the Department of Transport being a sponsor of rail infrastructure programmes and how those lessons might also apply to other projects and programmes.
In the past 15 years the department had sponsored 5 major rail programmes which are all large in scale and complex with construction expected to take around 10 year and costs ranging from £3.6bn to £21.4bn [HS2 – phase 1]. 355km of new lines have been planned or built since 1998 [High Speed 1, Crossrail and HS2] with 1255km transformed or modernised [West Coast Mainline Modernisation and Thameslink programmes]
The key messages were that the department has learnt a lot of lessons in its delivery of rail programmes. Governance arrangements for example on the West Coast Mainline Modernisation Railtrack had had held the sponsor and programme delivery roles. These were now separated on other programmes.
However, some lessons seem to be quite intractable; difficult to learn or to implement. These include; some aspects of business case development, getting the right skills in place and evaluating programmes at the end.
Geraldine argued that the main lesson for her was that ‘it is as important for programme sponsors to learn from their past experiences as it is for programme managers. People need to think about their own experiences and to draw on those from others [reflective learning] and not to try and go and reinvent the wheel.
Four categories of issue on rail programmes
In their presentation they discussed the issues under four category headings (view slide 8 for diagram). This is a simplified version of the one in the NAO Executive report Lessons from major rail infrastructure programmes.
The key ones for me, were those based around benefits realisation and its close cousin, the business case.
Considering the benefits big picture
NAO argue that achieving the full range of potential benefits from investment in rail infrastructure requires a wider focus than successfully delivering the construction work.
The Department has recognised the wider benefits its rail programmes can deliver but achieving regeneration is not always straight forward. For many programmes however, such wider benefits are critical for achieving value for money.
For example, on High Speed 1 we estimated that the value of benefits to passengers are likely to be around £7 billion while the net cost to taxpayers will be around £10 billion.
Substantial other benefits need to be demonstrated, such as from an evaluation of development around stations, for the original benefit–cost ratio of 1.5:1 to be achieved. 5.3 Expected developments around the new High Speed 1 station at Ebbsfleet have not yet occurred several years after the line opened. The Committee of Public Accounts attributed the slow progress in regeneration at Ebbsfleet to a lack of leadership from central government.
HM government has recently sought to address this by announcing, in March 2014, plans to create a garden city of 15,000 new homes at Ebbsfleet. These plans will be supported by public sector investment, and coordinated by a dedicated development corporation.
In contrast, the department’s delivery body, London and Continental Railways, have worked for many years with private sector developers to develop land around the High Speed 1 stations at St Pancras and Stratford (see slide 12).
The benefits of regeneration and growth
Bringing about regeneration and economic growth does not sit in the Department for Transport’s core area of expertise but it is now focusing on how these will be achieved while its major rail programmes are still being planned and built.
For example HS2 has established a ‘Growth Taskforce’ to advise on how to achieve maximum potential for economic growth from the High Speed 2 programme, during its current planning stage.
In response to the Taskforce’s recommendations, the Department has committed to fund the development of local growth strategies and develop a business case for setting up a national regeneration company to help local areas develop and implement their strategies.
They further advised that benefits and outcomes need to be evaluated in order to inform future programmes. This, in turn, could also help the Department negotiate larger contributions for future programmes from those who benefit most from proposed investment. In my view this make perfect sense – i.e. the simple principle that those who benefit most pay most!
Getting the sums right or lose public support
So far as the business case is concerned, it is absolutely vital that any economic analysis is sense-checked to ensure that the results are realistic.
In the phase 1 analysis for High Speed 2 an initial benefit–cost ratio of 2.4:1 was reported. This was restated to 2.6:1 in the 2013 NAO report. After correcting these errors and changes to assumptions, the current ratio, of 1.4:1 - which is more in line with equivalent programmes.
The conclusion of the NAO is that the necessary reduction in the ratio only served to undermine public confidence in the decisions it made on the basis of the higher ratio. Confidence can also be reduced if key assumptions are perceived or challenged as unrealistic. The effect of loss of public confidence through poor stakeholder engagement is to delay projects and / or derail them.
Coming soon - NAO on over-optimism at #apmbmsummit
Chair of the APM Programme Management SIG