Lessons Learned Don’t Work: Why a Data Centric Approach May Hold the Key, 22 November 2018, Bristol

BAWA was packed for this very popular event, which looked at why lessons learned processes don’t seem to be able to prevent the same mistakes being made time and again, and how an alternative approach based on data analytics may assist with improving project performance and project success.

Our speaker tonight, Martin Paver, CEO of Projecting Success, is passionate about ensuring the project profession is ready for the future.

Martin started with reviewing the current learning from experience approaches. Despite many reports over the years looking at project success factors, including APM reports, the profession is still repeating the same mistakes. We seem to struggle to learn and apply lessons and to be able to avoid the avoidable. There is a huge amount of experience throughout the profession in individuals, but of course no one person can learn everything and so the challenge is how to leverage that collective experience to reduce project failures.

Lessons data is rarely available from the private sector, but is more generally available from public sector and government department projects, but even so often had to be requested through freedom of information requests. From the examples it was clear that lessons data was very limited and not very helpful. Government Departments do not usually have a systematic strategy for lessons learned, but leave it to the discretion of project teams. Even though the National Audit Office share reports, they have no power to insist Department’s change.

Martin has collected some 20000 lessons over the last two years globally and reviewed them. The nature and quality of lessons is poor. The main problem seems to be lessons are simply recorded, they are not analysed to understand the root cause and correlations between lessons and no actions are identified to address them. Lessons should be an integral part of risk management processes, but organisations simply fail to link them together.

Moving on to how data may be used to improve project performance.

Different stakeholders have differing needs from a data set. Assurance, project team, supply chain, future projects, require different perspectives from the same data set.

Approaches to improving project performance need to be tailored to the organisational context. With one off projects it is difficult to build up expertise and to learn, with repeat projects, such as organising large events, it is easier to learn and hone practice to improve performance.

Taking Crossrail as an example, there are two data sets: programme technical information on how to build, run and operate the railway, and programme control information for the build project: the exhaust plume. The challenge is how to collate these disparate data sets to join them up.

Data analytics offers an approach to leveraging this data, which is often siloed. Project Managers want to drill down into a lesson to understand the context and relevance and how it can impact their project.

Martin looked at examples of scheduling, risk and benefits and how data could be used to make recommendations as to the way forward.

Advance data analytics can be seen on a value escalator, moving from low value, easy to do, Descriptive Analytics (what happened – hindsight), through increasing value and difficulty, Diagnostic Analytics (What did it happen – a level of insight), Predictive Analytics (What will happen – higher insight), to optimisation and Prescriptive Analytics (How can we make it happen - foresight), that latter being in the prevention zone.

Machine learning was illustrated using the example of a contract bid process, where patterns were found in the data set to inform the bid strategy for a particular contact opportunity. The key issue is that you have to systematically collect data from previous projects to be able to mine the information you are looking for.

Martin referred to Bent Flyvbjerg’s work on major projects, from which he concludes that the root cause of project issues and risks are primarily internally driven which can lead to optimistic estimates and plans, and strategic misrepresentation. The solution to counter these biases, is to take the outside view, to pool and apply lessons from external projects and use hard objective data.

So why are organisations not doing this? Lack of vision and political will, lack of skills, lack of quality data (often siloed), lack of understanding of the investment case.

In summary, organisations are not leveraging the valuable experience that they already have. Project, programme and portfolio lessons are not differentiated from technical lessons. Lessons learned should not be seen as a tick box exercise, it needs to be seen to add real value to improve project performance. LFE teams cannot easily demonstrate a return on investment, and so are vulnerable during organisational change initiatives. Machine learning and artificial intelligence is coming and will be transformational for the project profession and future jobs.

Martin is trying to develop a community to work on these issues, to be able to demonstrate the art of the possible, the following link will take you to his web site where you can find the slides and how you can get involved.

 

Martin Gosden

SWWE Branch Chairman

 

This presentation can also be viewed on APM slideshare.