Lessons learned
Posted by Alec Ray on 19 May, 2010 - 20:12
In Project Management we rarely seem to apply Lessons Learned - Why does our profession continue to make the same mistakes.
Lessons Learned (both good and bad) can possibly be the most powerful Project Management Tool available
Posted in
The concept of Lessons Learned is potentially very powerful to future projects.
I believe that the reason we keep making the same mistakes is because the "Lessons" are only "Learned" if and when they are taken into account on future projects. At the end of a project the capturing of what went well and what didn't only results in "Lessons Recorded".
If you examine the "Lessons Learned" reports from previous projects (assuming you can still find them) you'll undoubtedly see the same familiar items appearing time and again - sure, sometimes they'll have been avoided, but recurrence shows that they are not being learned - merely recorded.
In my experience, we remember to go and write Lessons Learned reports far more often than we go and look at someone else’s. Experience, being the tough teacher it is, means that there is a good chance you remember your own lessons, so there is even a temptation to not write them down – “nobody looks at them anyway do they?”
The big spectacular failures are often publicised and visible so we don’t see so many of those being repeated, but the un-noticed project killers survive to ambush the next project.
Rather than writing reports that are filed away to satisfy our processes, maybe a more proactive approach would help. I’ve worked within organisations where we would sit down in PM forum meetings specifically to share lessons learned from all projects from the whole PM team – more of a “push” rather than a “pull” approach. This heightens awareness of problems, lets you see where the same thing is a happening and find solutions to prevent them from happening again. It also allows you to highlight the good things and not just focus on the problems. Obviously not rocket science, so why don’t we see more of it?
Until we manage to do a bit more learning from our recorded lessons we should expect to see the usual suspects for some time yet - as the saying goes, those who cannot learn from history are doomed to repeat it.
I think I would dispute the assertion that lessons learned are rarely applied. On the contrary in my experience most people perform their day to day work and make decisions based on their own best judgement or the judgement and advice of those directing them. In some, and probably the majority of cases, this judgement will be based on personal experience of similar or comparable circumstances. If that experience doesn’t exist first hand or within the immediate team, it certainly does within the vast array of reference material easily available to anyone with access to the internet. For a project manager to be placed in a situation which is entirely novel, where he cannot, to a certain extent, call upon his own experience or of those he knows or draw upon the documented experiences of others, must be exceptionally rare.
This is the application of lessons learned at its most fundamental. As project managers we all work that way. To say that we rarely apply lessons learned is therefore selling ourselves short, in fact I would even go so far as to say that we routinely apply lessons learned.
That of course is not to say that there is no room for improvement, because there are many examples of the same problems cropping up again and again.
I absolutely agree that the documenting of lessons on most projects is not particularly good. In many cases it simply isn’t done at all, and where it is undertaken (often only because it is mandated by corporate procedures) it is all too often seen as a tick in the box exercise and therefore often carried out by those not best placed to do so and not that thoroughly. However at least it sometimes does get done. What almost never happens is a review of relevant lessons reports by new projects. PRINCE2 does mandate it, but it simply doesn’t happen in a meaningful way.
Really until a much more fundamental focus is placed upon drawing on lessons learned reports this situation won’t really changed. In problem situations very rarely is the PM challenged as to whether a given problem had occurred on a similar project, whether it was foreseeable, and to what extent measures had been taken to avoid its recurrence. Perhaps more focus on holding PMs to account in this way would result in using lessons learned reports more effectively.
I would suggest that at the moment the following statements are often true:
1. No lessons report available
2. No active attempt is made to obtain lessons report
3. When a report is available it does not contain sufficient information or detail, or is not of sufficient quality to be of much benefit, and consequently diminishes the value of such a report making future attempts to refer to lessons learned reports much less likely.
4. Reports often focus on the negative and rarely identify what went well.
There are undoubtedly other factors in play of course. These are some observations from my personal experience:
a) There is often a perception that when a new project is set up – especially if preceding projects have been problematic - that a conscious effort should be made to wipe the slate clean – this usually means an entirely new team (immediate lack of continuity and no first hand experience of what went before) with new opinions about what will work and what won’t. Too much reference to an earlier “unsuccessful” project is generally viewed as “not a good thing”, so even if a lessons learned report does exist it may well be never be looked at. In such situations it is little wonder that the same mistakes are made again, and even worse, mistakes are made on the new project in areas which were successful on the earlier project.
b) Large corporate clients often rigidly mandate a way of working which there is no option but to comply with. Those procedures might have been developed by another part of the business and blanket applied to the entire organisation and might be completely inappropriate for the project in hand. In these circumstances the problems are often absolutely foreseeable but occur anyway because the business often values adherence to their procedures as essential. Even if they didn’t it is often extremely difficult to obtain permission to deviate from those procedures. When problems do occur in these cases everyone always agrees that the problems were caused by the internal procedures but this fundamental issue is never resolved, and it is far too often written off as “well if that’s what head office says....”.
c) There are “personalities” involved and/or an environment which is not conducive to working this way. In large organisations I have worked in the project managers are often very competitive and there is a real desire to improve upon others’ successes and there is often something of an adversarial environment. On the face of it this should be a positive thing with the PM actively attempting to establish what went well and what could have gone better and making tweaks to an existing formula. I think this more usually manifests itself along the lines of another PM adopting a completely different approach in order not to be seen to be adopting the same approach as a colleague (rival?) regardless of how successful his project. More worryingly this can also mean that there is not an active effort made by a successful PM to pass on good points to others, or for that matter to flag up obvious pitfalls. With a former employer of mine the PM’s were scored on a monthly basis against a whole series of performance criteria with scores then published to show you how you were performing against your colleagues - those at the lower end of the scale effectively being told they had better improve with performance related pay, disciplinary action etc being held over our heads. Bluntly not a knowledge sharing environment at all, and one fostered entirely by the organisation who should have been promoting one.
The old techniques just do not work. Everyone claims to write LL reports. No one reads old LL documents. There are better ways of passing on experience but so few use them. It is such a shame.
Completely agree with your comments. My experience in Projects just started, however, when gather team for LL, focused on "things go wrong " mainly, and just few good things... The next project it´s quite challenging.... and need to go back review history since some phases of the project are very similar to the previous one.....I think if we invite a team member of first project to share with the new one "Lessons Learned", if hearing from someone, can give opportunity to ask, share and why not... may be adjust the projects activities...
Gloria
Cranfield University School of Management has been doing some research into what helps projects to succeed or contributes to failure. They have found that the biggest differentiating factor between organisations that generally succeed with their projects and those that don't is "the willingness to publish and distribute lessons learnt".
So its not just enough to close out the project and to create a Lessons Learnt report - the reports have to be made available to pothers in a way that makes them want to read and apply. And, once again, the key capability is communication - organisating the critical information in a way that that makes it appear relevant and easy to undertsand, making the different stakeholder groups are aware that the information is available, ensuring that they know where to find it, and arranging things so that they can quickly turn the information presented into useful actions.
best regards
Paul Rayner, ProgM
First of all, it shouldn't be 'lessons learnt' rather lesson captured
The issue is how to share past learning and having been involved in knowledge management studies I have to say that the only way real learning gets shared is through conversation, the trick from a business point of view is to enable the conversation - just as is happening here!
Even when lessons are captured their future interpretation depends critically on the skill of the writer, we have all read dry history books and some that bring the past dramatically to life. The latter are the powerful influencers.
For the future tools such as social websites, twitter etc. have the greatest potential for enabling conversation and therefore ability to share knowledge/lessons. It is not the capture of lessons that is hard - it is the release that’s tough!
I believe the most effective mean to transfer lesson learnt is via informal routes such as discussions, forums, telephone conversation.
As noted before, most lesson learnt although captured are not being communicated out, and key learnings mostly remain with the individuals involved. Also for major complex projects, what you can actually capture down in a report is only a small percentage. Hence from my experience when a lesson learnt report is thought to be useful to the current project, it is usually followed by a tele-coversation with the author who will put you in touch with other SMEs.
This is also why forum such as these is such a big step forward, as it promotes a community of practice (CoP) who people come together and discuss their own learnings and experience.
Formal channels driven down from top management often results in an ineffective formal process who people only pay lip service to.
My view is companies need to faciliate CoP like this one within their organisation and bring people together and the learning will come through naturally through participation.
A lot of sense said here. Obviously individuals learn fro their "lessons", hopefully, but for organisational learning that is different. Some organisations are good at getting PMs (whatever their job title) togther to talk through lessons learned and share that way, others "publish" key points to their intranet. One of the problems is simply retrieving the information if an organisation is running many projects. I have started asking if developments in text based searches, maybe driven by emerging semantic reasoning, might be helpful here. In the meantime CoPs like this hold tremendous potential - but where will we hide these gems ? Perhaps a Lessons Learned blog rather tha just General Discussions??????
Ian
...... and the whole area is one I am getting PM students to look at
Alec
Lots of good debate here and there are a number of issues and a number of ways to communicate LL. My thoughts are as follows:
1. Don't wait for the end of a project to carry out LL. Projects generally split into a number of phases and will overlap with other projects that are approaching the same phase in their project life cycle. I have mandated in my PM Procedures that LL reviews are required at the end of each formal phase. LL may well get learned quicker for both the project being reviewed and other projects
2. A key issue for me is also that of implementation. Its all very well religously carrying out LL and issuing reports but there needs to be plan behind it to implement those lessons. In many instances the ability to do this is outside of the influence of the project as it may be process related or just too late for that project. The LL process therefore needs to consider who is responsible for taking the LL report forward.
3. For 2. to work the organisational structure needs to be right. Process issues can be referred back to those with functional/governance responsibility and project specific issues can be kept with the PM
4. For LL that are not process related but may benefit future projects published guides or aide memoires may be helpful as prompt to ensure the LL are not missed. I am working on a consolidated risk register and LL log on an internal intranet site to facilitate this accessable by all project teams
5. Finally and as importantly as above you cannot beat a bit of old fashioned good communication. PM get togethers and mentoring to ensure the LL get through.
Hope this helps
Neil
Hi Alec,
I have a question for you regarding how lessons learnt will facilitate an effective development of cost and schedule data for future projects;
Kezner (2010) Describes lessons learned as a post-mortem analysis in information system and emphasises the importance of conducting and becoming proficient in post-mortem analysis meetings. He also states that we need to identify the rights and the wrongs in the project, what recommendations/solutions can be made, how when and to whom the information be disseminated.
Silly Question; Is it possible for the capturing of information to assist in lessons learned post mortem to be overlooked or not entirely identified? Is it also possible that project manager (PM) that is providing the recommendations does not have the experience/expertise to provide appropriate lessons learned? Or am I completely on the wrong track due to existing full proof methods in conducting lessons learnt that I am unaware of?
How do we address efficiency, cost savings and waste elimination if the lessons learned haven’t been identified in the first place?
Reference:
Kezner, H., (2009), Project Management: A Systems Approach to Planning, Scheduling, and Controlling, 10th Edition, John Wiley,
It helps to learn from successes – the ODA have published 100s of ‘lessons learned’ on their web site at http://learninglegacy.london2012.com/
Probably another ‘first and best’; capturing lessons learned during the project so they can be applied immediately.
I wanted to offer up a rough concept and should appreciate that members kick around to build robustness. Consider the following;
Step 1
Any member may offer up to a potential new area of this APM website brief suggestions and thoughts (perhaps not more than 200 words) as lessons captured.
Step 2
There be a number of attributes that have to be entered by the member:
- which APM Special Interest Group such lesson relates to
- which APM Body of Knowledge section such relates to
Step 3
Hopefully the computer would then auto categorize.
Step 4
Later the person whom posted the lesson has options to sort and select lessons by either APM SIG or APM BoK section.
Step 5
At a future date there be an option for upload of file attachements to augment the lesson brief in the form of a template, checklist or case study.
It's just an idea, hope it appeals.
Kind regards
Richard
Knowledge Economic City
Saudi Arabia
www.madinahkec.com
I've written a blog post on the APM website supporting the idea that lessons learned is at the centre of creating a world in which all projects succeed - the vision laid out by Mike Nichols at the APM Awards.
A number of people (Sir Peter Gershon, Tim Banfield of the NAO, Martin Cobb) have suggested that the 'answers' to project success are out already out there. The challenge, then, seems to be the dissemination and adoption of those answers (or the right combination of answers). Do people agree with that? Do projects ever fail or succeed for completely unique reasons?
I'm interested in Richard's idea; I wonder whether there's a future role for the APM website in creating a lessons learned archive that project professionals can access to help deliver their projects.
I agree with Scott,
Anyone who believes it is possible to run all projects successfully clearly has no concept of risk management; and as has been clearly demonstrated in numerous studies, attempting to ‘avoid all risk’ inevitably creates worse outcomes then embracing the management of risk (for more on this see: http://mosaicprojects.wordpress.com/2011/07/30/scope-of-improvement/ ).
Certainly access to historical information in the form of validated lessons learned will be a valuable way of helping people who want to deliver successful projects. The challenge is developing a way to make the information accessible.
I agree with NeilGee's first point that lessons should be identified through the life of a project from start-up to close-out. The danger of waiting until the end of a project to carry out the identifying, capture and analysis of lessons is that most project team members are already focussed on the next project, project momentum is low and most see this as a 'box ticking' exercise. Only a fraction of the lessons that could be valuable to future projects are recorded and passed on. Even if you have an effective method of communicating your lessons throughout your organisation and learning from them, the most important element (identify and capture) has been compromised.
Therefore, I now create a Lessons Log during Project Start-up and record anything that could help to create a best practice for my projects. I encourage its use at the Project Kick-off Meeting and regularly review it along with the RAIDs (Risks, Assumptions, Issues and Dependencies) management documents at Checkpoint Meetings and Gate Reviews. This really makes a difference when populating the Lessons Log and makes the job of writing the lessons into the End-of-Project Report so much easier.
For a classic example of ignoring lessons learned in a simple design exercise see my latest blog at http://mosaicprojects.wordpress.com/
Focusing on one aspect of a design experience caused a classic blunder in another!
Lessons learned represent valuable intellectual property both to PMs and to other areas of the company. This IP can be used to deepen the relationship between supplier and customer. Case in point:
As the former PM Executive for a regional Caribbean IT company we developed our lessons learned in collaboration with our customer, a rapidly expanding telco. There was initial resistance on the customer's part but we persisted. The process "neutralized" blame by focussing on the successes and mistakes of both parties and representatives of both parties signed off on the final LL document. This created more trust between the parties and deepened our relationship. We also shared our lessons learned, both successes and failures, with our sales group as similar opportunities were brought forward. From that perspective, LLs can be a market diffentiator, either as, "we can do the same thing for you based on proven success", or, "We can do it successfully based on what we have learned so don't risk working with our competitor who does not have the benefit or our experiene".
My LL authoring PMs had a unique perspective as each project phase brought them into contact with different units of the company and our suppliers. For example, the last hop in the delivery chain may reveal that an inter-island cargo plane is too small to load a rack through the cargo door. This can be an unforeseen hold-back to a critical delivery schedule. This information goes back to the design, logistics and purchasing group so that futures situations can be avoided.
When lessons learned provide a competitive advantage, I do not see them being shared with third parties via professional association websites, particularly in markets and regions where both project management needs more maturation and businesses are trying to expand their market penetration based on maturing delivery experience.
Mark Hopkins, PMP
Thank you Pat Weaver for the link to the ODC and the importance of learning from success and for the link to your blog. Both full of thought provoking information.
I read somewhere that the culture of the company is more important than the strategy. Does it follow that the culture of a company decide if lessons are learned?
Hi Alec and APM colleagues. A great discussion on PM Lessons Learned and many good points raised.
I am currently close to completing (June 2012) my Masters Project Management (research). I have a strong interest in PM Lessons Learned. Over the last 12 months I have enjoyed learning about the KM World. My final project/thesis will be ‘Exploring factors that impact knowledge management dissemination of project management lessons learned’.
The focus of this study will be to understand why the majority of projects do not disseminate lessons learned to organisations. Knowledge and project management literature suggests that the lessons learned process in practice rarely happens and does not work well and fails to deliver the intended results. The study will address the significant factors that impact the dissemination of project management lessons between the project team and the organisation. The literature review will focus on the areas of: knowledge; knowledge management; knowledge conversion; learning; organisational learning; lessons learned practices; and culture. So far, the literature review suggests there is limited research on how knowledge management, learning and culture impacts project management and project temporary organisations.
The literature review highlights project management literature gaps around people, learning, technology and process. The people factor is the most likely to negatively influence the dissemination of lessons learned in organisations. A conceptual lessons learned model has been derived and based on a swiss cheese model where the variables people, learning, technology and process need to align and be effective to disseminate lessons learned.
By undertaking this study I anticipate a better understanding of the significant project technology, learning, process and people factors will be established. This will assist in the dissemination of the Project Management lessons learned practice being improved.
Reading this APM post, provides good practitioner support of the problem we have. In short I would say the process is broken. Why?
A recent report (2011) from the Australian Victorian Ombudsman, finds that despite all the research, previous Ombudsman and Auditor-General reports, ‘...there are few signs that any lessons have been learnt in the public sector.’ http://www.ombudsman.vic.gov.au/resources/documents/Investigation_into_ICT_enabled_projects_Nov_2011.pdf ‘...A new and more disciplined approach is required if the government is to avoid being faced with continuing cost overruns and failures to deliver.’ The report highlights the difficulties and inconsistencies in ICT procurement with the Victorian Government Solicitor’s Office stating ‘...Government agencies tend to operate independently and there is difficulty in capturing and implementing learnings from ICT projects.’ What we see here is not un-common across the public and private sectors; it is just that the reporting of the public sector problems is open to the general public via government reports.
The Project Management Institute (2008) Project Management Body of Knowledge (PMBoK) guide identifies the importance in collecting and documenting lessons learned, and implementing process improvements. The PMBoK knowledge areas reference the lessons learned process. However in practice it rarely happens and does not work well. Milton (2010) has found that 80 per cent of 74 organisations that attempt lessons learned, 60 per cent are dissatisfied. Williams (2007) found that 62.4 per cent of 522 project practitioner responses had a process for learning lessons and of those only 11.7 per cent followed the process.
The project management PM World Today recently posted an editorial on Lessons Learned but Knowledge Lost (Pells 2011). In response Wideman (2011, p.1) a recognised project management global expert stated: ‘...in spite of all the technology that is available to us today, we have not yet found a presentation format that captures the essence of this wisdom in a way that is relevant to future usage, readily searchable and easy to store. ...we have a serious cultural problem. ...we are probably condemned to continue to throw away the valuable resources.’ This open discussion again highlights the significance of project management, knowledge management and the lessons learned process and the impact that technology, learning, process and people factors have on the problem.
I have established a http://www.pmlessonslearned.info// blog to support and document the research, I would encourage you to have a look and provide feedback.
Regards, Stephen Duffield
@invictaprojects
References available on http://www.pmlessonslearned.info//
Babu Shana's question is a good one!
Making effective use of lesson’s learned is a cultural trait – the alternative and easier option is to ‘sweep problems under the carpet’. However, the trigger to invest in ‘learning from mistakes’ (and successes) has to be an experience that highlights the potential value and ‘experience’ is gained by learning lessons… maybe Stephens blog has something to add?
Babu, On the culture topic my research to date looks at a possible alignment of organisation just culture and lessons learned. I have provided a couple of posts on the pmlessonslearnedblog if you’re interested. Having worked in the aviation industry for most my life as a project manager I do see potential alignment of pm lessons learned and a just culture environment.
On Pats discussion about learning I very much agree about the value part of the learning exercise. I also have a couple of pmlessonslearnedblog posts on learning that may also be of interest.
Would be great to see some feedback, look forward to future contributions.
Stephen
@invictaprojects