Critical lessons from Boeing 737 MAX

Configuration management failure. A complex structure that attempted to keep different parts of the organisation happy rather than focusing on how best to build the aircraft. Failure to form a single project team across the multiple design centres. An overly aggressive schedule… such that key issues were ignored early in the project life cycle. Failure to address issues when they were first identified, resulting in snowballing costs… once the problems were finally faced up to.”

Given the high profile of the Boeing 737 MAX case – which includes two commercial flights that have ended in crashes (a Lion Air flight in October last year and an Ethiopian Airlines flight in March, in which a combined total of 346 people died) – you might think that this is a list of project failures from the ongoing investigation into those incidents. But it’s not. It’s from the Catalogue of Catastrophe page for the Airbus A380 project, which ran from 2000 to 2007.

Most major aircraft projects – from earlier iterations of the 737 to the A380 and Boeing’s own 787 Dreamliner – leave a trail of ‘lessons learned’ papers detailing the challenges faced by individual project teams, programme managers and the executive leadership. But the 737 MAX project’s tragic outcome is a reminder that there can never be too many lessons to learn on safety.

The Boeing 737 timeline

The Boeing 737 dates back to 1964 and has arguably been the most successful aircraft design ever. It has been refreshed several times – the 737 Classic entered service in 1984, and the 737 Next Generation was launched in 1993.

In 2006, after finishing the 787 Dreamliner (a project fraught with delays and cost overruns), Boeing considered a brand-new design for its narrow-body workhorse. But the decision was delayed. Then, in 2010, Airbus launched the A320neo with new fuel-efficient engines. By 2011, it had broken order records as airlines jumped at the opportunity for lower operating costs.

To compete, Boeing decided to stay with the existing 737 design. Boeing’s chief financial officer told investors that tweaking the airframe to carry larger CFM engines would equate to 10 to 15 per cent of the cost of designing a new aircraft. And it had to deliver fast or risk losing orders to Airbus. The first 737 MAX rolled onto the tarmac in December 2015 – and in 2017 it was approved for commercial flight. Then, in October 2018, a 737 MAX operated by Lion Air crashed on take-off. In March this year, a 737 MAX in the Ethiopian Airlines fleet went down in similar circumstances – and the model was promptly grounded.

The prime suspect is a new automated Manoeuvring Characteristics Augmentation System (MCAS), fitted to account for changes to the aerodynamic characteristics of the airframe as a result of the new engine configuration. The software seems to have forced both aircraft into a nosedive due to erroneous information from angle-of-attack sensors.

So what are the lessons of the 737 MAX programme for project managers? Four spring to mind – although it is important to point out that the project managers, software experts and assurance professionals that Project spoke to are referring to general project management principles rather than commenting directly on the 737 MAX case, which remains under investigation.

Define and defend project safety parameters

Much of the coverage of the 737 MAX has centred on the MCAS software. Boeing’s own fix for the aircraft is a software patch to solve the override. But questions remain about whether time and budget were prioritised to too great a degree right at the project’s outset.

Colin Hammond runs ScopeMaster, a business specialising in tools to improve IT projects, and he has also been a project and portfolio manager at organisations such as Royal Mail and Whitbread.

He thinks one project management problem that is often overlooked is the question of specifications. Getting those right is crucial to quality – and safety. “The requirements in projects are often misunderstood. You can end up with a system that performs as designed, but if the thinking behind the specifications was flawed – which is a human problem, not a coding one – you leave gaps. That can have tragic consequences.”

On the 737 MAX, it appears that the original design required two sensors to trigger a change in the aircraft’s behaviour – G-force and angle of attack. At some point, the design was tweaked to rely on one sensor detecting the plane’s angle. Without the back-up, in some circumstances the aircraft would overcompensate for a problem that didn’t exist. Sticking to original specifications or setting them in stone might have helped.

“It’s true that, in general, executives care most about the ‘time’ part of the iron triangle,” says Hammond. “But on many software projects, you waste a lot if you skimp on quality. Fixing bugs can be 40 per cent of your time and budget. If you focus on quality at the start – making smarter choices about scope and specification, for example – you’ll end up spending less time and money, not more.”

What’s more, safety implications must be first and foremost in the project manager’s mind. “The longstop on any process is the project manager,” says Albert Lester, a retired engineer and seasoned project manager at companies such as Tarmac and Foster Wheeler. He has also been involved with several standards bodies. “[Project managers] might not be as technical as the individual design engineers, but they absolutely must be able to ask ‘Is it safe?’ and feel confident in the answer.

“Safety really hasn’t been given the prominence it needs in the project management literature,” he adds. “The only place I can recall that made it explicit was British Rail – there was an ‘iron rhombus’ where the extra point was safety.”

But for many project managers, safety is implicit in the ‘quality’ corner of the iron triangle. “The product absolutely needs to be good enough,” says Hammond. “But doing lots of testing doesn’t actually add up to proper quality assurance. For the project to deliver on ‘quality’, you need defect avoidance, as well as defect prevention.”

Have a single vision for complex projects

The 737 MAX mission was simple: retool for a bigger engine as fast as possible. But moving the engine forward and up on the wing affected the aerodynamics. That meant tweaking the surface controls, which required changes to the flight software. And although these changes were designed to be minimal, that in turn affected the required pilot skills.

This complexity of dependencies is always a safety risk in projects.

“Software developers love to be challenged,” says Hammond. “But complexity also creates dependencies that the project team can have a hard time monitoring. In general, we’d recommend simplifying and breaking elements down. Great things happen when you have smaller teams of highly collaborative people working together on a well-defined mission.”

When new issues emerge, staying on top of safety becomes more challenging too. Some commentators have wondered whether deviations from accepted agile project management methodology contributed to Boeing’s failures on the 737 MAX.

Pamela Stacey is now head of programme and corporate assurance for HS2. In a previous assurance and risk role at a major pharmaceutical company, she saw how important it is to evolve safety approaches over the life of a project. “In that environment, patient safety is paramount – from drug discovery to testing and managing the product’s life cycle,” she says. “You need to be able to move fast enough to ensure you’re still focused on the correct risks and not missing risks that emerge over time.”

In the case of the 737 MAX, problematic dependencies seem to have been fixed with ‘kludges’ – engineering workarounds. According to an investigation by Vox, “Boeing’s team decided to make the warning light [alerting pilots to a discrepancy between two angle-of-attack sensors] an optional add-on.” Might this have been picked up had there been better project assurance focused on safety dependencies?

Outsourced project work requires enhanced oversight

Some experts have questioned Boeing’s decision to outsource the MCAS software to overseas developers. Bloomberg’s analysis, for example, led with the fact that “[Boeing’s] subcontractors have relied on temporary workers making as little as $9 an hour to develop and test software, often from countries lacking a deep background in aerospace.” It cited Mark Rabin, a former Boeing software engineer, who said that code was often shuttling back and forth between Boeing and developer HCL due to errors.

“When you’re outsourcing, the issue of good specifications and design remits is even more important,” says Hammond. “But people often discount cultural norms when they’re managing projects like this. I’ve seen situations where outsourced developers simply don’t challenge the wisdom of the original specs, for example, because their culture is less questioning or they have a commercial imperative to be as ‘easy to work with’ as possible.”

Managing contractor partnerships is a key focus at HS2. “You need to have the dialogues in place to ensure trust in a third party,” Stacey says. “That can be a challenge for project assurance. How often do you check their work? Or visit their facilities? Then you also need to ensure that their work dovetails with other contractors. For us, it’s HS2’s reputation on the line – so you have to be sure what’s going on.”

Speak truth to power – especially when You’reunder pressure

With senior management, key regulators and politicians all eager to get the 737 MAX on sale, the pressure on the project teams to get the job over the line must have been intense. It’s here that company culture comes in. How well supported are project managers who discover impediments to progress? How open is management to receiving bad news?

“If the project manager believes they are going to get the sack if they report unflattering metrics, they may well suppress the reality until they have secured the best solution,” says Charles Mills, programme director at CPC Project Services and formerly a senior project manager at Transport for London, where he headed up London Underground’s portion of the Crossrail project.

“You often end up with heroic delusions – project managers who convince themselves they can overcome issues with a bit of ‘D-Day spirit’. But that can be disastrous,” he says. Having an executive culture that is open to the truth – no matter how inconvenient – really makes a difference.

“In many organisations, project managers don’t have the authority they should have,” Lester argues. “They need to be able to stop a project if they have concerns; they’re the managing director of the project, if you like.

“Today we have professional project management disciplines, but project management still hasn’t risen to the level of influence it should,” he adds. “Delivering safety is one of the criteria that can genuinely make project management more prominent and respected.”

That’s one reason why a discrete project assurance function can be helpful. “Because you’re not involved in the day-to-day stuff, you can retain a focus on the bigger picture,” says Stacey. “Project teams are often driven by deadlines and hitting budgets. That means external assurance can be better placed to make the case for issues such as safety.”

“Having an assurance function with status, credibility and respect that is able to report bad news to ensure the safety of the people on a project – and ensure that the project itself will be safe – is hugely valuable,” adds Mills.

That’s as much a cultural question as it is one of process and rigour. And for the Boeing 737 MAX teams, it’s a lesson to be learned no matter the outcome of the current investigations. When it comes to safety, everyone who flies will want to know that future project managers can draw on a robust institutional memory and a credo of putting safety first.

What’s your appetite for risk?

How often does your project team, or its sponsors, actively discuss risk appetite? “Understanding what risks can and can’t be taken, and for what rewards, ought to be a much bigger part of project planning,” says Pamela Stacey, head of programme and corporate assurance for HS2. “It’s a guiding principle. At HS2, safety is our number-one concern – and people might say that makes our risk appetite too low. But it needs to be that way to avoid other priorities displacing it at any point.”

The other misconception she has come across is confusion between ‘risks’ and ‘issues’. “The difference is things in the future versus things now,” she explains. “People on a project often prefer to address what they can see happening. It feels tangible, and a solution can be found. But they forget that an issue cropping up is just a risk that has crystallised.”

Her prescription? Look at what the immediate issues tell you about how risks are developing and might surface in the future. “If you don’t have that analysis, you’ll struggle to update your risk profile thoroughly. Issues tell stories.”

Stacey, who also sits on the committee of APM’s Assurance Specific Interest Group, adds: “If you’re focused on the next milestone or stage gate, you might stop looking forward at risks that could emerge later – or the overall project outcomes.” So look beyond a post-project ‘lessons learned’ process – and adapt dynamically to the way risk evolves.

Richard Young is consulting editor to Project and an occasional chair of APM events