AI in project management: where it comes from, what it is capable of and what comes next

Recently, there has been a lot of interest in the potential of large pre-trained language models, such as OpenAI's GPT, to transform the field of project management. As the leader of the Agile Management special interest group of the German IPMA branch, (GPM Deutsche Gesellschaft für Projektmanagement) I’ve been actively researching the topic of artificial intelligence (AI). I have focused on the most promising areas of research that lie at the intersection of business and AI.

One of the biggest obstacles to using AI or machine learning (ML) in project management is the lack of accessible and structured data sets. However, recent advancements in machine learning have greatly increased the potential for AI/ML use cases in project management. The realisation that deep neural networks can be used for data pattern recognition and natural language can be used as a basis for data has led to considerable progress in computer-based language understanding and processing. Additionally, transformer-based approaches to natural language processing have improved AI's ability to understand and contextualize text.

What is AI capable of?

In 2021, we started experimenting with models like GPT2/3 and GPT-neo, as well as other models that are either open source or available to researchers at low cost. We were interested in assessing the information contained in these AI natural language processing (NLP) models and whether these large pre-trained language models could be considered as knowledge bases. We developed a maturity model for AI-generated answers to project management related questions. The model’s level 0 would mean ‘illegible output’, levels 4-6 indicate a ‘factual correct answer’, while levels 5 and 6 would in addition assess whether the correct answer relates to a conventional, or a rather agile project management mindset.

The results were surprising. The GPT3 model performed well above a level of 4, with almost 75% of all answers rated 4 or higher. In fact, this model would have a good chance of passing a theoretical project management certification exam. Other models, however, did not perform as well, with smaller open-sourced models producing coherent sentences but often not providing answers that were relevant to the question. Additionally, attempts to train these models with information from the A Guide to the Project Management Body of Knowledge (PMBOK® Guide) resulted in even lower performance.

In 2022, we continued our research in AI and project management using public databases of publicly funded projects. These databases contained detailed information on a variety of innovative projects, with a focus on concise prose descriptions. We asked GPT3 to provide concise three to five sentence summaries of the projects and had the responses independently rated for readability, correctness, consistency with the original information and understandability. The results were once again impressive, with the language model delivering outstanding performance and receiving mostly top marks.

We also attempted to challenge the NLP model further by asking for its estimate of the cost and duration of the projects. The answers were more varied in this case, but in many cases, at least the order of magnitude was realistic. In some instances, the model even admitted that it couldn't provide a realistic estimate! This was just a curious experiment and we didn’t rely on the model's answers for decision making.

However, we did notice that when we asked the model multiple questions at once, the results became worse. The language model produced answers with information that wasn’t contained in the original project documentation, and also appeared to be incorrect on further research.

What does this mean for projects?

All the research we conducted was experimental in nature, but we collected and published a list of potential use cases where large pre-trained AI-NLP models could be beneficial for project management in the future. In my opinion, project management will greatly benefit from AI models in many different ways.

Tools like ChatGPT will help project managers communicate more effectively and become more recipient-focused both within and outside of the project. This is because generating content related to the project becomes much easier and requires fewer resources than before.
AI tools are great at providing new ideas and help overcome the fear of starting from scratch.
They can be used to challenge one's own positions and arguments, ultimately leading to better decision making within projects.
AI agents will become valuable sparring partners that can be consulted without any social barriers, providing answers to difficult questions and dilemmas that may have been previously avoided.
Lastly, with some further improvements regarding flexibility and teaching models new information, AI will become the central knowledge hub within projects. They will be able to know what information is available within a project, which version of which conceptual document exists, where it is located and whether or not the latest information was already incorporated within it.

It will be interesting to see what role AI agents will play in the future for project management and beyond. From a technical perspective, one of the most significant constraints currently faced by these models is the text-length limits. GPT3, for example, can only process a limited number of tokens at a time. However, models like ChatGPT are context-sensitive and can remember previous answers. This enables them to perform thought experiments like "If A was true, what would B say about A?"

This will be the foundation for future, more robust and flexible models like multimodal models which will be extremely helpful for project managers. Such models can help analyse incoming documents and check for inconsistencies with project requirements mentioned elsewhere within the project. However, for this to happen, models will need to be trained and updated much more easily and quickly to become experts in specific projects and thus be helpful agents in project contexts. Considering how quickly research breakthroughs are happening in this field, I am confident that we will see even more impressive advancements soon, and that this technology will be incorporated into our projects at every step along the way.

_References:_{Nuhn, H. F. R.; Oswald, A., Flore, A., Lang, R. (2022): AI-supported Natural Language Processing in project management -capabilities and research agenda. IPMA Research Conference 2022. DOI: 10.56889/nrkr7690}_{Nuhn, H. F. R. (2021): Organizing for temporality and supporting AI systems – a framework for applied AI and organization research. INFORMATIK 2021. DOI 10.18420/informatik2021-092}