How predictive model development is transforming project management

Reading Time: 7 minutes

Meeting deadlines is a critical element of project management. It goes without saying it greatly influences client satisfaction—as the old saying goes, time is money. And yet, Project Management Works found that only 39% of all projects are delivered on time, on budget, and with the required features and functions. This means the majority of projects grapple with missed deadlines, budget overruns, and, ultimately, unhappy customers. Why do so many projects fail to meet their objectives?  

Often, it boils down to inaccurate predictions and inefficient resource allocation. In the past, Project Managers relied heavily on their intuition and experience to estimate the time needed for task completion. As projects become more complex and with so many variables to consider, it is becoming increasingly difficult to make accurate predictions. But what if there was a way to move beyond relying solely on intuition and experience? In fact, a solution is within reach: predictive model development. 

Powered by machine learning (ML), predictive model development transforms how Project Managers forecast project completion dates. By leveraging historical data and intelligent algorithms, machine learning models can learn patterns and make data-driven predictions about future outcomes. This approach not only improves the accuracy of delivery date predictions but also helps optimise resource allocation and reduce downtime.  

In a recent project, I was tasked with demonstrating the power of machine learning in project management. I developed a machine learning model to predict project delivery dates, resulting in significant improvements in business efficiency, resource management, and customer satisfaction. 

Here, I outline the details of this case study, exploring the fundamentals of machine learning and how its application in predictive model development has transformed project management.

Machine learning fundamentals

Seasoned technology experts and C-level executives are likely already well-versed in machine learning’s capabilities. However, it’s worth revisiting the core principles that drive its effectiveness in project management. To briefly define the technology, machine learning is a subfield of artificial intelligence (AI) that focuses on enabling computers to learn patterns and make predictions from data without explicit programming. 

In the context of project management, machine learning algorithms can analyse historical project data, such as task durations, resource assignments, and actual completion times, to identify hidden patterns and relationships. This analysis allows the ML model to predict future project completion dates with improved accuracy.

There are three main types of machine learning:

1. Supervised learning, where the algorithm learns from labelled data to predict outcomes.
2. Unsupervised learning, where the algorithm identifies patterns and structures in unlabeled data.
3. Reinforcement learning, where the algorithm learns through trial and error, optimising its actions to achieve a specific goal.

In project management, supervised learning is often used for predictive model development, such as forecasting project delivery dates. Unsupervised learning can be used to identify clusters of similar tasks or projects, which can help in resource allocation and risk management. Reinforcement learning can be used to optimise scheduling and resource allocation in dynamic project environments.

The use of machine learning in project management offers several benefits:

• Improved accuracy of delivery date predictions.
• Increased efficiency in resource allocation and utilisation.
• Enhanced decision-making based on data-driven insights.
• Reduced risk of project delays and cost overruns.

By automating tasks such as project forecasting and resource allocation, machine learning can more accurately predict deadlines, foresee bumps in the road, and effectively control resources. However, this isn’t to say that ML will replace the traditional Project Manager; instead, their role will become more strategic, creative and human-centric.

The changing role of the Project Manager

The rise of machine learning in project management is not meant to replace Project Managers but rather to enhance their capabilities with new tools and insights. As AI takes over routine tasks such as data analysis and prediction generation, Project Managers can focus on higher-level responsibilities that require human expertise and judgment.

This shift demands a new set of skills for Project Managers, including data literacy, the ability to interpret and communicate AI-generated insights, and a deeper understanding of ethical considerations related to AI. Any Project Manager worth their salt will need to become adept at collaborating with AI systems, leveraging their strengths while mitigating their limitations.

For example, the Project Manager will play a crucial role in ensuring that AI is used ethically and effectively to enhance outcomes and foster a collaborative and innovative work environment. One primary concern is the potential for bias in the data used to train the algorithms. If historical data reflects existing biases in project assignments or performance evaluations, the machine learning model may perpetuate those biases, leading to unfair or discriminatory outcomes. 

Furthermore, the “black box” nature of some machine learning algorithms can make it difficult to understand how they arrive at their predictions. This lack of transparency can raise concerns about accountability and hinder the ability to identify and correct errors or biases in the model. The Project Manager must be conscious of this double-edged sword and will play a crucial role in analysing the ML outputs and making decisions from a holistic perspective. 

As such, it’s essential to prioritise fairness, transparency, and accountability throughout the model development and deployment process to ensure the ethical and responsible use of ML. This includes carefully selecting training data, using explainable AI techniques, and establishing clear lines of responsibility for the model’s predictions and their impact on project outcomes. 

Hand in hand with the exciting possibilities presented by predictive model development, I applied this level head to a project for a client facing substantial project delays and resource utilisation challenges.

How to develop a predictive model: A case study

As precision and predictability are paramount within project management, organisations constantly grapple with the challenge of accurately forecasting project delivery dates. Consider this: a study by Scrum Inc. found that 47% of Agile projects experience budget overruns, are delivered late, or result in unhappy customers. This highlights the urgent need for innovative solutions to enhance project planning and execution.

This project was initiated by a client of mine who faced precisely this challenge. They had a history of project delays and inaccurate delivery date predictions, which negatively impacted customer satisfaction and resource utilisation. They needed a way to move beyond relying solely on intuition and experience and leverage the power of data to achieve more predictable outcomes. Predictive model development was the logical conclusion.

The method

The company had two main data sources: SPGH, their in-house logged hours management system and work calendars. SPGH provided data on resource utilisation and time spent on various tasks and projects over the past 15 years, while work calendars offered crucial information on staff availability.

The primary challenge was the changing task structure over the 15 years, requiring continuous adaptation and normalisation of the data. To address this, I employed Pentaho Data Integration (PDI) for ETL (extraction, transformation, and loading) and orchestrated the integration with LiquidPlanner, the SPGH, and the machine learning model. PDI facilitated data transfer between SPGH and LiquidPlanner and executed Python scripts for the predictive model.

LiquidPlanner, a dynamic project management software, was used for its predictive scheduling and resource management capabilities. Its API enabled integration of the predictive model using Python‘s requests library to manage API calls and authentication. Python was used to develop the predictive model with libraries like scikit-learn for machine learning algorithms and pandas for data manipulation and analysis.

To determine the most accurate approach to predictive model development, I tested several machine learning algorithms:

• Multiple Linear Regression (LR): This algorithm modelled the relationship between independent variables and project duration.
• K-Nearest Neighbors (KNN): This algorithm classified samples based on the majority of their closest neighbours.
• AdaBoost: This algorithm combined multiple weak models to create a stronger, more accurate predictive model.

The most effective algorithm was AdaBoost. But how does AdaBoost work? It’s an ensemble learning method that combines multiple “weak” learners to create a “strong” learner. In simpler terms, imagine it as a team of experts with different strengths working together to solve a problem. Each weak learner makes a prediction, and AdaBoost assigns weights to each prediction based on its accuracy. The final prediction is a weighted combination of all the individual predictions. AdaBoost is particularly effective in handling complex datasets with high dimensionality, making it well-suited for predicting project delivery dates.

The data used for training the predictive model and project management were stored in relational databases managed by MySQL. Procedures in PL/SQL were created to generate the necessary features for the model.

Summary of approach

• The company’s in-house logged hours management system (SPGH) and work calendars were the primary data sources.
• PDI was used for data extraction, transformation, and loading (ETL).
• LiquidPlanner’s API enabled the integration of the predictive model.
• Python, with libraries like scikit-learn and pandas, was used for predictive model development.
• AdaBoost was the most effective machine learning algorithm for predicting project delivery dates.

The results

The predictive model was a resounding success for the client, providing a valuable tool for project management. The implementation allowed them to offer more reliable delivery dates to customers, which fostered trust and transparency, leading to increased satisfaction and loyalty. 

By accurately predicting project delivery dates, the model empowered the company to optimise resource allocation, minimise downtime and maximise utilisation while proactively identifying potential delays. This allowed for timely interventions and kept projects on schedule and within budget. As a result, this improved predictability enhanced the company’s reputation for reliability and efficiency, giving it a competitive advantage in the market.

This predictive model development process transformed the company’s project management processes, enabling them to deliver projects on time and within budget while exceeding customer expectations.

Summary of achievements

• More reliable delivery dates were provided to customers.
• Resource planning was optimised.
• Downtime was reduced.
• Delivery date prediction accuracy was improved.

Beyond intuition and towards an ML-powered future

This case study underscores the transformative potential of ML in project management. By leveraging historical data and intelligent algorithms, companies can achieve remarkable improvements in delivery date prediction accuracy, resource allocation and customer satisfaction.

But the real takeaway is the paradigm shift: moving away from intuition-based project management and embracing a data-driven approach. This shift empowers businesses to make more informed decisions, optimise resources, and navigate the complexities of modern projects with greater confidence.

The future of project management is undoubtedly intertwined with the advancement of machine learning. As AI-powered tools become more sophisticated and accessible, companies that embrace this technology will gain a significant competitive advantage. By automating routine tasks and providing valuable insights, machine learning frees up Project Managers to focus on strategic planning, creative problem-solving, and fostering stronger client relationships.
If you’re looking to enhance your project management capabilities and unlock the power of predictive analytics, consider partnering with a skilled Machine Learning Engineer on your predictive model development initiative. Outvise can help you connect with top freelance talent who can help you achieve your project management goals. If you’re not already, sign up as a client to start exploring the portfolio of professionals (including myself!).