How to actually manage an AI project?
As an AI project manager, you need to understand the basics of AI so you are always at the front edge of developing innovative AI products. This article will explain the fundamentals of Artificial intelligence and the steps included in building a best in class AI product. You will also get an essence about the strategies and approaches that can be utilized to make the project successful.
Let us start with the definition of AI.
“The science and engineering of making computers behave in ways that until recently, we thought required human intelligence.” — Andrew Moore
What AI can do?
AI can be applied to solve multiple business problems in almost all industries. Few of the use cases that can be solved using Artificial Intelligence are as follow:
· Face recognition
· Sentiment analysis
· Market segmentation
· Anomaly detection
· And many more!
You should also be well aware of the building blocks of AI: what exactly goes into building an AI product and who all work behind the scenes to bring an idea to life.
Machine Learning
Machine learning is one of the biggest segments of AI, it provides systems the ability to automatically learn and improve from experience without being explicitly programmed. We have various types of machine learning approaches.
1. Supervised ML — In supervised machine learning, we provide the machine with class labels and ground truths to have a clear idea of what we are trying to predict. Based on the provided output, machines try to learn and predict the unseen data or data with no class labels.
2. Unsupervised ML — In unsupervised machine learning, no output or class labels are provided rather we try to predict the patterns and trends in the data. One of the biggest methods which use unsupervised machine learning is clustering.
3. Reinforcement Learning — An agent gives rewards or penalties to make a machine learn in a predefined environment. To give a layman example — A parent teaches their child to recognize a dog.
Now let us move to the other very important part of artificial intelligence which is deep learning.
Deep Learning
This is also said to be the core of machine learning which is influenced by the workings of the human brain and how a human being learns. It mainly consists of a large number of neurons that help the machine learn patterns in the text, image data etc.
NOTE: As an AI product manager you have to make sure whether the data you have in hand is enough for making decisions or not. If not, you need more data from various sources to make your AI model more robust.
Models are not good or bad but the data that goes in and what decision-making approach you choose matter the most.
Now, let us try to understand the process of developing and deploying your AI product into the production environment.
- Data Ingestion — Every AI story starts with data and data is the most crucial element to build a model that performs the same way as it is expected to work. Data can be gathered from various sources. There are many companies selling data and it can also be gathered from various systems and databases of the company.
- Data Cleaning & Transformation — A number of analysts, scientists and engineers work on analyzing, observing and cleaning data to bring it to the expected state so that it can be fed to the model to work wonders.
- Model training — Once we have cleaned and analyzed data, we move on to training our machine learning or deep learning model with that data. Model training is like making a machine learn the patterns and structures in the data with the help of mathematical equations and functions so that it is able to understand and classify the unseen data when it is implemented in the real world.
- Testing & Validation — This is the evaluation stage, where we test the performance, accuracy and confidence of the model on the unseen data to confirm whether our model generalizes well or not. At this stage, we get a clear idea of whether our model is overfitting or underfitting and then we try to tune the hyperparameters and train the model again if required.
- Deployment — In this stage, we ask for the help of the DevOps team to deploy the model in the local environment, cloud etc. as per the requirements. Once the model is deployed and a sanity check is complete, most of our work is done and we move on to the maintenance and retraining of the model as and when required with the new/unseen data.
As we have understood the process of developing and deploying a data-driven product, we will now move forward to understand the approaches and strategies to make an AI product development project successful.
As an AI product manager, it is necessary to be involved in all the stages in the development process and make sure that everything is happening as expected and everyone working in the team must be focused on the achievement of one goal.
The following approaches and strategies should be used in order to complete each step in the best possible manner.
1. Business problem — One of the most critical and the hardest part of developing any product is to understand the business problems and needs. Though there can be many problems that a business might be facing, but as a manager, one must make sure that we go about solving the problem which really needs a solution and which will help the clients or their customers’ to get the highest value possible. If our solution is saving time and cost on the future operations of the company then that would be one of the best solutions we can offer to our client. In short, we should narrow down the business problem, define it, quantify it, understand who the stakeholders are, what their priorities are etc. It sometimes takes months just to understand the core problem.
Always try to put business problems before data and target only those problems which are feasible to be solved and provide maximum value to the client and their customers. Make sure to quantify the goals which are measurable, achievable and aligned with the client’s requirements.
2. Data — Data is the new oil but to some extent it is non-exhaustible. Data makes up the core of every AI product. It can be textual, visual like images, videos or even audio data. Once the business problem is defined and we have a clear idea of what we are going to develop, we must try to find out what kind of data we will use in developing our model, how we will collect that data. We should be very careful about the privacy and security related concerns of the data. The AI model can work on real-time data as well. So a pipeline of data can also be prepared to have a smooth flow of data to our model and get the predictions done without any issues. Most of the time data is not static and it keeps on changing. In that case, our model would not be able to understand the new data well and might even misinterpret the data which might lead to a loss. Hence, there comes the need for retraining the model with the new data which keeps the confidence and accuracy of the model very high.
The data that goes into the training of the machine learning model should be complete, relevant to the business problem and must be of high quality. All these factors are very important to build a state of the art model and solve real-world problems accurately.
3. Model Building — Once the high-quality data has been gathered, it is important to analyze and clean the data and train a machine learning model using it. Model building is not a one-step process, it requires various stages and operations to be carried out to decide the final model which goes into production. Most of the time, machine learning engineers develop a baseline model to evaluate the performance of various models against it.
Feature engineering is a process of creating tailor-made independent features which have a good correlation with the dependent features. It should be done in order to make the model understand the relationship well and perform more accurately.
Feature Selection is the process where you automatically or manually select the features which contribute most to your prediction variable or output which you are interested in.
Hyperparameter tuning is the process of tweaking and tuning the parameters of our model to get better accuracy on both training and test data. In simple words, it is like the modification of the mathematical equations to improve the performance of the model while making predictions.
4. Deploy & Measure — Once we have achieved the best model after all the tweaking and training the model, we deploy the model so that it can be used by the final users. We need to make sure whether the model is performing as per the expectations of the client or not. Also, whether it is able to achieve or exceed all the KPIs or not.
Data scientists are also involved in carrying out an A/B test to find out the best possible option between two available models. It also helps in understanding what business impact our model has. For e.g. Netflix executes many A/B tests before finally deciding on a feature on their application.
AI product should not be a static product which once developed and deployed does not require any maintenance. There must be always room for improving the performance and making the product better and more user-friendly.
5. Active learning — Active learning is very important for the AI model to work confidently. The process does not stop after deploying the machine learning model. Engineers are required to retrain the model so that the quality and accuracy of the model never deteriorates. A retrospection is also required to understand what could have been done better and to improve the efficiency and effectiveness of project management and engineering efforts.
I am very sure that you would have got an idea about the processes and approaches which are followed in developing an AI product. Now we will try to understand some of the crucial aspects of handling various stages of an AI product development project.
Domain Knowledge
AI can be applied to a large number of industries and domains. As a product manager one should try to understand the domain he/she is planning to develop an AI application for.
Customers’ pain points should be accurately assessed and competitive analysis should be carried out to understand the market situation better and to learn what all AI products your client’s competitors are developing and how those products help customers provide value. This can be used to reduce the efforts of the customers and make the journey easy for them.
Find out opportunities and alternatives and finally choose the one which provides maximum value to customers.
Project Statement
A project statement is a document that is created before starting any project in order to have a clear picture of the product we are developing. The project statement should answer these questions:
1. What problem we are solving? Define the business problem your client is facing and how are you planning to solve it.
2. How can you use AI? Think and mention the use of AI in your product and how it will help you make the product more efficient. Is AI really necessary? Never force yourself to use Machine learning just because it sounds cool. If the product can be developed with other technology and does not really require AI, go with those alternatives.
3. Data Requirement. Once you have decided to build an AI product, find out what data you will use to build the models. How are you going to collect it?
4. Scope of the product. As a user how your product will make your clients’ lives easier by solving their problems and by helping them increase their revenue and sales.
5. How do you measure success? Decide on the performance metrics against which you will evaluate your model and measure the impact and success.
Metrics
Metrics are a key part of evaluating and judging the performance of any AI product. You should select the right metrics which should be:
· Measurable
· Directly related to business problems
· Comparable
· Aligned with business objectives
The metrics which have the highest impact on clients’ business should be given the highest priority.
Example — Suppose a furniture manufacturer wants to develop an application which tries to predict machinery breakdown using IOT devices before it occurs, in order to be prepared and save time on operations. It would reduce any hindrances in work for them. For this use case, the time constraint is the most important factor as we would not want to get the alert after the machine broke down. Hence, we can set a KPI of getting an alert as 5–10 minutes before the IoT device sense something weird with the machine and provide an alert.
There are other evaluation metrics as well which help engineers and scientists find out how good a machine learning model is. Some of them are:
· Precision
· Recall
· F1 score
· MAE (Mean Absolute Error) etc.
Project Management Approach
Most of the project management is done using an iterative approach called scrum. If you are interested in learning more about the same visit the Atlassian website given at the end of this article.
Sprint should be effectively planned and everyone should work as a team towards the business goal. After planning the scrum and developing and delivering the first cut of the product, sprint review and retrospection should be done in order to find out what went right and what needs to be improved. Make sure that everyone is aligned with their tasks and not overloaded with work. Proper assignment of work is necessary to have harmony in the work environment.
There are various approaches that can be applied to manage a project. The Visual paradigm’s article explains it very well. It can be found at the end.
As a project manager, whether you are managing an AI project or any other project, you must understand that you are the key person who will make individuals work as a team and direct their efforts in the right direction.
A project manager must have an idea about who is working on what and if anyone is overburdened before assigning tasks. In this way, you can bring harmony to the project as well as in the organization.
I come across many project managers who think of themselves as bosses. This is one of the biggest mistakes one can make as a manager. We are required to be supportive to all the team members as a pillar, be respectful and humble because a project manager should make things work rather than forcing things on people.
Let us end this article with the following quote about project management.
“Project management is just like sailing in the rough sea with many passengers onboard and helm is in manager’s hand.”
