Current approaches to build an ML model
Most organizations were excited about adapting Artificial Intelligence and Machine Learning to automate most of the manual efforts.
“ But do ever thought of what are the necessary best practices required to ensure successful implementation of the model? ”
In general, its common point that these following points going to trigger:
- Hey, we need a well-experienced candidate with solid AI/ML capabilities, who productionize the models.
- We need to look for a scalable cloud vendor to deploy our model to host our page.
So the company recruited a Candidate “ Data Scientist ” or “ ML Engineer ” with good exposure to AI/ML and he got allocated a project on “ product recommendation ” on top of it there are some additional requirements.
For e.g.: Mr. X brought some goods online, and whenever he visits the page, in general, we recommend some alternatives products, now at this moment we also need his family members to be aware of similar product recommendations based on Mr . X’s purchasing behavior.
Though he was engaged with that product key stakeholders like Product/Project, Managers, Business/Data Analyst, Data Engineers, MRM Experts, etc . in all the phases from requirement gathering, market research, etc.
“It was a funded project ”
The Data got onboarded to his platform, what he was expected. He worked closely with the project manager and proposed estimates based on Phases, in-turn project manager updated his workable estimate to the senior – leadership team and that candidate happily started working on his task where initially he was exploring the data, performing statistical analysis, and other things.
During analysis, one question keeps on ringing in his ear’s “How to derive customer Mr. X family members purchasing behavior based on based on based on Mr . X purchasing behavioral DNA ”
He thought to explore some research papers and open-source packages, and finally concluded:
“It would be a great idea that I need to update my project manager, to first dedicate my time in my research to land up with a feasible solution with design, but he was not even confident whether such approach going to work or not because he was worried that ”
a) if I won’t get any open source package instead I need to implement a research paper from the roots
b) Even in the worst case if I won’t get even a research paper too then I need to spend time in skimming all related research papers to discover his experiments close to the requirements and start writing research papers proprietary to his company.
He went back to his project manager and stated his concerns later manager started worrying about that:
- How to convey to the senior leadership team, what might they ponder about his progress?
- Is it that the right thing to suggest senior management open up a new job requisition for a Ph.D. holder background in research specific to Recommendation Space.
Lessons Learnt: Though we can see lot of successful implementation of AI projects it differs from project to project based on Client or Customers’ needs and there are high probabilities that with such requirements one can hardly discover close end Open Source Packages related to Client’s requirement, in turn, such situation could lead to developing a new algorithm from scratch.
One should be much cautious about approaches for solving a problem before committing or proposing to Senior Management.
What are the Strategic approaches:
From project manager and recruiter perspectives:
- Before posting a job’s requirements in job portals, project managers and recruiter’s needs to be well aware of what type of candidate suits best for the job roles, whether a Data Scientist from a research background [Education: Ph.D. or Post Doctoral or a Data Scientist from non – research background [ Education: College Grads.
- Moreover, recruiters need to engage the right candidate.
From management perspectives:
- One should come out of the thought process that:
“ AI is a magic, can do wonders and reach beyond human capabilities ”
I am not repudiating one thought process and in turn admiring the way they think. It’s because of the way we watch Sci-fi movies then our psychology would promenade in that direction to attract such possibilities which were impossible now.
AI can automate most of the tasks but it had some boundaries to reach beyond human capabilities, which was a gray area now and it’s in research, we call it as:
“ Artificial Super Intelligence ”
Conclusion: To be successful in the space of digitization, automation, decision making, etc. and before launching or productionize the project, the project owner or fund initiator wishes to understand the nature of such technologies at a high level to avoid wonders at a later stage and should be given some bandwidth for research to come up with different possibilities to develop a model and come up with initial working model [prototype] with little funding and some scaled AI infrastructure with good computing power either In – Premises or in Cloud-based on the complexion of data.
