True ML Talks #13 Machine Learning, LLMs and GenAI @ Cookpad

The 13th episode of #TrueMLTalks features Jose Navarro, Cookpad's lead machine learning platform engineer. Today, we'll delve deep into the ML architecture at Cookpad, one of the world's largest recipe service platforms. Join the conversation as Jose discusses the challenges of building a successful machine learning platform, how they use Nvidia triton inference server to run models on GPU Infra and the LLM and generative AI use-cases being explored at Cookpad.
In our conversation with Jose, we covered the following aspects:

✅ Role of an ML platform engineer at Cookpad
✅ GPU infrastructure at Cookpad
✅ Jupyter hub, MLflow, and the automation pipeline at Cookpad
✅ Sources of data collection for running experiments
✅ How models are put in the retraining pipeline
✅ Nvidia's Triton inference server
✅ Pipelines or processes required for integration
✅ LLM and Generative AI uses-cases at Cookpad
✅ Evaluating the performance of LLMs
✅ Sources to stay up to date with LLMs
✅ Building a real-time inference stack with MLOps

_____________________________________
00:00 Start.
02:47 Structure of the ML teams at Cookpad
09:56 Size of the ML & platform teams at Cookpad
11:12 Managing GPU infrastructure
18:50 Jupyter Hub, MLflow, and the automation pipelines
20:24 Source of data when running experiments
24:36 Using Argo Workflows for retraining ML Models
28:35 Nvidia's Triton inference server and its benefits
34:58 Passing the model in MLflow to Triton for scaling?
38:40 LLM and GenAI usecases at Cookpad
44:04 Evaluating the performance of LLMs
45:21 Sources to stay up-to-date with LLMs
47:02 Essential components for MLOps

___________________________________
ABOUT OUR GUEST

Jose Navarro is an experienced machine learning platform lead at Cookpad. With a strong background in ML engineering and a deep understanding of building scalable architectures, Jose works on the platform team, supporting machine learning engineers and practitioners in delivering efficient and reliable ML systems. He plays a pivotal role in enabling the team to leverage innovative technologies and enhance the user experience at Cookpad, the largest community recipe service dedicated to making home cooking more fun.



ABOUT OUR CHANNEL

TrueMLTalks is a video series in which we interview machine learning industry professionals from Gong, Intuit, SalesForce, Facebook, DoorDash, and other companies. We provide an insightful understanding of their experiences managing complex ML pipelines and developing successful best practises, making it a valuable resource for professionals looking to stay current on the field's latest advances.

_____________________________________
ABOUT TRUEFOUNDRY

TrueFoundry is a PaaS for cross-cloud machine learning deployment that allows enterprises to accelerate model testing and deployment while maintaining full security and control for the Infra/DevSecOps team. We enable machine learning teams to deploy and monitor models in 15 minutes with 100% reliability and scalability, saving money and allowing models to be released into production more quickly, resulting in genuine business value. We deploy on the customer's infrastructure while keeping data privacy and other security concerns in mind.




#mlops #llm #nvidia #genai