The Sad Reality of AI Job Market w/ ML Engineer

This is a Part 1 of a conversation with Tanner Ducharme, Machine Learning Engineer & Data Scientist. As AI Industry is dominating the headlines we take a deeper look what doesn't get covered: the reality of AI Job market, challenge with integrating AI into a business, and controversy surrounding prompt engineering.

Say Hi on Linkedin 👋:

In the Part 2 we will dive deeper into what is machine engineering and how realistic it is for people to get into AI. Maybe hit that ✨Subscribe✨ so you don't miss it :} (thank you! ❤️)

⬇️Resources mentioned in this video⬇️

Timestamps:
00:00 "I will make a 1M dollar and retire in 10 years"
01:45 Why ML Jobs Are Down in 2023??
03:25 The Issue How ML is Taught
04:43 Why it is hard to implement AI into a business
08:14 The Rise of New Professions (eg. Prompt Engineering)

P.S. Tanner's audio failed us, but I think the conversation is super valuable and my hope is that captions will help out to understand when audio glitches. Sorry.

We are excited what you think!

[raybod]

Tanner explains reality of being in technology field very well. Timing is everything. You are hired because you have the greatest potential to create value for the company which justifies extreme pay. Only the top 1% get those tech jobs or the very lucky, but expectations are always high. Cutting edge tech jobs are at minimum 60 to 70 hours a week if you include time maintaining skills. Elon Musk led the way with tech job cuts with Twitter. Twitter had a tough transition, but is doing well with about a third of the previous staff.

[luihinwai]

Applied Scientist at Amazon here. A lot of AS/ML eng really are just doing engineering, building the data and ML infrastructure, integrating with the wider system of the company, etc. However, what's really needed are research skills. This is different from applying what's out there, try a bunch of different models and see which one works. Research is about the creation of knowledge. It's about coming up with new ideas/hypothesis, create experiments, and evaluate the results. To be really good at research, it's not about coding. It's about reading research papers. I would say you should spend at least 30% of your time reading papers, then 30% of the time doing the experiments, and 30% of the time launching and/or writing paper. While a PhD is not strictly required, it is the most systematical way to acquire these skills. At the end of the day, you need to ask yourself, do you have the ability to create knowledge? Knowledge needs to have a general applicability, and generality requires repeatable experiments, otherwise it is just "information".

[eternaldarkness]

I got into the AI job market at the end of 2022, I deliberately looked for a job at startups, because I thought that if I got a job at a big company I would handle 1 part of the pipeline, but I wanted to learn the whole process from data Engineering right through to implementing it into a production environment. So far I have developed processes for every stage, and now testing the full pipeline for some parts of the project. I am the only person working on the Machine Learning Team, so I have had to work out how to design each step myself with only the internet and the knowledge I got from a coding boot camp.

[anneaguirre]

Couldn't agree more. I am an "AI Engineer" on paper, but more than half of my work is Infra/AIOps. I'd say it's very rare to work as a model developer in this market. Amazing video btw! :)

[jongxina]

turns out our asian parents were right all along: become a doctor

[tolgahdur]

I think companies either hire software engineers with a background in ML to implement their model as Tanner says, or PhDs who do the training, etc. of the models. Someone who "only" has a Master but also doesn't have any significant software engineering experience is a bit in the awkward middle - you are not practical enough for an implementation role but not theoretical enough for a training/research role. That's probably the issue that Tanner is facing.

[theaugur]

What companies want from their ML engineers is to show them how to add value to the business. Either one needs to understand the business problems or partner with a team that knows those problems.

[charlesroberts]

Never try to time the market, job, investment, dating, or otherwise. Do it because you like it, otherwise you will be burned.

[reiniermorejon]

Kudos to you Goda, what an incredible channel you have built here. Congratulations!!!

[pluto]

one issue I am observing is that lots of companies skipped over conventional statistics that could help them and just jumped into ai to brute force solutions that arent going to work. They were sold a lie it could solve anything and they will get disappointing results and be averse to hiring for years to come.

[ub]

I second that the boom was in 2022. My gov lab was even peddling ai summer classes, which they've never done before. Then in 2023 it was contracts-no-renewal year lol.
Also in 2023, I vividly remember how nobody wanted to admit they were using ChatGPT to help write papers.

[lizziethelemon]

I have multiple degrees in engineering & business and a graduate degree in AI and I worked both in research and in industry. AI has multiple sub-fields. I'm always surprised when someone just studies ML by itself, and considers it enough to breeze through the industry, that's not enough. You either need to be a domain expert in some other field or at the very least have DevOps skills because most companies do not have large teams as companies are still assessing the feasibility and business value of AI. Having a team without domain knowledge of the problems they are working on increases the likelihood of those projects failing or not attaining the highest/best business value. It's perhaps not easy to have multiple degrees and work experience from various industries but if you can invest/sacrifice the time it will always pay off. When I did multiple degrees I just didn't want to be stuck in one field and when I studied AI it wasn't such a hype but everything changed so fast and now companies are scrambling and people are jumping on the AI bandwagon without a proper strategy. Take your time, do some proper research so you don't waste your time.

[MarcAyouni]

I finished my masters in AI a year ago, and I still can't find a job. Why? Because my area is not in LLMs and like it's been said in this video, all the jobs are for seniors.

[niederrheiner]

When he is finished wirh his phd in a year the AI hype bubble is gone 😂

[Clammer]

Love this video. One of those rare ones that remind you of the harsh reality out there with regards to jobs in this field. Of course, in any field, being able to land a lucrative job is not easy as some has mentioned in the comments with the key factor of whether you are at the right place at the right time. However, this video is all the more pertinent during this period considering the mad rush towards anything AI/ML-related and there is a danger that jobs in this field may be over-hyped.

[ub]

Becoming a better prompt engineer is, for the most part, a waste of time. I've noticed GPT4 has been getting better in the background. And of course, their whole goal is for the system to respond usefully without slinging special prompting tricks lol. For example, GPT4 used to completely rewrite my code and cut parts out, whereas GPT4o initially would just repeat the code back to me. Recently though, GPT4o can do 300+ lines of python and preserve my original program while implementing the prompted tweaks. This all happened recently without any announcements.

[TheClassyHacker]

Thanks for this video; let me know when you have a class for ‘integration engineers’.
P.S. Will say hi on LinkedIn 😊

[marcotroster]

Same issue here. I tried to find an AI job last winter. Tbh I got 1 or 2 jobs, but not the ones I wanted. So I decided to go the senior SWE route and become tech lead instead.

I've always seen AI in the broader context of process automation. I used to freelance as well and developed customized tooling to support work processes.

So yeah, as said in the video, you need to be a good software engineer and have AI skills on top nowadays. For me, AI is just another tool in my toolbox to develop cooler automations.

I cannot tell you the countless times I joined some CSVs with pandas and cleaned the data for some quick insights, etc. The practical data mining skills are underestimated IMO.

[hdtlab]

Degree matters. So I advise people with non-IT/math background to stop jumping into this sink hole. In today market, AI is a tool, it is no longer a field of knowledge. It is better to do something else that make use of AI, unless you’re top 0.1% who invent the wheel.

[DanFrederiksen]

AI is not just AI. Some can do what others have done before, some can do what no one has done before. There is a price difference between the two.