Most Research in Deep Learning is a Total Waste of Time - Jeremy Howard | AI Podcast Clips

Subscribe to this YouTube channel or connect on:

[Rhannmah]

The underlying problem to this is the pressure to publish "improvements", positive results. This is endemic to all fields of science research and drags them all down. Null results are frowned upon, but shouldn't be, because knowing what NOT to do is just as valuable information as knowing WHAT to do.

[ddd]

It's reality. In my university, which is actually great, some guys admitted that they write a paper just to write a paper. They know that this paper has no scientific value, but they just want to be published.

[Elocess]

Most of research has no immediate application, but that doesn’t mean they are a total waste of time. In fact, it is precisely the blind pursuit of immediate application that hinders scientific advance. Funding for basic research has been ever dwindling, and scientists have been forced to settle in the circle of their most immediate expertise to get a job. To solve our society’s problem you need to allow people some freedom of exploring seemingly irrelevant things, just like in reinforcement learning.

[sassort]

One advantage of doing simple, "useless" research is that we familiarize ourselves with the tools and practices of our field. This is especially true for PhD students.

[YashKumarAtri]

He is right, most of the research is just to increase the count of papers rather then actually solving real world problems.

[munkhtulgabattogtokh]

Being allowed to do "useless" stuff for my undergrad thesis was a such a luxury to me. Honestly, I did it out of pure passion and aced my degree thanks to it, securing a PhD scholarship.

[alexandredecarvalho]

I am half way through my phd and i can totally relate...

[scofield]

I think another reason for this problem is the feedback loop caused by people doing pure research needing to secure funding and looking at what other programs got funded. The majority of modern computing advancements can trace their roots to Bell labs in the 60s and 70s, and it's policy of hiring smart people and paying them to work on whatever interested them.

[mattiaracca]

As PhD student doing Active Learning for Human-Robot Interaction, this warms my heart. A little bit.

[ahumandoing]

It seems like the more things are discovered, the more things there are to explore and most papers or "research" is worthless but you never know if something could lead to something more worthwhile. Science continues to branch out into sub-disciplines and cross-disciplines. Most research is worthless, but you have to fund everything if you want to discover something.

[empathylessons]

In my experience in Deep Learning research, it's been 10% coding and 90% waiting for code to finish running.
But then again, I have no publications so I'm probably doing it wrong lol

[MosesMatsepane]

Came here to feed my confirmation bias and in turn, ended up learning something new. :) Four minutes and thirty-nine seconds well spent.

[hlen]

A friend of mine just beat SOA massively in translating sound in an emergency scenario, by implementing old embedding algorithms, likely because no one else cared to try...

[waterguyroks]

So many people commenting have completely misrepresented his position to the point that I don't think they're even listening to the words coming out of his mouth.

He's not saying basic science is pointless. He's criticizing the fact that the vast majority of the machine learning community is hyper-fixated on a very narrow set of problems. As a result, the majority of that research is "useless" as the field is bloated with inconsequential optimizations by people simply looking to get citations.

In fact, he's advocating for MORE basic science in under-explored fields that could still make huge breakthroughs and radically improve the state of machine learning.

It's frustrating that people don't even respond to the video but rather to the imaginary argument they constructed in their minds when they saw the video title.

[jwgu]

Stop blaming or laughing at the incremental research done by junior people — they need to minimize risk and get publication. It is these big shots/famous researchers’ responsibility to guide the fields to the right, or most risky directions!

[ethan_zoller]

I think a major role in why lots of research is a "total waste of time" is because the element of popularity/interest plays a role in what an engineer/researcher will want to put their time into and also what the public response will be. People want to see AI beat professional video game players, they don't want to read a technical report bloated with jargon and lots of stuff that will be hard to understand and stay focused on.

[kyokushinfighter]

After finishing my PhD in ML and working in real world ML (NOT in FAANG). I totally agree with Howard.

[kaptainkourasi]

I think there is something important that is missed here: Neural networks were 'useless stuff' for 20 years but some researchers kept on doing academic research on them. Academics continue to explore many directions and most of them end up being useless, but the whole point is that we don't know what will end up having an impact in the real world. That is why its important to keep an ecosystem that explores basic science with a long-term horizon, while companies focus on short term immediate gains.

[rudigerbrightheart]

He has a point here. In addition, many researchers stay in their corner sticking to a special topic for which they get reputation - and build a list of publications-, and this in turn becomes important for getting funded. And so the cycle goes on for years researchers doing the same thing.

[ramondiedrich]

Sometimes, people forget other scientific fields. I use deep learning in cancer genetics to create a genetic signature to help in prognosis. Since my dataset has almost 27000 columns, it is a great deal help me to help medics to help people. If we don’t have basic research, even apparently useless, science just dont exist.