Why Hadoop is Dying

Hadoop is dying. And it's happening fast. Learn why in the latest Intricity 101 video.

[KrogerHotdogs]

Did I really watch this all the way to the end to realize it was an ad.

[baigadil]

so basically you are saying "local distributed storage" is dying because "cloud distributed storage"
alright! so whats new!

[sukumaar]

Hadoop is not dying but evolving from the traditional definition of Hadoop.

[scottcameron]

Hmm. Well, maybe. Seems like there's always someone who wants to be the first to declare a demise. I remember over ten years ago reading an article that insisted Bluetooth was already finished.

[kumartechnocrat]

people were saying mainframe is dead some 20 yrs ago lol. But the truth is it's alive for at least next 30 yrs.These naysayers will be always there. Btw world ended on 2012 because mayan couldn't calculate beyond 2012.

[abdata]

Hadoop and big data, in general, is evolving. Using the right additional tools is certainly helpful. It really depends on your data needs.

[HamSolo]

1. Spark, Impala, Tez, Hive LLAP make query much faster now
2. By the time you upload a Terabyte of data to amazon cloud for analysis, the information is already old and useless. Specially with the organization that have daily change of more than 100GB.
I'd admit that small organization may benefit from your argument of cloud as they don't need to maintain the infrastructure but hadoop doesn't require that much love after the initial setup is done properly.

[AshishSingh-quns]

I am from traditional database background, was planning to learn hadoop and spark as a career switch, after watching this video..i need to rethink..could you please help?

[nocontentnoname]

Is hadoop used on a cluster of an organization's own computers that the org has to maintain and manage and the "cloud" the same thing on somebody else's computer that you pay to rent out based on your dynamic requirements of scale and have them maintain and manage. Is that the argument being had here???

[OliverVallack]

If I am sticking to the Azure ecosystem, do you recommend learning stuff like U-SQL in ADLA and Scala/PySpark for Azure Databricks or Azure HDInsight Spark clusters? The new Data Flows in Azure Data Factory V2 are very interesting too, the ETL processes are constructed visually with a nice a GUI which are then compiled into Spark executables that are executed on Azure Databricks clusters. Edit: I'm talking Data Engineering here and some analytics but mainly engineering.

[Privacy-LOST]

Hadoop is more than a Distributed storage system. It is a comprehensive ecosystem of dozens of specific software for dozens of types of usage combined.

[nathanbenton]

google technically didn’t make the initial setup of mapreduce. very nice vid though thanks

[sangmang]

In my opinion, Hadoop's high underlying operating costs are toxic. But Hadoop is evolving, unless ASF stops it.
Don't give up, elephant!

[sunayanakalekar]

what tool should we be learning then if hadoop is dying plz suggest

[samantrao]

An open source solution like hadoop will always carry some drawback compared to the licensed versions. The fight is not about which technology is better compared to others since hadoop will always be the base of big data solutions & technologies, which gives good insight to big data technologies. Besides in this fast moving technology, noone can tell what is gonna happen after 5 years.
And no offense but your statement "noone cares about hadoop" during start of the video sounded like amateur.

[Malstrm]

The Truth is Hadoop was never very good. It was not really new technology but revitalized OLD technology. Also the claims made around it were never going to be realized. It wasn't going to ever replace robust traditional databases nor should it ever have been considered for such a thing. A batch oriented technology is NEVER going to replace the need for thousands of simultaneous users running queries. If Hadoop had not been oversold it probably would be in a better place today but because it was over sold and many fortune 500 companies have had many high profile expensive failures many have soured on Hadoop. When the ROI of "Free" is costly consulting and failures to deliver on projects that often rival the cost of just leveraging large traditional data platforms that simply work its not a hard choice. That said there will always be another Hadoop because people are intellectually lazy and buzz word shiny object oriented they will always fall for it. The cloud and Spark have both helped put the nails in the coffin of Hadoop. I never believed in Hadoop or other proposed "warehouse killers". Big data existed long before Hadoop and in many cases Hadoop was sold in such a way that it was really more of a scam. So for me its good riddance. It merely sucked IT dollars down a black hole because so many executives and CIOs are too lazy to really understand a technology and too egotistic to stand alone against a hype machine.

[podunkman]

I'm studing hadoop subject and I wonder why I should have one in my company.
Hadoop is designed to keep files. But what is the sense to keep data in files? Nowadays we have dozens of databases; sql, none-sql. They all have fantastic speed, cluster abilities. What is the sense to play with some old fashion linux funs ideas?
Really, I'm asking. Can someone give me real life exaples where hadoop can be useful?

[ghansham]

some people want to attract view hence keep heading like.

[naveedfullstackjavadevelop]

IGNORE THIS VIDEO OK IM GRTTING THOUSANDS OF CALLS FIR HADOOP POSITION LOLZZ

[aperture]

I felt this video was bullshit back then when I was managing a large hadoop cluster to run ETL and OLAP. But till now, I have to admit that hadoop is not suitable anymore since the rise of cloud computing and no one is gonna build their own cluster anymore. Using the same old Hadoop is clunky and wasteful, most of people just choose another solution rather than stick with hadoop, unless you have infinite money to pay the traffic and storage. Hadoop is still great and there're a lot of things which can't be replaced in the hadoop stack (like apache hive, hbase and kylin), if somehow hadoop managed to simplify its architecture, I think people will head back to hadoop again