Apache Spark SQL With Apache Hive

=====================================
Apache Spark SQL With Apache Hive

Apache Spark
Apache Spark is an open-source unified analytics engine for large-scale data processing. Spark provides an interface for programming entire clusters with implicit data parallelism and fault tolerance.

Apache Hive
Apache Hive is a data warehouse software project built on top of Apache Hadoop for providing data query and analysis. Hive gives an SQL-like interface to query data stored in various databases and file systems that integrate with Hadoop.

Video Playlist
-----------------------

YouTube channel link

Website
Technology in Tamil & English

#apachespark #apachehive #sparksql

[bunnyvlogs]

basically we require connectivity only....u did it....great bro....

[anupjee]

This was very helpful. Thank you for posting the video..

[prakashgour]

Love it. Thanks man for the efforts 🙏

[indrajitghosh]

@Data Engieering which version of Hdp you are using, I had installed spark 3.x, I unable to view the hive database in spark, it seems that spark is showing databases of its own catalogue, how to solve the issue?

[dhananjayreddy]

Hi Sir, one question, whether it is Spark SQL or Hive, tables will be stored in HDFS right, then why would need these steps to access the tables from HDFS( and saying Hive tables)

[koolkravi]

Hi,

Did you try same with latest apache Hadoop 3.3.1, Apache hive 2.3.8 and Apache Spark 3.2.0

I installed all 3 with MySQL .

But spark sql queries are not working on spark shell. It seems spark sql is not able to use my sql hive metastore .

[ashimabisla]

I am using derby database in hive. I cannot find its connector jar file online

[sanjaykumarsingh]

can you please share the next steps, like creating a table, inserting data, and performing queries using spark?
Its urgent

[bhavitavyashrivastava]

how do i run hive queries with execution engine spark

[narayanareddy]

thanks for your video. could you please share the video integrate spark with hive with intellj idea. i am getting error like "WARN Hive: Failed to access metastore. This class should not accessed in runtime."

[poovendranalagarsamy]

Thanks for this video. how to do it other layers like join, aggregation, concat etc..,

[bunnyvlogs]

I have isssue while setting up pyspark in windows machine ...require ur assistance...i hope you may help me in this...please let me means of connection with you...

[AvinashKumar-nlbp]

Hi, When spark hive with code level video come.