Live Big Data Mock Interview | Techno Managerial #interview | PySpark, Hive, SQL, Python #question

I have trained over 20,000+ professionals in the field of Data Engineering in the last 5 years.

𝐖𝐚𝐧𝐭 𝐭𝐨 𝐌𝐚𝐬𝐭𝐞𝐫 𝐒𝐐𝐋? 𝐋𝐞𝐚𝐫𝐧 𝐒𝐐𝐋 𝐭𝐡𝐞 𝐫𝐢𝐠𝐡𝐭 𝐰𝐚𝐲 𝐭𝐡𝐫𝐨𝐮𝐠𝐡 𝐭𝐡𝐞 𝐦𝐨𝐬𝐭 𝐬𝐨𝐮𝐠𝐡𝐭 𝐚𝐟𝐭𝐞𝐫 𝐜𝐨𝐮𝐫𝐬𝐞 - 𝐒𝐐𝐋 𝐂𝐡𝐚𝐦𝐩𝐢𝐨𝐧𝐬 𝐏𝐫𝐨𝐠𝐫𝐚𝐦!

"𝐀 8 𝐰𝐞𝐞𝐤 𝐏𝐫𝐨𝐠𝐫𝐚𝐦 𝐝𝐞𝐬𝐢𝐠𝐧𝐞𝐝 𝐭𝐨 𝐡𝐞𝐥𝐩 𝐲𝐨𝐮 𝐜𝐫𝐚𝐜𝐤 𝐭𝐡𝐞 𝐢𝐧𝐭𝐞𝐫𝐯𝐢𝐞𝐰𝐬 𝐨𝐟 𝐭𝐨𝐩 𝐩𝐫𝐨𝐝𝐮𝐜𝐭 𝐛𝐚𝐬𝐞𝐝 𝐜𝐨𝐦𝐩𝐚𝐧𝐢𝐞𝐬 𝐛𝐲 𝐝𝐞𝐯𝐞𝐥𝐨𝐩𝐢𝐧𝐠 𝐚 𝐭𝐡𝐨𝐮𝐠𝐡𝐭 𝐩𝐫𝐨𝐜𝐞𝐬𝐬 𝐚𝐧𝐝 𝐚𝐧 𝐚𝐩𝐩𝐫𝐨𝐚𝐜𝐡 𝐭𝐨 𝐬𝐨𝐥𝐯𝐞 𝐚𝐧 𝐮𝐧𝐬𝐞𝐞𝐧 𝐏𝐫𝐨𝐛𝐥𝐞𝐦."

𝐇𝐞𝐫𝐞 𝐢𝐬 𝐡𝐨𝐰 𝐲𝐨𝐮 𝐜𝐚𝐧 𝐫𝐞𝐠𝐢𝐬𝐭𝐞𝐫 𝐟𝐨𝐫 𝐭𝐡𝐞 𝐏𝐫𝐨𝐠𝐫𝐚𝐦 -

30 INTERVIEWS IN 30 DAYS- BIG DATA INTERVIEW SERIES

This mock interview series is launched as a community initiative under Data Engineers Club aimed at aiding the community's growth and development

Link of Free SQL & Python series developed by me are given below -

Don't miss out - Subscribe to the channel for more such informative interviews and unlock the secrets to success in this thriving field!

Social Media Links :

TIMESTAMPS : Questions Discussed
00:00 Introduction
01:12 PySpark and Azure integration for pipelines
02:44 Analytics setup and data warehousing
04:38 Configuring Spark job
06:21 Spark optimization
08:58 Shuffling avoidance techniques
10:22 Understanding and minimizing shuffling
11:04 Initial Spark job steps for shuffling reduction
12:40 Spark job partitions
13:47 CPU cores and partition relationship
16:55 Partitioning and bucketing use cases
20:00 Hash functions and tables
23:40 Decreasing partitions
24:23 Coalesce vs. repartition
25:14 Dealing with data skewness
26:06 Partition skew solutions
26:34 Salting purpose
27:35 Scenario-based question
30:01 Narrow and wide transformation examples
31:31 Spark's lazy evaluation
32:25 RDD vs. Spark comparison
33:38 Optimizers in higher-level APIs
34:50 Out-of-memory error handling
37:24 Another scenario-based query
42:00 Job scheduling with Azure Data Factory
43:26 Coding questions

Music track: Retro by Chill Pulse
Background Music for Video (Free)

Tags
#mockinterview #bigdata #career #dataengineering #data #datascience #dataanalysis #productbasedcompanies #interviewquestions #apachespark #google #interview #faang #companies #amazon #walmart #flipkart #microsoft #azure #databricks #jobs

[jjayeshpawar]

The interviewer and interviewee both are very knowledgeable. Can we have more discussion between them? One more suggestion - interviewer can explain the possible answer after one round of discussion that will be helpful. Thanks

[BalaMurugan-]

29:17 repartition will work to reduce the skewness

[tulsimalviya]

Very knowledgeable interviewee and. Interviewer

[HemantkumarSharma-ns]

All output will be 10 rows...each 1 will go with every 1 same goes for 2

[singhjirajeev]

Assuming both tables (A and B) have a column named 'col' with the data provided (1, 1, 1, 2, 2 for A and 1, 1, 2, 2 for B), here's the count for each join type:
* Inner Join: 3
* Right Join: 5
* Left Join: 5
An inner join only keeps rows where there's a match in both tables on the join column ('col' in this case). In this example, there are three rows (1, 1, and 2) that appear in both tables.
A right join keeps all rows from the right table (B) and matches them with the left table (A) if there's a match on the join column. Here, all rows from B (including duplicates) are included since they all have a matching value in A.
A left join keeps all rows from the left table (A) and matches them with the right table (B) if there's a match on the join column. Any unmatched rows in B will have null values in the corresponding columns from A. In this case, both tables have the same number of rows (5), so the left join also results in 5 rows.