35. Important Pyspark Interview Question | LTIMindTree Interview

#pyspark #pysparkinterview #interview #dataengineer

Important Pyspark Interview Question

PYSPARK INTERVIEW QUESTION
WE HAVE 2 TABLE EMPLOYEE AND DEPARTMENT TABLE IS GIVEN

1. WE HAVE TO FIND THE HIGHEST SALARY BASED ON EACH DEPARTMENT NAME
2. WE HAVE TO FIND THE EMPLOYEE WHO IS GETTING HIGHEST SALARY BASED ON EACH DEPARTMENT NAME
3. WE HAVE TO FIND THE lowest SALARY BASED ON EACH DEPARTMENT NAME
4. WE HAVE TO FIND THE EMPLOYEE WHO IS GETTING lowest SALARY BASED ON EACH DEPARTMENT NAME

SOLVE USING PYSPARK AND SPARK SQL

Want more similar videos- hit like, comment, share and subscribe

❤️Do Like, Share and Comment ❤️
❤️ Like Aim 5000 likes! ❤️

➖➖➖➖➖➖➖➖➖➖➖➖➖
DATASET

emp_data=[('Manish' , 1 , 75000),
('Raghav' , 1 , 85000 ),
('surya' , 1 , 80000 ),
('virat' , 2 , 70000),
('rohit' , 2 , 75000),
('jadeja' , 3 , 85000),
('anil' , 3 , 55000),
('sachin' , 3 , 55000),
('zahir', 4, 60000),
('bumrah' , 4 , 65000) ]
schema= ["emp_name" ,"dept_id" ,"salary"]

dept_data = [(1, 'DATA ENGINEER'),(2, 'SALES'),(3, 'SOFTWARE'),(4, 'HR')]
schema1=['dept_id','dept_name']
➖➖➖➖➖➖➖➖➖➖➖➖➖

Azure data factory :

PYSPARK PLAYLIST -

➖➖➖➖➖➖➖➖➖➖➖➖➖
📣Want to connect with me? Check out these links:📣

➖➖➖➖➖➖➖➖➖➖➖➖➖
what we have covered in this video:

➖➖➖➖➖➖➖➖➖➖➖➖➖

Hope you liked this video and learned something new :)
See you in next video, until then Bye-Bye!

➖➖➖➖➖➖➖➖➖➖➖➖➖

[Evertus_]

can you do end to end azure data engineering project please, full video

[nitinpandey]

Bro instead of filter we can use the same condition in join say as
df1.dept_name == df2.dept_name and df1.max_sal == df2.sal.

[maheswarpalagiri]

Hi, are there any upcoming videos in this pyspark playlist or all the concepts covered ?

[investmantra]

bhai aapko idea hai kya macbook mei hum sql server docker ki help se chalate right, then sql server ko azure data factory se kaise connect karenge .. coz woh chl e docker ke through hai