DP-203: 21 - Azure data lake security - Access Control Lists (ACL)

Hey data engineers! RBAC is a great way to grant access to Azure data lake except a case when we would like to grant permissions at the more granular level than containers, e.g. for specific directories or files. Fortunately, there is one more method we could use to make it happen - use access control lists (ACL).

Join me in the 21st episode of my free DP-203 series where I talk about the following topics:
• What are ACLs?
• How do they work?
• How to set them up?
• How do they interact with RBAC?
• Finally, which method should we use based on two real-life use cases.

Enjoy!

▬▬▬▬▬▬ IMPORTANT LINKS ▬▬▬▬▬▬

▬▬▬▬▬▬ MEMBERSHIP ▬▬▬▬▬▬
Join this channel to get access to perks:

▬▬▬▬▬▬ CHAPTERS ▬▬▬▬▬▬
00:00 Introduction
00:47 Azure Key Vault revisited
02:18 ACLs
12:01 Demo
19:39 ACLs propagation
24:43 ACL + RBAC
27:37 Two use cases
41:31 Summary

[HamzaBray]

Thank you Tybul! Good ideas to learn to find out the best options in real cases!!
You are the best teacher!

[soumikmishra]

The whole time i was also thinking the same, why would i care so much of the security stuff 😀, and you answered it right at the end. You are the best instructor 🤠. Those 2 cases at the end were super useful to recap what we learnt. Thanks a lot!!

[KAshIfo]

Hi Tybul Thanks for this and this usecase example was great to understand which approach to use and when in real world kind of scenario.

[prabhuraghupathi]

Great series to learn about Azure Security and the end use case gives fair idea about how to select best from available services !! Thanks Tybul!!

[heljava]

Great explanations as always. Thank you!😊

[AshishStudyDE]

Wow such a good explanation. You have mastered it, which would have come with experience. I was a bit confused on the whole security part for dp203, watched your last 4 5 videos, now at least I can relate things. I am glad I found your videos which were uploaded just 1 day back.
If possible please make a summary of the last 4 videos. Keep making more content. 👍

[Ef-syqp]

Amazing explanations!! Thank you so much!

[onghuiling]

Thank you Tybul! You are the best teacher!

[michaelbyron]

great video once again, eagerly await the next one :)

[diegoalias]

Great explanation! Thanks a lot for sharing (Y) !!

[omarouaissi_sekouti]

Hey Piotr,
Thank you for this session,
You talked about secuirty to access to the datalake but you didn't talk about how to manage security stuff when we work with other storage services like : sql database, is it the same ? or you gonna talk about it in other session ? Thanks

[subhashkumar]

Hey Tybul, I wanted your opinion on one of the issues I am facing with writing a file to ADLS folder.
Below are the steps :
I have a logic app which has its managed identity on.
I granted this managed identity the ACL permission as "-wx" which stands for write and execute. I even added the default permission.
When I try to write the file to storage using the logic app it gives me an error : Authorization Permission mismatch.

So what i did was added storage blob data contributor role on the ADLS container and that worked but my question is why didnt it work with ACL??

And if ACL cant support managed identity then whats the point of having these permission. Its actually very frustrating. 😊
Looking forward for your answer.

[forsalemailid]

Great explanation - went through all your videos of Azure Data Lake Security. However, still unable to help my use case. I am trying to grant Read access to a specific blob within a folder to a User. When the user clicks the blob URL in the browser - it simply says "Resource not found". Yes I have anonymous access enabled to the storage account. If I don't, I get "Public access not permitted" error. So how can I make sure only the user who have access/ACL permission can open the URL and others can't?

[Lukkorable]

Hi Piotr, why should we propagate permissions using Storage Account Explorer instead of using Access Permissions in ACL? Initially, we grant read permissions to already existing files in the specified container. Then, we establish Default Permissions to govern any new file or directory that may be ingested in the future.

[dmitryzvorikin]

Thank you! These ACLs really look like UNIX filesystem permissions, has it been connected?

[jorgenamour]

Nice video, thanks a lot! After the configuration of ACLs, how can I get the list of folders/files that a user can access?

[TheMapleSight]

27:11 "You shall not pass!!!!" 🤣 Great episode. Btw. is security really that important to know for a Data Engineer? Aren't there teams that manage it or is it solely responsibility of the data engineer?

[bubyuguy]

hi there, thanks for the ACL details explanation.. it helped to solve my blocker, as IaM role doesn't solve folder level.

May I know what device/tools and software you are using to do the drawing in this session? IMO It's beautiful and helps alot to the viewer to understand when you visualized it, rather than just talk.
I'll get one, seems nice to use in Online Meeting.

[fekasng]

Thank you very much.. Keep it up sir. Please I want to ask you a question about data factory and synapse analytics. In real life scenario, which one is more robust in terms of cost and user friendly to use base on your experience. Both seem perform the same function even though synapse is unified.

[LATAMDataEngineer]

RBAC and ACLs can be used together to strike the right balance between granularity and ease of management. 🤝