Deep Dive Into the Repository Design Pattern in Python

In this video, I’ll take a closer look at the repository design pattern in Python. This is a very useful pattern that allows you to keep your data storage separate from your data operations.

🔖 Chapters:
0:00 Intro
0:37 Repository code example
4:13 About the pattern
8:04 Better software testing
9:36 Warnings and Caveats
11:24 Final thoughts

#arjancodes #softwaredesign #python

[yurykliachko]

this is a good pattern when your entity (Post in this case) is stored partially in different storages (sql DB + cloud storage, sql db + nosql DB etc), it hides all the complexity. Thank you for this guide!

[notead]

Hey Arjan,

I just want to say thank you.
I was able to land a job in data engineering thanks to your course and your videos on design patterns. Seeing your approach to building applications finally made it click for me that learning a language is the "easy" part, and that understanding _how to think about systems_ not only makes me a better developer - but is a super important, generalizable skill that goes beyond just programming. Maybe that's obvious for many, but I am really grateful for that insight.

[FernandoCordeiroDr]

I had to use this pattern recently. I was working on a Django app that had to work both with MongoDB and Postgres' PGVector. I created a repository for each and then a factory function that, based on environmental variables. determines which repo to be used. These repos are then used inside the methods of normal Django models. The main benefit is that adding an integration to another vector database is just a matter of creating a new repository.

[adjbutler]

I love your pattern videos! (I will even allow you to make up your own patterns)
Or do PART 2, 3, 4 on previous patterns! Your videos are amazing! Thank you

[prinsniels]

I use the pattern a lot, but in a more general way. I tend to write things on the base of interfaces, combining it with dependency injection makes things easy to test and allows for composable programs and great flexibility.

I tend to stay away from ORMs, for me they add an extra layer of complexity to programs and in analitics it quickly ends in writing straight SQL to your ORM, so cutting the middle man seems wise then 😅

Thnxs for the video!

[markasiala]

I also have used this pattern without knowing it simply by focusing on decoupling and dependency injection.

I have an abstract data class and an abstract FileIO class. Gives me flexibility on how I load data into the class or write it out. This helps me track changes in the data when I compare versions of the output data (e.g., I read in data from a user-friendly Excel file but write it out to pipe-delimited text, JSON, or YAML output where a simple diff tells me what changed).

[dadoo]

Thank you Arjan. I use this pattern in fastapi. Layer endpoint > layer services (logic etc) > layer repository with FastApi dependencies between these layers. I like it. Effectively, often I need more than simple CRUD operation and add it to my repository layer. It's not a good idea I think. Maybe we should create theses different method in services. But I like these pattern.
WHen I need to call external api, I create a repository also for that. For me repo = access to data

[axeldelsol]

This pattern is also very useful when you are wrapping a API offering CRUD routes for resources

Great video !

[mhl]

I always use this kind of Repository, but I didn't know, that I follow a pattern 😀. Thank you.

[SeliverstovMusic]

I use repository on top sqlalchemy. A have a base repo class with CRUD function. For every table, I create a new subclass, and all CRUD operation become available for the table. Magic =)

[ajflorido]

using this pattern with SqlAlchemy you can load different models dinamically and use the same repo to get the data for different db engines. For example we have models for postgres, oracle and mysql that with SA some columns definitions for the model are quite different and we load the correct model dinamically within the repo itself, so you can also decouple this pattern into another step for different engines.
Thanks Arjan!

[bachkhoahuynh]

The repository pattern is not about switching from SQL to noSQL. We call this switching effect the persistent ignorance principle. the main thing to consider to use the repository pattern is that you want to decouple domain logic from infrastructure logic. A repository is usually backed by an ORM because when you use raw SQL, you eventually implement some ORM's features such as changes tracking, proxy for lazy loading, ... I only use raw SQL for complex queries.

[demolazer]

This is awesome! I'm going to refactor my FastAPI crud operations for mysql with this pattern.

[sandeshgowdru]

Thanks a lot for making videos, I was looking for a architecture for getting data from multiple sources, I was looking into a combination of factory, strategy etc, But this pattern is perfect for my need

Once again thanks a lot for sharing this....

[dalenmainerman]

I actually used this one unintentionally
At the earliest stages of a project, all data was stored in a bunch of csv files (not my idea, not my decision)
Implementing all data-related operations with this pattern allowed me to migrate to the real database almost effortlessly

[rohailtaimourInc]

Hi @arjancodes, I’ve been really enjoying your videos and specifically how you always focus on how to test the code you demonstrate. Thank you for your content.

I was wondering if you can cover testing functions that are decorated? They pose an interesting challenge and I didn’t find it to be straightforward to test such use cases

[obsidiansiriusblackheart]

I find like most patterns, I have used this before but didn't know the name. Thanks for this awesome video! Your channel really helps me better understand coding and jargon in the field (I have ~10 years coding xp and 6/7 years work xp)

[rrwoodyt]

I like the separation and abstraction. It would have been interesting to see you make a class that could handle a generic dataclass, but that's beyond the scope of what you were trying to show. Maybe next time...

[devilslide]

I particularly appreciate the ease of mocking this repository. It's very convenient for testing the logic of services that utilize the repository class.

[muzafferckay]

Implementing repository pattern is not about switching from sql to nosql or vice verce. It decouple the business logic from the persistent layer this can be orm or sql language. As you mentioned implementing repository pattern limit querying, updating ... It is too hard to provide all feature that orm does. For example you have to define comparison operators such as in, grater than, less than etc. Your get method shold take set of relational fields to fetch as it is going to be used in different places. You have to define or, and and more complex query. Basically you have to define your own query language step by step as you need. And translating your query to Orms.

Otherwise you have to define too many different get methods for querying