How to Support A/B Testing in Your Python Code

In today's video, I am taking a close look at A/B testing and feature flags. A/B tests are a great way to establish what your end user likes. If you just make those decisions based on what you think will work, you’re basically flying blind because you don’t know what the customer actually prefers.

🔖 Chapters:
0:00 Intro
1:28 Example explanation
3:44 What are A/B tests?
4:39 A/B tests and privacy
5:56 Feature flags
11:10 Feature flags and continuous integration
12:23 Growthbook
19:39 Mixpanel
25:37 Final thoughts

#arjancodes #softwaredesign #python

[ivernik]

15:27 You don't need OR here. Just use empty string as the second argument in os.getenv.

[LeonardoCarvalho]

Just great and we'll prepared content. Thanks for sharing it Arjan.
Greetings from Brazil.

[AleksanderFimreite]

I think i have been part of one of YouTube's experiments for the last 10 years or so. =D
Where they gradually increase the amount of bad suggestions I recieve day by day to see where my breaking point is.
By this point the entire suggestions tab are basicly useless, and I'm stuck manually browsing the subscriptions tab.

[loic]

Great and very interesting video! Thank you!!

[Han-veuh]

If the codebase is the same no matter features are activated or not, how do we track which version of configs a user saw? Is that done by having logging statements that print the configs?

[TheParkitny]

Great explanation, thanks for sharing

[phxees]

Where’s the hoodie from?

Also great video.

[edward]

Very useful topic, thanks for covering it. BTW, is there any approach how to structure feature flags? I had a project where we tested and rolled out many experimental features, but the code looked really ugly and hard to maintan since we had many of flags.

[rrwoodyt]

Good content, but the side camera angle shots are really distracting. Was doing that an A/B test itself?

[SkielCast]

Really nice video! I believe it may be a bit of an overkill, but I would have created a decorator with parameters, taking the name of the event as parameter and getting it trigger whenever the decorated method is called. It makes little sense here but it could be a nice addition to scale this to a larger codebase.

[AustinWitherspoon]

Love the video! Just a mild criticism - the side angles that the video occasionally cuts to are kind of jarring and out of place. It makes it feel like the video is *trying* to be cool and fancy, and is mostly distracting.

You're great at presenting! And youtube viewers are incredibly conditioned to accept the "vlog format" of somebody talking straight to the camera and using jump cuts or zooming in and out from the same angle.

If I were you, I would drop camera B entirely (which is probably easier to film anyway!)

Again, love the videos generally though. You fill a great niche for mid/senior level python topic videos and I try to watch every single one!

[erranbahadurbk]

Can you please explain how you think your channel name?? Whats it means? Your name arjan?

[DotcomL]

Side question but I think an interesting topic: how would you deliver this application to users without leaking the API key for Growthbook?

[NostraDavid]

What I don't like about A/B testing is that companies tend to end up completely ignoring user feedback. They'll just make two versions, answer "which one is used more often?" and just ram that version down our throat. Looking at you, Reddit.

Anyway, there's usually something that something has been wanting for YEARS, but companies won't do it, because no A/B testing :(

That just ends up me not wanting to use it at all, because it's so impersonal.

Not that users are always right (only "in matters of taste", as the quote goes), but those large entities end up becoming impersonal.

[deemon]

I miss your normal voice but get your testing different audio stuff.

[legion_prex]

interesting content, but i don't like A/B at all.

[cezartorescu]

Hi Arjan. Hoe gaat het ? :) At 19:22 there is no link for the rest api in the top right corner :) Very interesting and your calm way of explaining makes your content also very relaxing ;) Cheers from Romania.