How to Support A/B Testing in Your Python Code

preview_player
Показать описание

In today's video, I am taking a close look at A/B testing and feature flags. A/B tests are a great way to establish what your end user likes. If you just make those decisions based on what you think will work, you’re basically flying blind because you don’t know what the customer actually prefers.

🔖 Chapters:
0:00 Intro
1:28 Example explanation
3:44 What are A/B tests?
4:39 A/B tests and privacy
5:56 Feature flags
11:10 Feature flags and continuous integration
12:23 Growthbook
19:39 Mixpanel
25:37 Final thoughts

#arjancodes #softwaredesign #python
Рекомендации по теме
Комментарии
Автор

15:27 You don't need OR here. Just use empty string as the second argument in os.getenv.

ivernik
Автор

Just great and we'll prepared content. Thanks for sharing it Arjan.
Greetings from Brazil.

LeonardoCarvalho
Автор

I think i have been part of one of YouTube's experiments for the last 10 years or so. =D
Where they gradually increase the amount of bad suggestions I recieve day by day to see where my breaking point is.
By this point the entire suggestions tab are basicly useless, and I'm stuck manually browsing the subscriptions tab.

AleksanderFimreite
Автор

Great and very interesting video! Thank you!!

loic
Автор

If the codebase is the same no matter features are activated or not, how do we track which version of configs a user saw? Is that done by having logging statements that print the configs?

Han-veuh
Автор

Great explanation, thanks for sharing

TheParkitny
Автор

Where’s the hoodie from?

Also great video.

phxees
Автор

Very useful topic, thanks for covering it. BTW, is there any approach how to structure feature flags? I had a project where we tested and rolled out many experimental features, but the code looked really ugly and hard to maintan since we had many of flags.

edward
Автор

Good content, but the side camera angle shots are really distracting. Was doing that an A/B test itself?

rrwoodyt
Автор

Really nice video! I believe it may be a bit of an overkill, but I would have created a decorator with parameters, taking the name of the event as parameter and getting it trigger whenever the decorated method is called. It makes little sense here but it could be a nice addition to scale this to a larger codebase.

SkielCast
Автор

Love the video! Just a mild criticism - the side angles that the video occasionally cuts to are kind of jarring and out of place. It makes it feel like the video is *trying* to be cool and fancy, and is mostly distracting.

You're great at presenting! And youtube viewers are incredibly conditioned to accept the "vlog format" of somebody talking straight to the camera and using jump cuts or zooming in and out from the same angle.

If I were you, I would drop camera B entirely (which is probably easier to film anyway!)

Again, love the videos generally though. You fill a great niche for mid/senior level python topic videos and I try to watch every single one!

AustinWitherspoon
Автор

Can you please explain how you think your channel name?? Whats it means? Your name arjan?

erranbahadurbk
Автор

Side question but I think an interesting topic: how would you deliver this application to users without leaking the API key for Growthbook?

DotcomL
Автор

What I don't like about A/B testing is that companies tend to end up completely ignoring user feedback. They'll just make two versions, answer "which one is used more often?" and just ram that version down our throat. Looking at you, Reddit.

Anyway, there's usually something that something has been wanting for YEARS, but companies won't do it, because no A/B testing :(

That just ends up me not wanting to use it at all, because it's so impersonal.

Not that users are always right (only "in matters of taste", as the quote goes), but those large entities end up becoming impersonal.

NostraDavid
Автор

I miss your normal voice but get your testing different audio stuff.

deemon
Автор

interesting content, but i don't like A/B at all.

legion_prex
Автор

Hi Arjan. Hoe gaat het ? :) At 19:22 there is no link for the rest api in the top right corner :) Very interesting and your calm way of explaining makes your content also very relaxing ;) Cheers from Romania.

cezartorescu
welcome to shbcf.ru