Prime Video Swaps Microservices for Monolith: 90% Cost Reduction

Prime video engineering team has posted a blog detailing how they moved their live stream monitoring service from microservices to a monolith reducing their cost by 90%, let us discuss this

0:00 Intro
2:00 Overview
10:35 Distributed System Overhead
21:30 From Microservices to Monolith
29:00 Scaling the Monolith
32:30 Takeaways

Fundamentals of Backend Engineering Design patterns udemy course (link redirects to udemy with coupon)

Fundamentals of Networking for Effective Backends udemy course (link redirects to udemy with coupon)

Fundamentals of Database Engineering udemy course (link redirects to udemy with coupon)

Follow me on Medium

Introduction to NGINX (link redirects to udemy with coupon)

Python on the Backend (link redirects to udemy with coupon)

Become a Member on YouTube

Buy me a coffee if you liked this

Arabic Software Engineering Channel

🔥 Members Only Content


🏭 Backend Engineering Videos in Order

💾 Database Engineering Videos

🎙️Listen to the Backend Engineering Podcast

Gears and tools used on the Channel (affiliates)

🖼️ Slides and Thumbnail Design
Canva


Stay Awesome,
Hussein

[mahdikarimi]

Timeless concept in software engineering: there's always a tradeoff. there's no silver bullet.

[joshuaifara]

They refer to AWS as another product because Prime video and AWS are actually two different companies, although they are bought owned by Amazon. I believe Prime video still pays AWS for using its cloud services, which makes Prime Video an AWS user as was stated in some parts of the text

[franciscokloganb]

Having worked with both there are definitely pros and cons. However, doing microservices correctly is very very difficult and most people tend to prefer microservice and serverless because it’s cool and fresh. At the end of the day monoliths should be a great default until there are proofs that your business requires something else.

[neal]

I think a more accurate description would be they moved from serverless distributed asynchronous nanoservices to monitor streaming data, to a synchronous microservice which is obviously more performant for real time video and data analytics.

This doesn't really have anything to do with a "monolith", just seems like they broke up the service too fine grained and saw the problems of that. This video streaming quality monitoring service is just one service in the bigger prime video system.

ECS tasks are designed for microservices, same idea as a pod in Kubernetes, can be scaled up based on rules.

Seems like a lot of people are caught up in the definitions of terms rather than choosing the right technologies and patterns to solve a particular problem.

[patrikplecho]

the payment part. Amazon is internally a customer of aws and each department in amazon is billed by aws separetely. Thats why they write about user paying

[sevurueva]

This wasnt a microservice vs monolith issue at all.

Read the Amazon article. The issue was more with choosing a serverless functions approach in a computationally heavy image processing task. Not much to do with it being a microservice.

Their two main issue was:
1. Having to pass large image files between functions. Higher S3 cost
2. Hitting serverless platform limits due to frequent statechanges.

Most of this issue could have gone away with properly partitioning the service and deploying in a platform that could handle intensive loads.

Both their issues made it clear it was a mistake for going with a serverless approach for a production workload. Might have been fine for a testing or smaller need.

[amr]

17:15 AWS Step function is actually different from the AWS lambda. it's like a workflow orchestration tool / state machine. I only know this because I was also initially confused by step function and thought it was the same as a normal lambda

[MartinezVM]

I don't understand why people call this a monolith. Thing is still decoupled to containers that are also fault tolerant, but just because the s#i7 is running in a solely EC2 instance per AZ or region (which is not specified but is has to) they dare to call it "monolith". Sometimes I believe this kind of articles are just marketing drived.

[SushritPasupuleti]

Serverless was always solid performance and cost wise when you had not so constant loads. A video streaming platform will almost always see traffic, and almost every out going stream is significant (not short lived connection/load) so it made sense for them to switch. Excited to check out the video now, because when I read the blog post last month, it left out a lot of details.

Thanks for the great content as always :D

[LimitedWard]

Maybe I'm getting lost in the terminology, but IMO "serverless" != "microservices". This is still a microservice in the sense that it completes a singular task within a larger ecosystem of services. I'm not sure where the rule came in that microservices needed to be serverless by definition.

[pemessh]

"There is a lot of missing information"
- Every AWS doc ever and Hussein.

[LuisPaolini]

I think that there are 2 things here that were not mentioned:
1) they seem to have realized that they didn't need to make this async (in the microservice approach they were using S3 + step functions as a queue
2) they seem to have realized that everything could fit into RAM, and they didn't need S3

[studiousguy]

*From the Amazon perspective, here are the missing links if anyone cares:*
1) AWS is a different company which has the rest of Amazon as a customer. Hence, they use the term "AWS user" in some places in the beginning. Prime Video is an AWS user and pays AWS for it, even though both have the same parent company.
2) AWS Step Function is a serverless workflow orchestration service. What's shown here are different lambdas which are probably the states of one Step Function workflow. State transition is the flow of control from one state to another. AWS doesn't charge users for writing a workflow with states, only charges them when actual state transitions happen in the workflow. "State transition" discussed at 16:45.
3) Entry point lambda sends start conversion request to media conversion service (discussed at 14:15) but doesn't get a response because it is not important. AWS Step Functions manage workflows behind the scenes and workflow requests/responses are not important in all transitions.
4) They show conversion service storing data to s3 (in the same diagram) and lambda doesn't call lambda directly probably because (I think) it is a known "serverless anti-pattern" if you call a lambda from another lambda. Google it. Discussed at 20:08. Also you can't call a lambda function with a media file as input, you have to upload to S3 and pass the bucket key (s3 url) to the lambda function.

[Yorgarazgreece]

If a microservice talks regularly to another microservice, it's probably a bad microservice.

Microservices should avoid "talking" to each other as much as possible. This is one of the reasons behind the concept of event driven microservices and event sourcing

[vitorhugodecastrosil]

The usage of s3 are so odd in the first service.
The article is not well written, is so confused. Your explanation as always, very good.

[boot-strapper]

its almost like putting a network call between every function call is a bad idea. SHOCKER.

[Vivek_____]

IMO, the monolith system should follow a Producer/consumer model with in-memory message queue (for now) in between. Media converter can be the producer, and starts converting and pushing work in the queue, and Detectors could be the consumer threads pulling from the queue. Not sure why there is a need for an orchestrator. In future, the in memory message queue could be replaced by some thing like a Redis based message queue to scale media converters and predictors separately.

[chris-pee]

Small correction, around 29th minute, what you call load-balancing actually isn't. Both ECS services are called every time, because all detectors need to run. They're just in different clusters to distribute the resource usage.

[mohamedabdelmaksod]

Hussein i believe the customer they pointed to in their article is not the content viewer but it is the customer who streams live events like sports games and can benefit from the monitoring tool provided by prime video to enhance their streaming process that changes a lot IMO with respect to reading the article from this point of view.

[vignansphone]

My 2 cents. AWS step functions is their orchestration tool (lambda is serverless product) . We use orchestration when we need a central party controlling the order to execute a flow like A-B-C-D( called as workflow in orchestration) where A, B, C, D are different network components (called as tasks/activities in orchestration). Every transfer from A to B and B to C is a state transitions and orchestration systems like step functions and temporal charge the customer(prime video team here) by state transitions - usually for any business flow they will quickly go up.