How to build a robust Payments service?

Build Your Own Redis / DNS / BitTorrent / SQLite - with CodeCrafters.

In the video, I explained the concept of idempotence in APIs, emphasizing its importance in ensuring consistent outcomes even with multiple executions. I highlighted scenarios like banking transactions and social media interactions to illustrate the significance of idempotence. I discussed the need for unique identifiers to maintain transaction integrity and prevent duplicate processing. Implementing idempotence involves strategic design choices to avoid unnecessary retries and potential errors. By incorporating common identifiers across services, robust and reliable APIs can be developed.

# Recommended videos and playlists

If you liked this video, you will find the following videos and playlists helpful

# Things you will find amusing

# Other socials

I keep writing and sharing my practical experience and learnings every day, so if you resonate then follow along. I keep it no fluff.

Thank you for watching and supporting! it means a ton.

I am on a mission to bring out the best engineering stories from around the world and make you all fall in
love with engineering. If you resonate with this then follow along, I always keep it no-fluff.

[YashHalgaonkar]

I work in a team at Amazon that is responsible for paying employees and I can testify that this is exactly how we have designed our APIs.

[gomzysharma]

I work at American Express, and yeah we use exactly same thing at backEnd in various flows and microservices, we call it as batch schedular jobs and re-trigger flows for retry and check mechanism, It helps to re-trigger(with definite counts only) the NOT-PROCESSED or Failure can be due to not-so mandatory checks or may be due to multiple reasons or system downtime/failure.

[utsavprabhakar]

Second approach is pretty awesome. Started following you recently, quality content !

[DaveLegend-go]

I love the fact that you make reference to real life applications and examples

[shayanhati]

I think the Approach 1 to not retry is not full proof - even if you don't retry at the client end there can still be network retries which might mean that the request was still send to the server.
For e.g the use-case of instagram like, if some error is thrown due to network or timeout and you don't retry from client it might have still received in the backend, so you would need to make the API idempotent in any case. So the next time the user click, the backend has to implement some kind of idempotence based on the userId - postId

[bharathmshetty]

So if a single entity is supposed to be updated, you have to make sure there is a transactions/an external record that keeps the status check? For example : Say you want to a update balance of a particular user, to make it idempotent you first create a get transaction record, update balance, update transaction status.What would happen if ther transaction status update fails? Is there way you could do this without making a network call say for example a high frequency system?

[romokdas]

What happens when a second request comes from the frontend for the same transaction even before the creation of the unique id in the payment gateway service?

[harshal]

You're doing great, excellent content

[NikunjGupta]

Awesome content 😎, I just have one query, second approach will work if retries happen from backend side, what if client (user) retries payment from front end, how does payment service know which payment id to use ? Or payment service will pass this payment id to front end in first call. I wanted to understand this flow

[barebears]

What language/stack would u recommend to someone new for backend development?

[nihalkashyap]

Can't the payment service itself generate the payment id instead of requesting the payment gateway to generate one? This would save one network call.

[crimsoncad]

I have a doubt with Approach 1 "Do not retry"
What if the payment is processed in the backend but we don't receive a response and it gets timed out.
In that case, we will show and error on frontend and there is a possibility that the user will try again after seeing the error, this will cause Double Payment.

[kumarsaurav]

Sql DB should be sufficient for managing the transaction Id for huge number of requests(like 1 M trans every minute) with Sharding?

[msddhoni]

Great content. Expecting more on the PG domain...

[AtiqurRahmanatiqwhiz]

I think first solution cannot be avoidedalways, Early in my career I had same use cases, where a user could made two request for some data inputs i.e say slow browser/network etc and we used make two entry in the db as we were not checking the already possible entered data.

[dinakaranonline]

great video. can you probably expand more on how the unique payment id can be generated and shared between front end / consumer and back end services?

also how to avoid / handle potential duplicate requests coming from front end / consumers ? there can be a scenario where front end / consumer is sending two different payment id for the same transaction - how can that be avoided ? or is the onus is on front end/ consumer to ensure correct unique payment id is shared all times ? If so, how can that be managed ?

[rjarora]

You still didn't explain how to Implement Idempotence in your own service. You just explained how to use idempotence feature provided by payment gateways