Google Accidentally Deletes $135 Billion Pension Fund, Chaos Ensues

In May 2024, Google accidentally deleted the private cloud of a multi-billion dollar fund in Australia, causing an unprecedented outage.

Chapters:
0:00 - Intro
0:25 - UniSuper Use Case
1:56 - Google VMWare Engine
3:03 - The Outage
6:24 - Sus Google Tool
7:23 - Backups
8:18 - Backup Song

Notes:
2. The specifics behind why UniSuper needed Google to manually create their GCVE cloud is not publicly known. The RAM limit in the video serves as an example.

Sources:

Music Credits:

Thriller Trailer Teaser Tense by Cold Cinema

[Lustanda]

“Who TF have multi cloud back up?”
“The guy that saved this company”

[ImTom]

Was so shocked that the company followed the 3-2-1 rule.

[kaseyboles]

Never Ever forget that "the cloud" is just a fancy name for someone else's computer and you're their mercy.

[demonslime]

whoever at unisuper pushed for double backups need an 8 figure check rn

[aenguswright]

Nice to know that a company was actually saved from this situation because they had actually followed best practices about IT... sadly rare

[xFlRSTx]

at least they had sensible defaults like deleting everything with no warning after 1 year

[mista_jamaican]

Typically we Australians don't exclaim "Crikey" during a major tech outage. We use more traditional technical language including words like "shit" and "fuck".

[glitchy_weasel]

The outro slaps harder than the Crowdstrike outage

[CrimeBeanus]

why reverse a linked list when you can reverse a lifetimes work

[Monotof]

To be fair i can easily see how this happens. Its unfortunately very common for engineers to build some tooling for testing or debugging in narrow engineering use cases. Then sales or another department comes along with a problem they need fixed WiTh tHe HigHeSt pRioRiTy and "borrow" the tool "just for this once". A year later the entire department ist misusing the tool or even some bastardized version of it for production use on actual customer data. No one fully understands what the tool does or what it was even meant to do, but it saves some time so everyone just uses it without a second thought.

[nickkyg]

Outro had no business being as fire as it was😂😂

[NomenNescio]

I am actually impressed that they managed to get back up and running in two weeks time after having their entire environment deleted, that's actually a great job by the unisuper it guys, hats off, kudos and respect for a job well done.

[dannyvoid]

We all know how this would have been resolved had the deletion been to something worth substantially less than $135B

[OtterSeaborne]

The thing about Australians is that black swans are just regular swans here.

Jokes aside, I'm not even surprised that UniSuper had that many redundant backups. Universities take their pension money seriously.

[lieutenantbites]

I love the redditor commenting "I'm a cloud engineer, it's 100% impossible to be google's fault" and was entirely incorrect. Classic reddit behavior

[BoloH.]

"As a cloud engineer myself, I can confidently say this is complete bullshit, it is 100% UniSuper's fault"

[somekindofhmm]

I thought this was a good video until 8:18 and then it became a great video. Then the chorus kicked in and it became the best video of all time

[uhhhwhatdoipick]

that outro is fire
kevin's kicking off his music career :O

[fang-penlin]

Can't wait for your CrowdStrike episode, it's gonna to be awesome 🤣

[Real_Halo_MC]

Return of the king

Can't wait for the future cloudstrike video