The Worst Website Launch of All Time

With a $464,000,000 budget and a timeline of three years, CMS and company attempt to build a website. Unfortunately, it does not go as planned.

Sources:

Chapters:
0:00 Intro
0:22 Part 1: Optimistic Beginnings
1:08 Part 2: The Dumpster Fire Begins
4:49 Part 3: Countdown to Launch
7:48 Part 4: The Last 40 Days
10:35 Part 5: Launch and Aftermath

Notes:
- at 0:54, this logo is fake, I couldn't find any logo for OCIIO (probably because it was short-lived and merged into CMS soon after)
- at 1:22, this merge moved technical staff from OCIIO to OIS, the Office of Information Services (now OIT, the Office of Information Technology) which is a subdivision of CMS. The bulk of the staff was put into a new CMS subdivision, CCIIO (Center for Consumer Information and Insurance Oversight), which was just OCIIO renamed and operating under CMS.

Music:
- Tensions Run High by soundridemusic
- Fine Dining by TrackTribe
- Impact Prelude by Kevin MacLeod
- Cool Vibes by Kevin MacLeod

[itsnumpty]

The fact they brought in experts after spaghetti code was written for years and they fixed it in such a short time says a lot about their planning and competence.

[timmy]

As a software engineer, I've worked on a bunch of government funded projects.

The last project was rather small, so I estimated that it would take 2 to 3 months to deliver. The project required a specific type of IOT wireless access point, which wasn't yet installed. So we scheduled a meeting with government officials, in order to discuss an optimal installation point for this new antenna. They agreed to install the antenna themselves, so we continued developing the backend of the project.

We completed the dev-work in less than two months, ahead of time. So we contacted the city officials, to share pairing codes for the antenna. They informed us that the antenna hadn't been deployed yet, so we had to wait.

We contacted them 1 month later, same story.
Again 2 months later, still no antenna.
Another three months passed, still no antenna.

We called them back after 6 months, begging to deploy those IOT devices to free up some office space. Explaining that we could active them remotely, once they installed the antenna. They agreed, we deployed the IOT devices, then everyone forgot about the project.

It took the government 2.5 years in total, to install the antenna. Which is about an equivalent amount of work, as installing an wifi AP. They then asked for a financial compensation, claiming that we delivered the project two years behind schedule.

A government funded project, in a nutshell... KEEP AWAY!!!

[napoleonVT]

"They didn't document anything, but that's pretty standard, we can give them a pass" I've never been more insulted by something I completely agree with.

[vittordecastro]

As any government project they were leaning towards the oldest platform they could find 😂😂

[lucaspepe]

"The production launch is the end-to-end testing". It gives me the same vibe as the meme of the dog that says everything is fine while sitting in a house on fire. 🤣

[evanbelcher]

This project took 3 years. I genuinely feel that if they hired a single skilled full-stack engineer, a graphic designer, and a ux designer, paid them good money for those 3 years, and gave them access to domain experts to set the requirements, they could have easily cruised to the finish line on this.

[Dalamain]

Seriously you make some of the best software documentaries, I love it!

[nielsvanderveer]

This is basically every pitfall that a project may encounter in 13 minutes. Seriously impressive to encounter all of this in one project 😂 Especially the idea that more engineers will speed up the delivery makes clear the managers had no clue about software engineering.

[semicolontransistor]

The US visa application website, also crated by CGI, is probally the worst website I ever had to deal with. It has a number of pages which you would need to complete for the applicaion, however they have an extreamly short time out of maybe 5 minutes or so before they would boot you out of the system. Although you can save your progress, saving is only possible once an entire page is completed. It was extreamly furstrating when you go dig though the draws for a document needed to complete a question near the end of a page, only to return to see that the system has timed out and anything filled in on that page is lost. It has gotten to the point that I re-entered questions for a page so-many times due to the time out that I memorized the entire page.

[Nadia]

Shoutout to the government project I worked in 2017, I've heard they're still using an IIS with php 5.6 with the vendor directory commited to the repo.

[MHX]

Your humor with the editing is amazing

[xplinux]

That was an astounding trainwreck from beginning to end! I lived in Singapore for a few years prior 2020, and I was spoiled by the quality and slick interfaces of all the government technology over there, created by their in-house SWE agency GovTech. Everything from the SingPass app (for national ID and SSO) to their tax portal was extremely solid and modern looking, rivaling most commercial apps. It's so fucking sad how far behind the US government is, when it comes to software engineering.

[sortsvane]

You should consider launching a patreon or memberships... The content quality is top notch.

[ttuurrttlle]

This is very interesting, but I feel like I still have no clue how this actually could have happened on the ground. Like yeah, there was no decisive leadership, stupid managerial practices, old technology, changing requirements... it does sound like a perfect storm for things to go badly, except that's kinda par for the course in professional software development.

It does sound like everyone managing it was at fault from contractors to government officials because of various bad decisions and the project was a nightmare to work on. It sounds like everything boiled down to terrible management, but I'm still sorta surprised about how that happened to such an important project. It's not like the federal government hasn't made working websites before...

I would have liked to see more specifically how that GUI-generated code played a part, cause I can see that being a problem. I've definitely felt restrained and exasperated by bad code I could not change.

[AndreiTache]

As someone who doesn't know anything about web dev, I find it so ridiculous how over complicated everything seems to be. There is no way a basic webpage and form colector should require 50 services built by 10 different teams

[leodler]

That Office of Consumer Information and Insurance Oversight logo just doesn't sit right with me lol

[andrewchang]

It’s funny how it took hundreds of people to build this and still fuck it up, but nowadays, this is probably a system design question for an entry level position at a tech company lol

[MCasterAnd]

You should really do a video on when Helse Midt (Norway's healthcare organization that covers the middle area of Norway) chose to change their journal system.

They started with a bidding process. All vendors welcome. Pretty early in the bidding process however they decided to disqualify DIPS, who was delivering the same journal system to Helse Sør-Øst, Helse Nord and Helse Vest - the three other govt. healthcare organizations in Norway. Yes, they really disqualified the one piece of software that was used literally everywhere else. Who did they go with? Well, good'old american EPIC - and they chose to call the new system "Helseplatformen". Did it go well? NOPE.

It has been a shitshow from day one. Most notably, 16 000 critical letters to patients were discovered to not have been delivered, which delayed a whole boatload of appointments and also put some patients lives at risk (Helseplatformen blamed this on user error - yes, really, 16 000 missing letters was the result of a user error made by several hundred users was to blame on the users, not the system). After this, the director of Helseplatformen stepped down due to the huge wave of criticism following the reveal of these issues.

One doctor actually saved a patient life after being unsure about wether or not a letter was sent. He asked a colleague to directly contact the patient to ensure he got the information he needed. The patient had a blood clot. Had the hospital not reached out to him directly, he could have died.

Currently, 44 individual cases have been confirmed where patients have not received the help they needed in time, directly as a result of Helseplatformen.

In a survey conducted among the staff of one of Helse Midt's largest hospitals, 27% of the nurses were considering quitting due to the issues.

The feature for referring patients to another hospital, which is a critical feature of any hospital journal system, was non-existent when the system was first implemented. After several months this feature was finally implemented, but it caused a massive amount of manual labour for the hospital staff - who are already understaffed.

Some hospital directors are quoted in meetings discussing the backlash from doctors and healthcare workers as "echo chambers", "hate groups" and "whining from angry doctors" - which isn't really helping the issue. The healthcare on their side are claiming that they are providing the directors with "constructive criticism" about legitimate issues with the system.

In addition, the price for the new system has increased by 35%, from 3.7 billion kroner to 5 billion kroner - and it has been implemented in less than half of the healthcare institutions in the region.

UX/UI wise it's also a complete mess. It's one of the worst pieces of software I've ever seen in modern times. If you search for Helseplatformen on google you can see their take on inputting a date/time in the system. It's a complete scandal.

Now, a ton of doctors and healthcare workers have protested against Helseplatformen - with some healthcare institutions outright refusing to change to the new system, instead opting to use the old one. In the meantime Helse Sør-Øst, Nord and Vest are looking at this and wondering why the hell they didn't go with DIPS, so they could all use the same system and be able to share journals across all hospitals...

[TheShnitzel]

Man, you're a really great storyteller and your videos are truly awesome.
I'm glad I randomly stumbled into this channel. You deserve way more recognition!
As an engineer myself I find these videos very interesting and valuable. Keep 'em coming!

[rebelcat_]

"The production launch is the end-to-end testing." I will now be incorporating this philosophy into all of my future endeavors.