How Roblox Went Down For 73 Hours

A look into what happened behind the scenes during the longest outage in Roblox history.

Sources:
blog.roblox.com/2022/01/roblox-return-to-service-1…
www.hashicorp.com/resources/how-we-used-the-hashis…
roblox.fandom.com/wiki/2021_Roblox_outage/
roblox.fandom.com/wiki/Timeline_of_Roblox_history/…
news.ycombinator.com/item?id=30013919
raft.github.io/
www.lmdb.tech/media/20130329-devox-MDB.pdf
www.lmdb.tech/doc/
db.cs.cmu.edu/mmap-cidr2022/
   • Free To Use Gameplay | Roblox | 4K | no co...  

Chapters:
0:00 Intro
0:33 HashiStack Explanation
4:47 Outage Investigation
8:20 Root Causes Found
11:30 Return to Service
12:19 Slow Leaders
15:56 Resolution

Corrections:
At 9:44, the default unbuffered channel in Go does not hold any items and has a buffer size of 0. Sends to such a channel are blocked until another goroutine is ready to receive the value. The illustration in the video shows a *buffered channel of size 1* however the overall point still stands.

Music Credits:
Firecracker by LEMMiNO (   • LEMMiNO - Firecracker (BGM)  )
Impact Prelude by Kevin MacLeod
We're Finally Landing by Home

Twitter: twitter.com/kevinfaang/
Instagram: instagram.com/kevinfaang_yt/