masudio - tech

A robust distributed system can tolerate partial failures in the system - that means that the system should continue to work as expected even if parts of it are failing.  There are 3 main 'partial failures' that the field of Distributed Systems tries to solve for: 1.) Out-of-sync system clocks 2.) Process pauses 3.) RPC requests with no response Paxos is one algorithm that tries to solve for these by using consensus among nodes to decide on the sequence and order of events, but system administrators must tune the timeout parameter to find the best middle ground between waiting too long for a node (one computer in the network of systems) and not waiting long enough. The blockchain algorithm gets around the idea of time entirely by deciding on an order of events once, and then getting as many nodes as possible to agree and persist the actual ordering itself.  Here's an excerpt straight from Satoshi's paper itself: The solution we propose begins with a timestamp...

I recently switched teams at Facebook.  I went from working on user-facing search technologies to running the distributed systems that allow other teams at my company to move fast.  To be clear, I used to bake the break, now I build the bakers an oven!  Although I had some background in these systems, things change quickly and the field is deep with decades of knowledge to pull from and build upon.  I needed a refresher! I found one - a friend recommended a book called 'Designing Data Intensive Applications'.  It sounded a bit off-topic - the title didn't have 'distributed' or 'systems' in it!  Turns out it gave me exactly what I needed.  DDIA is the best introduction and deep dive into all the areas of expertise that the DS specialty has to offer - and wow is this field deep!  The book requires a bit of background - you'll need to have a bachelor's in CPSC or similar experience, as the first few chapters build on that knowledge.  Once the ...