masudio - tech

Let’s talk about an oft-overlooked NoSQL database type. It’s got all the best parts of a k-v store and allows limited SQL-like query-ability too! They’re called document databases and they’re all the rage. A document database stores objects that can be serialized to JSON or some similar serialization format. These JSON ‘documents’ are keyed by an ID, similar to how k-v stores work. When you want to fetch an entire document, all you need is the key for it. But the magic of document databases allows you to fetch only pieces of a document and also to fetch data from multiple documents using selection criteria that mirrors basic SQL query functionality. This is all made possible by the tree-like structure that documents in a doc DB must conform to. JSON data can contain keyed fields, nested structures, and lists. Using this structure a doc DB can extract specific pieces of a doc so that the entire doc doesn’t have to be returned and parsed in the application layer. Query-ability...

Most uses of Redis will focus more on latency and availability rather than consistency - that’s because at its core, Redis is essentially a cache. Generally speaking, you store things in Redis in memory and you update or read them extremely quickly. You need to make sure that the cache is always available, so in most cases you’d only choose Redis if you’re leaning towards an A class system (A for Availability) rather than a C class system (Consistency). However, it’s important to know that a replicated instance of Redis is capable of giving you different levels of consistency up to and including read-after-write consistency - the kind of consistency that guarantees data reads from anywhere that happen after a successful response to a write will receive that write. Even if the read goes to a different replica than the write did. What Redis can’t give you is linearizability - the guarantee that any set of observers of the system will only be able to see a single copy of the syste...

When building large-scale software systems today, you have to make tradeoffs.  You can't have an ACID compliant data store with infinite storage/throughput/connections that's always available in any part of the world with super low latency where clients can read/write concurrently without any risk of inconsistencies that's free.  If you could, the problem would be solved and our industry could go build spaceships at SpaceX or retire and make sourdough every Sunday. Instead, we need to make tradeoffs.  Does our product/system need ACID semantics?  Is latency more important?  Can we allow certain types of data inconsistencies for a short time in favor of availability?  How much are we able to spend so that we don't have to sacrifice as much? These are some questions that everyone building a large-scale software system has to grapple with in the design phase.  A great way to begin your thinking is using CAP Theorem - or at least what it's slowly be...

XFN (cross-functional) work is one of the challenges of a senior engineer in most tech companies.  Broadly, it means to interact with team members of a different team than your own.  Concretely, this can mean anything from aligning goals or gathering feedback from other teams to inform your roadmap, to pair programming to flesh out the design of a new interface.  Your team wants you to make progress on the goals through XFN, but also to make the team look good (ie competent, smart, motivated) to those who are judging.  In my organization, this type of work is reserved for more seasoned engineers - although you're interacting with others on system designs, a lot of it is not the type of stuff taught in CS classes.  It's about personal interactions. When you first start talking to a member from a different team, it's important to ensure they feel that you're someone they want to work with.  You should be able to describe the system you own or are buildi...