Showing posts from December, 2019

ChatGPT - How Long Till They Realize I’m a Robot?

I tried it first on December 2nd... ...and slowly the meaning of it started to sink in. It's January 1st and as the new year begins, my future has never felt so hazy. It helps me write code. At my new company I'm writing golang, which is new for me, and one day on a whim I think "hmmm maybe ChatGPT will give me some ideas about the library I need to use." Lo-and-behold it knew the library. It wrote example code. It explained each section in just enough detail. I'm excited....It assists my users. I got a question about Dockerfiles in my teams oncall channel. "Hmmm I don't know the answer to this either"....ChatGPT did. It knew the commands to run. It knew details of how it worked. It explained it better and faster than I could have. Now I'm nervous....It writes my code for me. Now I'm hearing how great Github Copilot is - and it's built by OpenAI too...ok I guess I should give it a shot. I install it, and within minutes it'

The Curious Case of the Document Database

Let’s talk about an oft-overlooked NoSQL database type. It’s got all the best parts of a k-v store and allows limited SQL-like query-ability too! They’re called document databases and they’re all the rage. A document database stores objects that can be serialized to JSON or some similar serialization format. These JSON ‘documents’ are keyed by an ID, similar to how k-v stores work. When you want to fetch an entire document, all you need is the key for it. But the magic of document databases allows you to fetch only pieces of a document and also to fetch data from multiple documents using selection criteria that mirrors basic SQL query functionality. This is all made possible by the tree-like structure that documents in a doc DB must conform to. JSON data can contain keyed fields, nested structures, and lists. Using this structure a doc DB can extract specific pieces of a doc so that the entire doc doesn’t have to be returned and parsed in the application layer. Query-ability

Consistency in Redis

Most uses of Redis will focus more on latency and availability rather than consistency - that’s because at its core, Redis is essentially a cache. Generally speaking, you store things in Redis in memory and you update or read them extremely quickly. You need to make sure that the cache is always available, so in most cases you’d only choose Redis if you’re leaning towards an A class system (A for Availability) rather than a C class system (Consistency). However, it’s important to know that a replicated instance of Redis is capable of giving you different levels of consistency up to and including read-after-write consistency - the kind of consistency that guarantees data reads from anywhere that happen after a successful response to a write will receive that write. Even if the read goes to a different replica than the write did. What Redis can’t give you is linearizability - the guarantee that any set of observers of the system will only be able to see a single copy of the syste

CAP Theorem Explained

When building large-scale software systems today, you have to make tradeoffs.  You can't have an ACID compliant data store with infinite storage/throughput/connections that's always available in any part of the world with super low latency where clients can read/write concurrently without any risk of inconsistencies that's free.  If you could, the problem would be solved and our industry could go build spaceships at SpaceX or retire and make sourdough every Sunday. Instead, we need to make tradeoffs.  Does our product/system need ACID semantics?  Is latency more important?  Can we allow certain types of data inconsistencies for a short time in favor of availability?  How much are we able to spend so that we don't have to sacrifice as much? These are some questions that everyone building a large-scale software system has to grapple with in the design phase.  A great way to begin your thinking is using CAP Theorem - or at least what it's slowly been crystallize

XFN Development - What's it all About?

XFN (cross-functional) work is one of the challenges of a senior engineer in most tech companies.  Broadly, it means to interact with team members of a different team than your own.  Concretely, this can mean anything from aligning goals or gathering feedback from other teams to inform your roadmap, to pair programming to flesh out the design of a new interface.  Your team wants you to make progress on the goals through XFN, but also to make the team look good (ie competent, smart, motivated) to those who are judging.  In my organization, this type of work is reserved for more seasoned engineers - although you're interacting with others on system designs, a lot of it is not the type of stuff taught in CS classes.  It's about personal interactions. When you first start talking to a member from a different team, it's important to ensure they feel that you're someone they want to work with.  You should be able to describe the system you own or are building on a white