Uber's Michelangelo vs. Netflix's Metaflow

  Uber's Michelangelo vs. Netflix's Metaflow Michelangelo Pain point Without michelangelo, each team at uber that uses ML (that’s all of them - every interaction with the ride or eats app involves ML) would need to build their own data pipelines, feature stores, training clusters, model storage, etc.  It would take each team copious amounts of time to maintain and improve their systems, and common patterns/best practices would be hard to learn.  In addition, the highest priority use cases (business critical, e.g. rider/driver matching) would themselves need to ensure they have enough compute/storage/engineering resources to operate (outages, scale peaks, etc.), which would results in organizational complexity and constant prioritization battles between managers/directors/etc. Solution Michelangelo provides a single platform that makes the most common and most business critical ML use cases simple and intuitive for builders to use, while still allowing self-serve extensibi...

Consensus: The Hard Kind

You're on a team, undoubtedly.  You have been tasked with solving a customer problem and you have a design ready and waiting for review.  One team member reviewed an early version and asked for some tweaks, but after an iteration they agreed it was the optimal path forward.  You open it up to a wider audience for further review.  But then another team member pipes up...

"This design doesn't work - it's just not possible with our current setup"
Well, damn!  What do you want me to do?  Do I need to cycle through every team member and get them to give their feedback, iterate and incorporate it and then present it to everyone?  Even if I do that, by the time I get around the group, someone will disagree with the design again!

Calm down, breathe - this is what it feels like to learn.  And what you've just experienced is what I like to call the 'Hard' version of Consensus.

Every team is different in how they deal with this consensus problem, but if the input of your teammates is important to your work, then your team will solve it with some version of what follows.

First, gain experience in the domain to gather the constraints.  You need to understand the problem domain deeply so that your first iteration of design starts somewhere reasonable.

Second, propose a reasonable solution.  With enough domain experience, you'll be able to anticipate what the feedback will be and correct for it where you can and defend it where you can't.

Third, share independently to each team member who you know will have the most critical feedback.  Record what they say and make it a dialogue - it's a conversation, not a dictation.

Fourth, update the solution to incorporate the feedback - where there are conflicting constraints, get the relevant team members in the same room (or the same IM thread, or post, or whatever medium you collaborate on) and hash it out - the conflicts will become apparent to your teammates and with the right level of trust, you'll be able to brainstorm and agree on a solution.  Repeat this step as many times as needed.

Fifth and finally, 'present' the design to a wider audience.  'Presenting' could mean an in-person team meeting, or it could mean a document sent to the team email list or group - again, it depends on the medium your team uses to collaborate.  Make sure the reasons for each decision were documented along the way - but don't explain every single one of them in the wider presentation.  It's too much info for those seeing it for the first time.  Instead, have those explanations ready when questions come along so that you can hand-hold your audience through the same discovery process you went through to come to the same conclusions.  You may still discover new issues/changes necessary in this wider meeting via feedback, but if you've done steps 1 to 4, there should be enough support in the room to help defend against any large modifications.

This series of steps is simple and straightforward on paper, but it requires assets that one can only earn through experience: patience, EQ, the trust of teammates, crucial conversations and - oh, yeah - technical know-how.  It's no wonder senior engineers come few and far-between.  But the best part of earning these skills is that they apply to so many other parts of life, unlike coding and system design.

Comments

Popular posts from this blog

ChatGPT - How Long Till They Realize I’m a Robot?

Architectural Characteristics - Transcending Requirements

Laws of Software Architecture