Uber's Michelangelo vs. Netflix's Metaflow

  Uber's Michelangelo vs. Netflix's Metaflow Michelangelo Pain point Without michelangelo, each team at uber that uses ML (that’s all of them - every interaction with the ride or eats app involves ML) would need to build their own data pipelines, feature stores, training clusters, model storage, etc.  It would take each team copious amounts of time to maintain and improve their systems, and common patterns/best practices would be hard to learn.  In addition, the highest priority use cases (business critical, e.g. rider/driver matching) would themselves need to ensure they have enough compute/storage/engineering resources to operate (outages, scale peaks, etc.), which would results in organizational complexity and constant prioritization battles between managers/directors/etc. Solution Michelangelo provides a single platform that makes the most common and most business critical ML use cases simple and intuitive for builders to use, while still allowing self-serve extensibi...

Don't NOT Repeat Yourself!

Sometimes duplicate code is good!!



...what?  What do you mean?  You're 'on to me'? .... Ok ok ok hold on, just hear me out!

Picture it: you're writing unit tests.  Headphones on, hoody blowing in the cold wind from your AC unit in your dark apartment.



You've written all your tests, they're passing and you're feeling great.  You refactor.  The tests have a lot of duplicate setup code, so having the DRY sense of humor you've got, you mop up.  Get it all lookin' fine and tidy.  Common methods for all the setup, some parameters to handle the different configurations of the unit tests. Freshhhhhhhhhhhh :D


You push your code to test env - it breaks - OH SHEEIT.  You missed a few edge cases.



No prob, no prob! Just add a few unit tests, slip in an if-else here and there in your production code, and you can get your changes in and make it to office before the 2pm happy hour! Nice - easy peasy.

But WAIT.  The edge cases you missed require additional configuration, so you have to add some parameters to the test utility methods - and some of it's really out of place, so actually you gotta break up the utility methods to add flexibility...which requires changing each of the tests to accommodate that...and you're writing C++ so there's no reliable refactoring plugins for this exact thing....ugh.....you consider Ctrl+Z-ing back to the beginning of time.



It's not to say that refactoring and DRYing up your code is bad - we all know it's the best thing since sliced bread! But you have to time it right.  For brand new pieces of code (You're extending and not modifying RIGHT?!), you need your code to be stable enough so you have a clear enough idea what the different things it needs to do are - THEN you refactor, DRYing it up in the process.

Some people would say it's better to accept the cost of re-refactoring your already-de-duped code - that's the correct process they'd say - you're supposed to refactor as things change and requirements are added/uncovered.  But in reality, you code and check-in in chunks - and you really should be finishing one whole chunk to the point that it's stable and well-tested before you put on your refactoring hat.  You want to make it to happy hour don't you?

Comments

Post a Comment

Popular posts from this blog

ChatGPT - How Long Till They Realize I’m a Robot?

Architectural Characteristics - Transcending Requirements

Laws of Software Architecture