Actually, it happened just before my current holidays.
I had prepared a whole system to feed and use a machine learning model. My colleague and some others had been working on a great thing, all encapsulated, all abstracted for my system.
My last day at the office, they had it ready.
I install their thing, load one model and launch one dummy prediction: error. I try with other input data: error
I try debugging a bit more, errors all the way. Knowing them, I asked if they wrote some unit tests.
"Sure we did"
I find the tests, yes there are some. And I notice:
"Hey, I see that in all your tests, you're making more than one prediction at a time (=aka using a matrix with more than one row)
- yeah, and it work fine
- in the project, we're doing one prediction at a time, did you try it with one prediction?"
He tries: error, that was totally what I said.
I started ranting on loosing the scope of the project, why we do tests in the first place.
Then, I grabbed my coat, said "see you in one week" and let them rework their code.
I was so angry at them, it seemed so basic to just check that 👹