FUCK DIAGRAMS

Had some interviews with some companies around. At this point, even though I’ve been a Machine Learning Engineer for 7+ years now. I’m convinced that 90% of companies that claim to be “AI-Powered” just means that they use python for their backend.

Man I have no idea how my company is running as stable as they do. Every time I peak under the curtain of some piece of machinery I find such bad practices…

Just found out our in house database manager only supports listing all objects in a table, updating objects by first reading each row you need to update and only support “select *” queries.

This is after having to argue with some engineers that using http or grpc when interacting with the new service I’m writing in the none-jvm language is better than writing our own driver for their custom rpc and service discovery system.

But like honestly I’d be mad if these decisions had a visible performance impact on the business, but it somehow doesn’t… this is bizzaro world where all I learned from my 8 years experience as a professional goes out the window…

Airflow… airflow… I hate you so freaking much, you are a bloated piece of software that packages wayyy too much and increase the complexity of any solution we built on top of you. Plus unit-testing and integration tests are wayyy too difficult with you. But man… your recent UI changes are a massive welcome.

A year ago I was tasked with either upgrading airflow to version 2 or to migrate the code to another tool. Naively I thought “well might as well upgrade to avoid a rewrite”. Little did I know that the reason airflow didn’t scale out well for us, was due to people over the years not having a grasp on airflow primitives like “pools”, “workers” and “operators”. Ended up refactoring the entire codebase for both infra and DAGs anyhow AND upgrading the beast AND lower cost by a factor of 2 (from $100-$150 daily to $50-$70 daily).

But seriously feels like I could’ve solved the scheduling issue with literally any message queue+decent library (like celery or Faust) and I’d have half the headache.

Shortumation is a better way to develop and review your automations in home assistant without needing to install a third party automation engine like node-red.

My kid is hacking the mainframe xD

C=consultant
M=Me
D=my Dumb boss

M: so how are you guys planning to implement the block all accounts feature?
C: oh it should be easy! We will just loop over every account and lock it!
M: what about implementing a flag that just blocks anyone from accessing the site till further notice?
C: what? I’m sure it’ll work. Just need a list of all accounts, we don’t need anything fancy!
M: what happens when we want to revert back to the pre-block state?
C: oh, so we will just unblock everybody
M: even people who were previously blocked for good reasons?
C: i guess so, unless you think otherwise
M: we r….
D: listen! We just need to be able to block all accounts, who cares about this details! So long as we block all accounts! We need this nuclear option in case something bad happens…
M: but what about when that bad thing passes and…
D: when it passes it passes who cares!

Arghhh so much rage here… like first at the stupid engineering design of looping over all of the accounts instead of using a simple flag. Like 1 http call (from one microservice to another) is a lot better than O(n)… not to mention, we won’t have to deal with failures and retries.

And second for my boss being a dumbass… ok you deal with being to afraid to unblock people after we used this “genius nuclear option”!

Need my vaccine receipt to go to a wedding. And I’m in line to get the pdf document. On the one hand this is some cool implementation for a load balancer, on the other hand this so strange… be online in a line to get a document -.0

I feel like the developers of this helpdesk tool are testing their jokes on prod lol

Just posting my stress ball!

Wife wanted me to make sure that we can watch the entire season 12 and 13 of degrassi while in the hospital (which has no good internet connection), but there is no good torrent or anything that lets you get it downloaded locally.

So I wrote a super basic script to scrap the season from YouTube and download it. Now just having fun looking at it.

Getting tired of old business types managing software projects... I really don’t care if you have fifty years of experience in managing loans in banks, please don’t try to manage software or data projects... this is sooo out of your realm of understanding.

PS I actually like this person, they are a wonderful human, but keep nosing in areas they have no knowledge, and derail the conversation from talking about things we actually need to discuss. Like “how do we build this?”

Some of these things are not like the others. One of these people is a tv scientist not an actual data engineer or data scientist, while another is an activist and while is extremely respected, has no room in a data+ai talk -.-

Waiting on a baby girl in a few weeks from today! Excited! Scared!
Any other future/current dads with babies with tips on how to handle a work from home with a baby?

Me: why are we paying for OCR when the API offers both json and pdf format for the data?
Manager: because we need to have the data in a PDF format for reporting to this 3rd party
Me: sure, but can we not just request both json and PDF from the vendor (it’s the same data). send the json for the automated workflow (save time, money and get better accuracy) and send the PDF to the 3rd party?
Manager: we made a commercial decision to use PDF, so we will use PDF as the format.
Me: but ...

It feels so good actually doing something that saves the company a buttload of money.

Just optimized some ML models. Now it costs a tenth of the money to run them :P

Just came across this sentence in an oped... some strange times we live in. I still remember when I was getting into tech how everyone was afraid of Microsoft... but to be honest google & apple are doing more EEE than Microsoft so...

The 3 cardinal rules of software:

1. Shit gets old
2. Shit gets better
3. Shit will not work

I’m trying to update a job posting so that it’s not complete BS and deters juniors from applying... but honestly this is so tough... no wonder these posting get so much bs in them...

Maybe devRant community can help be tackle this conundrum.

I am looking for a junior ml engineer. Basically somebody I can offload a bunch of easy menial tasks like “helping data scientists debug their docker containers”, “integrating with 3rd party REST APIs some of our models for governance”, “extend/debug our ci”, “write some preprocessing functions for raw data”. I’m not expecting the person to know any of the tech we are using, but they should at least be competent enough to google what “docker is” or how GitHub actions work. I’ll be reviewing their work anyhow. Also the person should be able to speak to data scientists on topics relating to accuracy metrics and mode inputs/outputs (not so much the deep-end of how the models work).

In my opinion i need either a “mathy person who loves to code” (like me) or a “techy person who’s interested in data science”.

What do you think is a reasonable request for credentials/experience?

Anyone else excited for the purism and pine phones to hit the main market? Wanna hear what the devRant community is thinking of this?

A really fun story to read. Makes me think of my own work shenanigans. Good to know Uber is just as fallible!

https://mobile.twitter.com/stantwin...

What are people unfiltered thoughts on apples new ARM processor? Especially if you are a Mac user?

(I'm an OS hopper, though my main machine is running ubuntu right now, and work machines are usually mac with windows with wsl)

This is not a happy rant...

Got a new well paying job. Moving up in life. But my wife is not in the same place... She had quite a few career struggles and just lost her current job... It wasn't even a job she went to school for, just whatever she could find...

Now she's getting quite depressed. Luckily she's not envious at me, but does keep saying how lucky she is to have me.

I really want to help her somehow, but this is really a thing I just don't know how... And it just looks like she's not handling it too well. Joking about suicide and crying about being useless... She also keeps saying that all she can do now is be a housewife. We did seek out help for her. But still... I really want to give her better support. I feel useless here.

Got the best compliment on an interview :3

Submitted a coding take home assignment. Was told by the tech lead that the code looked like they wrote it themselves! There was nothing they would change in terms of style or approach :3

DevRant needs a save for later option... I see lots of long interesting posts, I start reading them and then something comes up (like a phone call or I need to do something). Then by the time I come back the post is gone, and I’m sad...

That moment when google ranks gmail with a low speed score... oofff...

https://developers.google.com/speed...

This is so true!

Too many companies in the past years have been focusing on moving rendering to the client, and not optimizing their webpages that we lost sight of making efficient and fast websites.

So today I told my boss “nodejs without typescript is like sex with strangers without a condom. Sure it’s fun and all... but then you get an std”

A big project in my company. Had some annoying race condition that caused data to get deleted when two processes finished in the wrong order they hit the dB and override each other’s work.

Long story short. Fixed the bug and in the process the codebase shrunk by 60%. I didn’t have to delete the rest of the code, but the bug was due to a function in the legacy section of the code, and found out that it was the only function used in that section.

So I deleted it. Rewrote the function so it upserts. And bam. Smaller, cleaner code :)

Workstation for last few hours. Just sitting here chilling while waiting for my car to get serviced.