Happy Diwali 🪔

I failed NNN :(

!rant

After over 20 years as a Software Engineer, Architect, and Manager, I want to pass along some unsolicited advice to junior developers either because I grew through it, or I've had to deal with developers who behaved poorly:

1) Your ego will hurt you FAR more than your junior coding skills. Nobody expects you to be the best early in your career, so don't act like you are.

2) Working independently is a must. It's okay to ask questions, but ask sparingly. Remember, mid and senior level guys need to focus just as much as you do, so before interrupting them, exhaust your resources (Google, Stack Overflow, books, etc..)

3) Working code != good code. You are an author. Write your code so that it can be read. Accept criticism that may seem trivial such as renaming a variable or method. If someone is suggesting it, it's because they didn't know what it did without further investigation.

4) Ask for peer reviews and LISTEN to the critique. Even after 20+ years, I send my code to more junior developers and often get good corrections sent back. (remember the ego thing from tip #1?) Even if they have no critiques for me, sometimes they will see a technique I used and learn from that. Peer reviews are win-win-win.

5) When in doubt, do NOT BS your way out. Refer to someone who knows, or offer to get back to them. Often times, persons other than engineers will take what you said as gospel. If that later turns out to be wrong, a bunch of people will have to get involved to clean up the expectations.

6) Slow down in order to speed up. Always start a task by thinking about the very high level use cases, then slowly work through your logic to achieve that. Rushing to complete, even for senior engineers, usually means less-than-ideal code that somebody will have to maintain.

7) Write documentation, always! Even if your company doesn't take documentation seriously, other engineers will remember how well documented your code is, and they will appreciate you for it/think of you next time that sweet job opens up.

8) Good code is important, but good impressions are better. I have code that is the most embarrassing crap ever still in production to this day. People don't think of me as "that shitty developer who wrote that ugly ass code that one time a decade ago," They think of me as "that developer who was fun to work with and busted his ass." Because of that, I've never been unemployed for more than a day. It's critical to have a good network and good references.

9) Don't shy away from the unknown. It's easy to hope somebody else picks up that task that you don't understand, but you wont learn it if they do. The daunting, unknown tasks are the most rewarding to complete (and trust me, other devs will notice.)

10) Learning is up to you. I can't tell you the number of engineers I passed on hiring because their answer to what they know about PHP7 was: "Nothing. I haven't learned it yet because my current company is still using PHP5." This is YOUR craft. It's not up to your employer to keep you relevant in the job market, it's up to YOU. You don't always need to be a pro at the latest and greatest, but at least read the changelog. Stay abreast of current technology, security threats, etc...

These are just a few quick tips from my experience. Others may chime in with theirs, and some may dispute mine. I wish you all fruitful careers!

Worst dev team failure I've experienced?

One of several.

Around 2012, a team of devs were tasked to convert a ASPX service to WCF that had one responsibility, returning product data (description, price, availability, etc...simple stuff)
No complex searching, just pass the ID, you get the response.

I was the original developer of the ASPX service, which API was an XML request and returned an XML response. The 'powers-that-be' decided anything XML was evil and had to be purged from the planet. If this thought bubble popped up over your head "Wait a sec...doesn't WCF transmit everything via SOAP, which is XML?", yes, but in their minds SOAP wasn't XML. That's not the worst WTF of this story.

The team, 3 developers, 2 DBAs, network administrators, several web developers, worked on the conversion for about 9 months using the Waterfall method (3~5 months was mostly in meetings and very basic prototyping) and using a test-first approach (their own flavor of TDD). The 'go live' day was to occur at 3:00AM and mandatory that nearly the entire department be on-sight (including the department VP) and available to help troubleshoot any system issues.

3:00AM - Teams start their deployments
3:05AM - Thousands and thousands of errors from all kinds of sources (web exceptions, database exceptions, server exceptions, etc), site goes down, teams roll everything back.

3:30AM - The primary developer remembered he made a last minute change to a stored procedure parameter that hadn't been pushed to production, which caused a side-affect across several layers of their stack.

4:00AM - The developer found his bug, but the manager decided it would be better if everyone went home and get a fresh look at the problem at 8:00AM (yes, he expected everyone to be back in the office at 8:00AM).

About a month later, the team scheduled another 3:00AM deployment (VP was present again), confident that introducing mocking into their testing pipeline would fix any database related errors.

3:00AM - Team starts their deployments.
3:30AM - No major errors, things seem to be going well. High fives, cheers..manager tells everyone to head home.
3:35AM - Site crashes, like white page, no response from the servers kind of crash. Resetting IIS on the servers works, but only for around 10 minutes or so.
4:00AM - Team rolls back, manager is clearly pissed at this point, "Nobody is going fucking home until we figure this out!!"
6:00AM - Diagnostics found the WCF client was causing the server to run out of resources, with a mix of clogging up server bandwidth, and a sprinkle of N+1 scaling problem. Manager lets everyone go home, but be back in the office at 8:00AM to develop a plan so this *never* happens again.

About 2 months later, a 'real' development+integration environment (previously, any+all integration tests were on the developer's machine) and the team scheduled a 6:00AM deployment, but at a much, much smaller scale with just the 3 development team members.

Why? Because the manager 'froze' changes to the ASPX service, the web team still needed various enhancements, so they bypassed the service (not using the ASPX service at all) and wrote their own SQL scripts that hit the database directly and utilized AppFabric/Velocity caching to allow the site to scale. There were only a couple client application using the ASPX service that needed to be converted, so deploying at 6:00AM gave everyone a couple of hours before users got into the office. Service deployed, worked like a champ.

A week later the VP schedules a celebration for the successful migration to WCF. Pizza, cake, the works. The 3 team members received awards (and a envelope, which probably equaled some $$$) and the entire team received a custom Benchmade pocket knife to remember this project's success. Myself and several others just stared at each other, not knowing what to say.

Later, my manager pulls several of us into a conference room
Me: "What the hell? This is one of the biggest failures I've been apart of. We got rewarded for thousands and thousands of dollars of wasted time."
<others expressed the same and expletive sediments>
Mgr: "I know..I know...but that's the story we have to stick with. If the company realizes what a fucking mess this is, we could all be fired."
Me: "What?!! All of us?!"
Mgr: "Well, shit rolls downhill. Dept-Mgr-John is ready to fire anyone he felt could make him look bad, which is why I pulled you guys in here. The other sheep out there will go along with anything he says and more than happy to throw you under the bus. Keep your head down until this blows over. Say nothing."

"A different error message! Finally some progress!"