Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
-
So to sum up the typical process at our department:
We completely and entirely murder the idea of CI by all working on the same branch. It's even worse than having a merge day... If more than one dev is working on the same file you need to wait until the other person is done!!
We aggressively violate the idea of "continuous delivery" by having negligible and/or outright ZERO automated testing
We completely stomp out the prospect of "continuous deployment" by requiring authorization, scheduling, and then MANUAL deployment (for manager monitoring reasons?) of any code.
This will either be a very, very long road (unlikely) or management will implement the worst bastardization of "CI/CD" of all time, increase onboarding with our fucked up processes, and then pat themselves on the back as if they did something (this is what will happen)
Kinda like how they solved our DDoS problem with obfuscation and had a meeting showing off all their grand solutions... -
The process they suggest is not a bad one when there is a good reason and build properly. If you have environments automated you can have review and business acceptance on your MR/PR. You merge it when it is accepted to the dev branch. Than the Dev branch is stable and can be deployed by business whenever they want it.
Only hotfixes need to bypass this.
For us this worked because usually we would have some features ready that business wanted to ship bundled and sometimes with a notice. Also sometimes the feature needs contend and that is not ready yet. -
@Oktokolo Two people. One person requests, one person actually "puts the keys in and stars the engine"
This is after a request and approval process by the manager of the group.
It's not bad IMO and not something that has ever gotten in the way (except when there's nobody paying attention on the other end and your deployments go out 30 mins late or two deployments are out of sync when they need to go together)
...but it's definitely something that I think is a bit ridiculous to not only keep but improve if you're on a "CI/CD" kick... -
@prodigy214 I agree, but our process has too much approval, and too much manual intervention.
Even if QA and managers approved pre-deployment and then deployment was automatic, I would be happy.
But no, we have approval process by managers before it even hits the deployment schedule, then we build like 5 minutes before deployment time, then exactly at deployment time, a QA needs to be there to press the button to move.
Random QAs will move random things. So if you need to move two components, like front end and back end at the same time, you'll never get them exactly in sync and sometimes even terribly out of sync.
If one QA does both, it takes at least a few minutes for them to switch over to their second task.
If two QA does them, then they can be closer together or MUCH further apart based on how quickly they actually deploy.
It's messy and it just puts me into a ranting mood that managers what to "improve" it by keeping it fundamentally the same. -
@bioDan We get 3% raises per year and can't get anything else without getting promoted. There's like one or two promotion opportunities per year assuming you meet the requirements and you need to compete with coworkers, of which we have a significant backlog of under-positioned people (who are more skilled at the interview by virtue of doing it multiple times)
I would personally love to leave if I didn't want to move away from here soon. I don't want to start another job just to move away from it in a year or so, as soon as I get comfortable with the culture of the new job.
Oh well. -
@AlgoRythm yeah that sounds like a convoluted process. Thing the question I with all controls whether automated or by process is what does this actually protect, what does it actually prevent and does is this the way to do it. If you already do acceptance before shit is merged what is this process adding?
Unfortunately I fully agree with @fullstackchris what the fuck does an automation professional do in an organisation that is against automation. It's admirable you tried. -
@AlgoRythm: It might be overkill in most industries. But in principle, i don't see, why having a four eyes policy should collide with the technical side of CI/DI. It only means that two (instead of one) human have to confirm in the correct order before a production deployment really happens.
In your case, it sounds like you don't have too much steps, but actually miss one. You got the "pre-deployment" approval, then the aproval by QA. And that one also triggers actual deployment - which of course then is pretty asychronous. But if you add a third step, where someone finally triggers all actual deployments all at once, it all gets deployed as synchronized as possible. So you actually want more steps - not less. -
@PAKA having CI/CD without automation is too inefficient to be feasable, and you can have multiple branches as part of the CD in your CI/CD pipeline. It depends on the development methodology of the organisation
-
@AlgoRythm i gather you can start snooping around, i bet you'll be able to find a more productive place
-
@PAKA automation is the CD Part of CI/CD. I don't really see how feature branching is related at all to CI, as long as you have a branch you can commit or merge to frequently to avoid "merge day". We just merge up the tree when we finish work, and I like to have feature branches to isolate what each of our changes do in a sprint, since we don't have automated testing
-
@bioDan there is no such CI/CD without automation but having automated build doesn't mean you are doing CI/CD. Practicing CI/CD requires entire team to work in specific way for example coders need to keep their code in state that is always releasable by using feature flags, testers no longer have a say in determining releasability because that's pipelines job and so on
-
@AlgoRythm feature branch is by definition long lived branch. If you have long lived branches that are not frequently integrated into mainline then you are not doing CI. CI/CD is a way of working for entire team, not something you get by installing Jenkins. If you have no automated testing then you are not doing CI either
-
@PAKA Fair enough point. At some point if our team really needs to go through with this CI/CD thing we would probably stop feature branching
-
@PAKA if you have an automatic build and deploy to a ACR / EKS (or any other production deployment target) as part of your pipelline, then by definition you are doing CD.
Also, having automatic merges to dev / main of PRs which are based on feature branches are an acceptable or even encouraged methodology of CI. Also, not all feature branches are long lived. -
@bioDan Sorry but that is not enough to call it CD. Having automated build is not the same as practicing CI/CD. Also you are wrong about PRs, they are very hard to to do with real CI and not encouraged practice at all as you should be integrating frequently which means about every 5-10 mins.
-
Sorry, but my real world experience and understanding of the matter says otherwise. Integrating every 5 or 10 minutes is incorrect as it depends on the size of the team you're working with.
Conitnuous Deployment is opposed to fixed time constraint deployments, not about the rate of which you are CDing. -
@PAKA 5-10 minutes is ridiculous, I've heard every day is the minimum
-
@bioDan @AlgoRythm Well I hope you live to see a real CI/CD in some project. Just remember that what you are doing is not CI/CD and when it inevitably starts causing problems the fault is yours and not in DevOps methodology.
-
@PAKA thanks, same to you.
i hope you live to see real CI/CD too
So my department is "integrating CI/CD"
Right now, there's a very anti-automation culture in the deployment process, and out of our many applications, almost none have automated testing. And my groups is the only one that uses feature branching - one of the few groups that uses branching at all beyond "master, dev"
So yeah... You could see how this is already ENTIRELY fucked from the very beginning.
First thing they want to do is add better support for a process... Which goes directly against CI/CD.
The process is that to deploy to production (even after it is manually approved by manager), someone in another department needs to press a button to manually deploy. This, as far as I can tell, is for business rule reasons rather than technical ones.
They want us to improve that (the system will stay exactly the same with some streamlined options for said button pressers)
I'm absolutely astounded at the way our management wants to do something but goes in exactly the opposite direction. It's like the found an article of what CI/CD was and then took notes on exactly what not to do.
rant