The more and more I use CICD tools, the more I see value in scripting out my deployment with shell scripts and Dockerfiles that can be run anywhere, to include within a CICD tool.
This way, the CICD tool is merely a launch point for the aforementioned deployment scripts, and its only other responsibility is injecting deployment tokens and credentials into the scripts as necessary.
Anyone else in the same boat as me?
I’d be curious to hear about projects where my approach would not work, if anyone is willing to share!
Edit: In no way does my approach to deployment reduce my appreciation for the efforts required to make a CICD pipeline happen. I’m just saying that in my experience, I don’t find most CICD platforms’ features to be necessary.
All the build logic is coded in python scripts, the jenkins file only defines the stage (with branch restrictions) and calls the respective script function.
This means it works on all machines and if we need to move away from jenkins integration with a new ci platform would require minimal effort.
You’re not advocating against CI like the meme seems to be, but rather for CI builds to be runnable on human’s machines and the results should be same/similar as in when running w/in the CI system. Which is what CI folks want anyway.
I’ve found Docker helpful when I want to use it to build binaries or use CLI tools that may not be available directly on the CICD platform. Also, Docker makes it easier to run the same code on MacOS that I ended up running on a Linux CICD server.
What would you consider to be overuse of containers?
Most of those things mentioned aren’t bona fide needs for me. Once a developer is deploying their project, they’re watching it go through the pipeline so they can quickly respond to issues and validate that everything in production looks good before they switch contexts to something else.
I see what you’re saying though, depending on what exactly is being deployed, the policies of your organization, and maybe expectations that developers are working in another context once they kick off a deployment, it could be necessary to have alerting like that. In that case it may be wise to flex some features of your CICD platform (or build a more robust script for deployment that can handle error alerting, which may or may not be worth it).
I come from game dev. We do lots of checks on the data that all kinds of people can screw up. So it’s important these situations are handled automatically with an email to the responsible person. A simple change can break the game, or someone might commit an uncompressed texture so the memory usage jumps up.
I don’t think there is a single right or wrong answer but to play devils advocate making your CI tooling lightweight orchestration for your scripts that do the majority of the work means you lose the advantages of being able to easily add in third party tools that you want to integrate with your pipeline (quality, security, testing, reporting, auditing, artefact management, alerting, etc). It becomes more complex the more pipelines you are creating while maintaining a consistent set of tooling integrations.
Real talk- I agree with this meme as truth.
The more and more I use CICD tools, the more I see value in scripting out my deployment with shell scripts and Dockerfiles that can be run anywhere, to include within a CICD tool.
This way, the CICD tool is merely a launch point for the aforementioned deployment scripts, and its only other responsibility is injecting deployment tokens and credentials into the scripts as necessary.
Anyone else in the same boat as me?
I’d be curious to hear about projects where my approach would not work, if anyone is willing to share!
Edit: In no way does my approach to deployment reduce my appreciation for the efforts required to make a CICD pipeline happen. I’m just saying that in my experience, I don’t find most CICD platforms’ features to be necessary.
This is pretty much what we do as well
All the build logic is coded in python scripts, the jenkins file only defines the stage (with branch restrictions) and calls the respective script function.
This means it works on all machines and if we need to move away from jenkins integration with a new ci platform would require minimal effort.
You’re not advocating against CI like the meme seems to be, but rather for CI builds to be runnable on human’s machines and the results should be same/similar as in when running w/in the CI system. Which is what CI folks want anyway.
Yeah, except for the Docker part
What’s wrong with Docker?
TBF, the problem isn’t Docker, it’s overused containerization
I’ve found Docker helpful when I want to use it to build binaries or use CLI tools that may not be available directly on the CICD platform. Also, Docker makes it easier to run the same code on MacOS that I ended up running on a Linux CICD server.
What would you consider to be overuse of containers?
What about related tools such as viewing artifacts such as for example total memory usage, and graphing that in the browser.
And sending emails, messages etc in case of a failure or change.
Most of those things mentioned aren’t bona fide needs for me. Once a developer is deploying their project, they’re watching it go through the pipeline so they can quickly respond to issues and validate that everything in production looks good before they switch contexts to something else.
I see what you’re saying though, depending on what exactly is being deployed, the policies of your organization, and maybe expectations that developers are working in another context once they kick off a deployment, it could be necessary to have alerting like that. In that case it may be wise to flex some features of your CICD platform (or build a more robust script for deployment that can handle error alerting, which may or may not be worth it).
I come from game dev. We do lots of checks on the data that all kinds of people can screw up. So it’s important these situations are handled automatically with an email to the responsible person. A simple change can break the game, or someone might commit an uncompressed texture so the memory usage jumps up.
Then you would probably enjoy concourse
I don’t think there is a single right or wrong answer but to play devils advocate making your CI tooling lightweight orchestration for your scripts that do the majority of the work means you lose the advantages of being able to easily add in third party tools that you want to integrate with your pipeline (quality, security, testing, reporting, auditing, artefact management, alerting, etc). It becomes more complex the more pipelines you are creating while maintaining a consistent set of tooling integrations.
Honestly, CI is only meaningful on bigger projects (more than 100 man-hours invested in total). So I most often go without.
But I do see its point.