Continuous Integration & Delivery main ideas

In the process, developers often have to update their services and deploy them. When the team and the number of services are small, this is not a problem because releases and deployments are rare. Tests, release management, publishing artifacts, and deployments can be run manually. But over time, the number of services and tasks increases, the cognitive load increases even more, and the release cycle starts to fail if you don't do it often and get bogged down in running its individual steps.

Let's look at the typical process of feature implementation/bug fixing for the majority of projects:

• Create a new branch from git;
• Implement feature, write unit tests to cover new functionality;
• Run tests, static code analysis, etc;
• Create merge request and wait for code review;
• Merge branch;
• Build and package the application;
• Publish the artifact in artifactory;
• Deploy the new build(s) to several environments.

This process is repeated for every task.

If it takes 10 days to implement, debug, and test features and 1 hour to build/rebuild, that seems reasonable. But if you have a dozen small tasks and go through the whole process for each task, it seems expensive. But you have to go through it to make sure you're not introducing additional errors into the application – that's what quality is all about.

The goal of the Continuous Integration (CI) process is to create a consistent and automated way to build, package, and test applications. When the process is consistent and reproducible, teams are more likely to make more frequent code changes, resulting in better collaboration, faster delivery preserving quality. Also, CI provides feedback to developers that new code changes haven't broken the application and can be deployed right away. As a result of the CI process, we get well-defined quality gates for application code.

Continuous Delivery(CD) starts where CI ends. The CD automates the deployment of applications in selected environments. Most teams work with multiple environments that differ from production, such as development and staging environments, and the CD provides an automatic way to deploy changes to each of them.

Continuous Integration and Continuous Delivery require automatic verification at all stages as their goal is to provide users with correctly working applications. Automatic verification is often implemented as a set of automatic unit/integration/contract/acceptance/smoke/performance tests and other automatic tests in all sorts, shapes, and sizes required for your application.

It should be noted that CI/CD was created primarily for faster development cycles and maintaining quality. And, of course, it starts with creating the right development process – teams that want to integrate CI/CD into their project often start by choosing a branching strategy, choosing when a feature branch should be merged with the main branch, choosing how to build, test and package the application, etc. Development teams practicing Continuous Integration use a variety of methods to control and verify that code is ready for production, and they can be defined by the Software Development Life Cycle(SDLC). It can be various automatic tests, code reviews, code coverage, and other quality gates.

Most CI/CD tools allow developers to run builds on-demand triggered by commits in the version control system or on a specific schedule. Teams need to discuss a build schedule that works best according to team size, expected development speed, deployment frequency, and other application considerations.

• The CI/CD process was created primarily for faster product development, preserving product quality
• Automate the processes to the maximum, the most time-consuming operations first
• For faster bug/problem detection, ensure that merges are frequent and builds are quick
• All the processes are consistent – every change is going through the same set of well-defined quality gates
• Builds are always reproducible – the build is commutative and associative, if the code is not changed, the build remains unchanged
• Be able to recreate all required artifacts. Ideally, have all infrastructure as code – the entire build process is described in any DSL language (e.g. groovy for Jenkins or YAML for Docker files)
• Well-coordinated work of developers, testers, and DevOps engineers. CI/CD is not possible on the project if the whole team is not working together (as a team).

The main form of verification on the CI/CD pipelines is automatic tests. Automated tests can include functional tests that are developed at the end of each sprint and are aggregated into a regression test for the entire application. These regression tests then inform the team if a new code change breaks one or more tests developed in all functional areas of the application where there is test coverage.

Regression tests are just the beginning. Performance testing, API testing, static code analysis, security testing, and other forms of testing can also be automated. The idea is that these tests can be run either from the command line, via a webhook, or via a web service and that they respond with status codes.

Once the testing is automated, continuous testing implies that the automation is integrated into the CI/CD pipeline. Some functional and unit tests can be integrated into the CI, which flags before or during the integration process. A test that requires a complete production environment, such as performance and security testing, is often integrated into a CD and performed after the artifacts are delivered to the target environments.

CI/CD supporters claim the benefits of faster time-to-market, faster customer feedback, improved developer productivity, reduced technical risk, improved quality, and increased customer satisfaction. I do not question any of these claims. What I am skeptical about is the cost of moving to CI and CD, the complexity of changing the company's software development culture, and the practical difficulty of maintaining a product that is always shipped. If you are realistic about all this and plan your remediation strategies well, you can benefit from continuous integration and delivery, no questions asked.

There are also large monolithic systems, where without additional modification (for example, large computer systems for banks with releases 3-4 times a year) CI/CD is not suitable, because such tasks require a complete renovation of the entire architecture.

Jenkins/Hudson, GitLab CI, TeamCity, Circle CI, Bamboo, Microsoft Team Foundation Build, GoCD, AppVeyor, Codeship, Drone

BitBucket Pipelines, Heroku CI, Travis CI, Codeship, Buddy CI, AWS CodeBuild

1. CI is essential to create a consistent and automated way to build, package and test applications. The main idea here is to create well-defined quality gates for code changes
2. CD automates the deployment of applications in selected environments
3. Automatic verification is the basis for CI/CD
4. There are many tools available on the market to create your own CI/CD pipelines.

Additional materials

• Release It!: Design and Deploy Production-Ready Software
• Continuous Delivery