Thought Leadership
CI/CD Joined Forever; Never The Same
Andrew Fong
Two core practices make up the DevOps cycle in DevOps: Continuous Integration (CI) and Continuous Delivery (CD).
Their functions are intertwined, streamlining the flow of code from development to production. This interconnected flow leads to the misconception that they represent the same business process and that one technical platform can address both.
This blog post aims to debunk this misconception by exploring the unique aspects of CI and CD, highlighting why each requires a specialized technical system.
CI and CD: Unique Business Processes
CI and CD are the underpinnings of the DevOps process. However, their purposes differ significantly.
CI's primary job is merging developer working copies into a shared mainline. It concentrates on ensuring code modifications from various developers coexist without conflicts via frequent integrations and automated testing.
On the flip side, CD picks up where CI leaves off, taking the tested artifact or codebase and ensuring its reliable release to production. This operations-oriented process includes configuration management, deployment automation, and environment orchestration.
Two significant issues typically arise when treating them as the same process:
Unhealthy organizational conflict - driven by a speed vs. quality tradeoff
Lack of correctness due to the breadth of the problem space
Presenting CI and CD as a single business process can excessively increase cognitive overhead at the engineering task level, given their significant differences.
This overhead causes engineers working at the task level to be frustrated with the technology and business processes. For example, debugging a failing or flaky test in CI significantly differs from applying database change during CD.
This is compounded when a team is asked to build a CI/CD system with unified technology to optimize either cost or the perception of creating a single “flow.” This results in over or under-investment in certain areas reducing reliability, efficiency, and effectiveness.
When leadership believes these are the same process and builds systems and organizations reflective of this, the cognitive load on engineers increases.
A more sustainable approach is recognizing that these are separate processes and that the context switch at this juncture is healthy.
Despite their interconnected nature, CI and CD cater to different aspects of the software development lifecycle. Recognizing this distinction necessitates treating CI and CD as separate business processes.
The Need for Different Technical Systems: Separation of Concerns
Two technical systems follow if CI and CD represent distinct business processes. Each process comes with unique needs, constraints, and objectives that a single system cannot meet.
The differences are across these three main categories:
Static vs. Dynamic Environments
Varied Workloads
Differences in Algorithms and Interfaces
Static vs. Dynamic Environments
CI runs in a static environment with controlled, repeatable inputs.
CD runs in dynamic environments and must adapt continuously to change.
Essentially CI strives to be idempotent while CD cannot.
CI, a developer-centric process, aims to merge code changes into a central repository multiple times daily. It uses automated testing to ensure any new code's integrity within a controlled, stable environment to ensure consistency and repeatability.
On the contrary, CD, an operation-centric process, focuses on deploying code to dynamic environments. CD must accommodate this diversity, considering production environments' complex and ever-changing nature.
The nuanced difference between managing a controlled environment (CI) and deploying to diverse, dynamic production environments (CD) reinforces the need for their separate treatment.
Varied Workloads
CI and CD differ in workload patterns, influencing their design and optimization choices.
Due to frequent code integrations and accompanying multiple automated tests, CI demands significant computational resources. Most CI have a large number of “runners” that build and test artifacts and then collect them. Additionally, these systems implement caching subsystems for artifacts. The system does not need to operate in real-time; it can post-process logs from the workers and re-run workers that failed.
CD coordinates, manages, and validates releases to diverse production environments. The architecture of a CD system looks like a control plane. The CD control plane needs to handle real-time events across your production environments. The control plane must also be fault tolerant and able to reconstruct the state to perform operations safely.
The differences in workload mean there is minimal overlap in system design.
Differences in Algorithms and Interfaces
The distinction between CI and CD extends beneath the surface, resting on the workload differences.
CI requires parallelism, test selection, and flake management, whereas CD necessitates convergence, drift detection, and working with cloud interfaces.
This disparity requires separate technical systems.
Continuous Integration Concepts
Test Parallelization: CI systems should facilitate the parallelization of tests to save time and maximize resource utilization.
Test Selection: Efficient CI systems employ algorithms to identify and run only those tests affected by recent code changes.
Flake Detection and Remediation: CI systems must detect and manage flaky tests that could undermine the testing process's reliability.
Modern Continuous Delivery Concepts
Convergence: CD systems must align the system's current state with the desired state. (see Myth of the CD Pipeline Below)
Drift Detection: CD systems must detect drifts in the production environment's actual state over time and trigger corrective actions.
Cloud Interfaces: Modern CD systems interact with various cloud platforms, handling their unique APIs, services, and idiosyncrasies to deploy software.
The Myth of the Pipeline
While pipelines are excellent tools for the CI process, their efficacy for CD is not as straightforward.
The complexity and dynamism of the production environment, requiring diverse configurations, feature toggling, and orchestration, are not ideal for a linear process like a pipeline.
Simplifying these multi-dimensional operations into a pipeline will result in inflexibility and oversimplification, leading to bottlenecks, slower responses, and deployment issues in modern cloud environments.
We have written extensively on why the convergence approach is superior in the following posts:
The Myth of a Unified UI
One might yearn for a unified CI/CD UI, even with the technical distinction established.
While this desire is common, understanding and embracing the data differences and the unique jobs both systems must accomplish helps engineering teams and businesses move forward.
A more sustainable approach is to:
Identify and measure system data where you want to perform a context switch.
Establish clear principles for the Software Development Life Cycle (SDLC) process. We have outlined ours in our Golden Path post.
Embrace good context switches!
They tell an engineer, “Here be dragons,” and all dynamic environments have dragons. This is why building self-driving cars is hard as well — the world is constantly changing.
Conclusion
While CI and CD form the cornerstone of the DevOps process, it is essential to understand why melding them into a single technical system is both counterproductive and dangerous!
Recognizing and respecting these differences enables us to build more effective, efficient, and resilient DevOps processes, expediting and enhancing software delivery's safety and reliability.
If you’d like to learn more about delivering software to production, schedule a time to chat!
