CI/CD with Jenkins and Docker

Start your Jenkins journey with the ideas everything else builds on. This module walks you through what CI/CD really means, introduces Jenkins and its plugin ecosystem, and shows how Jenkins compares to Jenkins X for GitOps and Kubernetes deployments. After this you will understand where Jenkins fits and be ready to set up your own environment with Docker.

Set up a lab you can keep and practise on for the whole course, using nothing but Docker. You will understand the two-container design: a Jenkins server plus a Docker-in-Docker (DinD) server, wired together with shared volumes and credentials so Jenkins can build and run containers later. You will also see why Docker Compose gives you this whole environment as code, ready to reuse.

Get a working Jenkins server running in minutes. You will clone the bootcamp repository, run docker compose build to bake a custom image with the Blue Ocean and Docker plugins pre-installed, then bring it up with docker compose up. You will find the initial admin password in the container logs, install the suggested plugins, create your admin user, and log in ready to build jobs.

Downloadable Lab Guide

Welcome to Module 2 of our course on Essentials of Jenkins for DevOps Practitioners! In the previous module, we laid the foundation by understanding the concepts of Continuous Integration and Continuous Delivery (CI/CD), exploring the Jenkins ecosystem, and setting up a Jenkins server using Docker.

In this module, we’ll dive into the practical aspects of using Jenkins to build a CI pipeline manually. We’ll take a hands-on approach, working with a real-world use case to understand the fundamentals of Jenkins.

Think of this module as your first cooking lesson in the Jenkins kitchen. Just as a chef starts by learning the basics of knife skills, ingredient preparation, and recipe reading, we’ll start by mastering the fundamental skills of using Jenkins.

We’ll begin by exploring the Jenkins Dashboard and UI, familiarizing ourselves with the various components and navigation. It’s like getting acquainted with your kitchen layout and tools.

Next, we’ll introduce a use case scenario and guide you through forking a Git repository containing a Node.js application. This is similar to gathering the necessary ingredients for your recipe.

With our code repository ready, we’ll dive into the core of this module – creating and configuring Jenkins jobs. We’ll create separate jobs for building, testing, and packaging our Node.js application. This is analogous to the different stages of meal preparation – chopping, cooking, and plating.

But a series of isolated jobs doesn’t make a complete pipeline. We’ll learn how to connect these jobs using upstream and downstream configurations, orchestrating them into a logical flow. It’s like sequencing the steps in a recipe to ensure the dish comes together perfectly.

Finally, we’ll add a touch of visual appeal to our pipeline using the Build Pipeline plugin. This will give us a graphical overview of our connected jobs, making it easier to understand and monitor the flow of our CI process.

By the end of this module, you’ll have gained practical experience in constructing a CI pipeline manually using Jenkins. You’ll understand the role of each job, how to connect them, and how to visualize the pipeline.

This hands-on knowledge will form the foundation for the next module, where we’ll explore the more advanced concept of defining the entire pipeline as code using Jenkins Pipeline.

So, let’s put on our chef’s hats and get started with Module 2: Jenkins Fundamentals – Building CI Pipeline Manually. Get ready to get your hands dirty and learn by doing!

Find your way around the Jenkins dashboard so nothing feels hidden. You will tour the build queue, executor status, and the Manage Jenkins area, where you set the Jenkins URL, configure executors to match your CPU cores, add build tools like Maven and JDK under Global Tool Configuration, and manage plugins and nodes. After this you will know where every important setting lives.

Meet the app you will build pipelines around: a Spring Boot Java application that uses Maven and connects to a database. You will see the three-job CI plan (build, test, package) and why keeping the mainline build green matters. Then you will fork the Git repository into your own account so you can commit changes, trigger jobs, and follow every lab hands-on.

Build your first Jenkins job and watch it compile a real Java application. You will organise work inside a folder, install the Maven Integration plugin, set your Maven version in Global Tool Configuration, then create a Maven project that clones your fork's main branch and runs the compile goal. When you click Build Now, the console output shows the build going green, proving the code compiles.

Add the test and package jobs by copying the build job, so you move fast without rebuilding config. You will run unit tests with the Maven test goal, then package a runnable Spring Boot JAR using package with skipTests, archive the artifact with a post-build action, and stamp each build with the short Git commit hash as its version. Now you have build, test, and package all working.

Turn three separate jobs into one ordered sequence. You will learn the upstream and downstream idea and wire build, test, and package together using post-build actions and build triggers, so triggering the build job runs the whole chain in order. You will also see the two ways to fire this off from Git: an incoming webhook, or Jenkins polling the repository for changes.

See your job chain as a real pipeline and make it run on its own. You will install the Build Pipeline plugin to create a view that shows build, test, and package with colour-coded status, then set up Poll SCM using cron-style scheduling so Jenkins checks your repository and triggers the pipeline only when there is a new commit. A test commit proves the whole flow runs automatically.

Now you put it into practice. Follow the lab guide step by step to build a simple continuous integration workflow in Jenkins for the sample Java application, from source checkout through build and test. By the end you can set up a working CI job on your own, which becomes the foundation for writing it as a pipeline-as-code Jenkinsfile in the next module.

This module moves you from manually clicking through Jenkins jobs to the modern way of building CI. You will write your pipeline as code using a declarative Jenkinsfile, launch it with the Blue Ocean UI, and set up two-way integration between Jenkins and Git so commits trigger builds and status flows back to your repository. You also get multi-branch pipelines that create a job for every branch automatically.

Understand why teams moved from manual jobs and scripted Groovy pipelines to the cleaner declarative Jenkinsfile. You will learn the structure with a simple formula (pipeline, agent, triggers, tools, stages, stage, steps, post) and watch a sample pipeline run as a pipeline job. After this you can read a Jenkinsfile, tell scripted from declarative at a glance, and see the stage view execute without any extra plugin.

Turn your manual build, test, and package jobs into a real declarative Jenkinsfile placed at the root of your repository. You will start from a scaffold to avoid syntax trouble, set the agent, add the Maven tool configuration, and use the Snippet Generator to create shell steps for mvn compile, test, and package. After this you can write a clean working Jenkinsfile for a Java app yourself.

Create and run your pipeline the modern way using Blue Ocean. You will connect Jenkins to GitHub with a personal access token, add credentials, set scan triggers so changes get picked up in a minute or two, and get the two-way integration that shows build status right on your commit history. After this you can launch a Jenkinsfile pipeline from GitHub and watch the stage view live.

See why the pipeline Blue Ocean created is really a multi-branch pipeline and why that matters for team workflows. Jenkins scans every branch, and any branch with a Jenkinsfile gets its own pipeline automatically, then gets cleaned up when the branch is deleted. After this you understand how feature branches, pull requests, and CI feedback on GitHub fit together for collaborative development.

Learn the fastest way to change a pipeline without editing raw Groovy by hand. Using the Blue Ocean visual editor you will add an archive artifacts step and a pre-step to stamp the build with the commit hash, then save so Jenkins commits the updated Jenkinsfile back to your repo. After this you can add stages, steps, and parallel stages visually and know which changes still need the code.

Time to build it yourself. Follow the lab guide to create a modern CI pipeline with a declarative Jenkinsfile, launch it through Blue Ocean, edit it in the visual UI, and explore multi-branch behavior and the Jenkins-to-Git integration. By the end you can write pipeline as code and run it the way real teams do today, ready for trunk-based development next.

This is an optional deep-dive module for anyone who writes a lot of Jenkinsfile code. You will look more closely at scripted versus declarative pipelines, the structure and syntax, and some best practices for keeping your Jenkinsfile clean and maintainable. If you just want to keep moving with the main use case, you can safely skip ahead; if you want stronger Jenkinsfile skills, this section is for you.

This module is about defining and enforcing real development workflows. You will pick a branching model, trunk-based development, and learn to enforce it by combining GitHub and Jenkins CI. Along the way you set up branch protection rules, pull request based workflows, and mandatory code reviews. After this you understand how these pieces work together to keep your main branch safe and your team's practices consistent.

Your branching model shapes how your team develops and ships. Here you'll compare Git Flow, with its develop and master branches, feature branches and release branches, against the simpler trunk based development model that keeps the main line locked and uses short lived branches. After this you'll know why trunk based development means fewer merge conflicts, and why the model you pick decides how complex your workflow becomes.

This is where you learn what it actually means to enforce trunk based development. You'll see how to lock down the main branch, allow changes only through short lived branches, and let code in via pull requests that must pass the CI pipeline and get code reviews from peers. After this you'll be able to list the exact policies, protected main, mandatory build and unit tests, required reviewers, that keep bad code out of your main line.

Watch the whole setup end to end. You'll generate a GitHub personal access token to push, see why direct pushes to main are a problem, then add branch protection rules that require pull requests, code reviews, and passing Jenkins status checks before any merge. After this you'll be able to protect your own main branch, wire in required status checks, and even block admins from bypassing the rules.

Now do it the right way. You'll use git reset --hard to undo a direct commit to main, then create a feature branch, push it, and raise a pull request that the multi-branch pipeline picks up and tests automatically. You'll also fix the common trap of a PR pointing at an upstream repo instead of your own. After this you'll run a clean branch, push, PR, and status-check flow on your own repository.

A pull request stays blocked until someone approves it, so here you'll learn how to make that review happen. You'll add a collaborator to the repository, using a secondary GitHub account if you're working solo, request a review, approve the changes, then merge safely and clean up the branch locally with git branch -D. After this you'll run the full code review and merge step of trunk based development yourself.

Your turn to practice. Use the lab guide attached to this lecture to set up branch protection, raise a pull request, get a review, and merge into a locked main branch on your own. Revisit the earlier lessons wherever you get stuck. Once you've implemented trunk based development yourself, you're ready for the next module, where we integrate container based delivery with Jenkins using Docker.

This module is where Jenkins and Docker come together. You'll start using Docker to build your jobs, then go further and automate the container image build and publish straight from Jenkins. By the end you'll understand the two phases ahead, Docker based build agents first, then full container image delivery, and why this is how real DevOps pipelines tie their tools into one flow.

Running jobs with agent any limits you to whatever global tools Jenkins has installed. Here you'll see why switching to Docker based agents gives every stage a clean, disposable container with exactly the tools and versions you choose. You'll also learn why phase two needs per stage agent configuration and a global agent set to none. After this you'll understand the docker-in-docker setup, shared certificates, and DOCKER_HOST wiring behind the integration.

Before changing your real pipeline, prove the plumbing works. You'll build a small smoke test pipeline with agent docker and a Maven image, picking the right Docker Hub tag, 3.9.6 on Eclipse Temurin JDK 17 Alpine, to match your global tools. Then you'll exec into the Docker instance and watch the container appear and vanish. After this you'll be able to confirm Jenkins can reach the Docker host and run jobs in containers.

Time to convert your real pipeline to Docker agents. Using the Blue Ocean visual editor, you'll set the global agent to none, then add a Docker agent and image to each stage, build, test, and package, without hand-editing the Jenkinsfile. You'll commit to a new branch to respect your branch protection rules, then watch a fresh container spin up per stage. After this you'll have per stage Docker agents running your CI, phase one complete.

Write a Dockerfile that packages a Maven-based Spring Boot app into a container image. You will pick the right base image, set the work directory, copy the source, run maven package while skipping tests, expose port 8080, and launch the jar. By the end you can build the image locally with docker build, run it, check the logs, and open the running app in your browser.

Cut your image size down by moving to a multi-stage Dockerfile. You will see why build tools, Maven, and source code should never ship in the final image, then split the build into a build stage and a slim runtime stage using an Alpine JRE base like Eclipse Temurin. After this you can shrink an image from 457MB to around 211MB and ship only the jar and runtime you actually need.

Move the image build out of your laptop and into the pipeline. You will add a scripted stage that uses docker.withRegistry, tag each image with the short git commit hash, and push to Docker Hub with extra tags like latest and dev. You will also run this stage in parallel with the package stage and learn to read the log when a missing credential makes the build fail.

Fix the failing push by adding Docker Hub credentials to Jenkins the right way. You will create a global username-password credential using a Docker Hub access token instead of your real password, match its ID to the one in your pipeline, and set the stage agent so it runs on the Jenkins server. By the end your pipeline builds, tags, and publishes images to your Docker Hub account on its own.

A quick look back at everything you built in this continuous integration journey with Jenkins. From a simple pipeline to pipeline as code with a Jenkinsfile, Blue Ocean, two-way GitHub integration, branching policies, and container-based delivery with Docker. Use this recap to see how the pieces connect before you carry these Jenkins CI skills into your own projects.