CI/CD using Jenkins and Docker

This Project will help you to setup a CI/CD pipeline using Jenkins and Docker. It
includes automation using Jenkins Pipeline/Groovy scripting language, it uses sonar for
code quality and artifactory for artifactory management.

Tools:
Jenkins- CI/CD

Git/GitHub — Source Control Management

Docker — Container

JaCOCO — Code Coverage Tool

Gradle — Build tool

Ansible — Configuration Management

Ansible/Github/Docker/Cucumber/

Continuous Integration Pipeline

Configuration Management with Ansible

Continuous Delivery Pipeline

Clustering with Docker Swarm/Advanced Continuous
Delivery

How To Install Jenkins In AWS EC2 Instance

How to install Jenkins in AWS EC2 instance

Hello Everyone

Welcome to simplilearn .
In this project we will explore one of the most popular CI/CD tool Jenkins. We will try to
cover each aspect of Jenkins with a demo.

What is Jenkins?

Jenkins is a self-contained, open-source automation server which can be used to

automate all sorts of tasks related to building, testing, and delivering or deploying
software.

Components of Jenkins:

• Master Node: The system where you install and run Jenkins.
• Managed Node: Target systems managed by Jenkins.
• Repository: Version controlled system where you keep your code.
• UI: User interface to manage and configure Jenkins.
• Pipeline: Means of continuous deployment of your code to target systems from the
version control system.
• Plugins: Means of enhancing the functionality of a Jenkins.
 Next, we are going to learn how to install Jenkins.

Jenkins Installation:

Prerequisite:

• One EC2 Amazon Linux 2 instance with internet access

• Java 8 runtime environments

1 #####################################
2 ## Jenkins installation on AWS EC2 ##
3 #####################################
4 ## Create an EC2 instance with Amazon Linux 2 AMI
5 ## Amazon Linux 2 AMI (HVM), SSD Volume Type
6 ## Connect to your EC2 instance
7
8 ## Update all packages
9 sudo yum update -y
10
11 ## Install Java
12 sudo yum install java-1.8.0-openjdk-devel
There are several ways you can install Jenkins in your master node (EC2 instance in
our case).

• Using Docker
• Using Yum
• Using a War file
I would prefer the 1st method, using docker which comes with blue ocean plugin
preinstalled and this will be used in rest of the Jenkins blog series.
Using docker:

1 #############################################################
2 ## Jenkins installation on AWS EC2 using docker blue ocean ##
3 #############################################################
4
5 ## Install Docker.
6 sudo yum install docker
7
8 ## Add the ec2-user to the docker group so you can execute Docker commands without using sudo.
9 ## Exit the terminal and re-login to make the change effective
10 sudo usermod -a -G docker ec2-user
11 exit
12
13 ## Enable docker service
14 sudo systemctl enable docker
15
16 ## Start docker service
17 sudo systemctl start docker
18
19 ## Check the Docker service.
20 sudo systemctl status docker
21
22 ## Run docker jenkins blueocean container
23 docker run \
24 -u root \
25 --rm \
26 -d \
27 -p 8080:8080 \
28 -p 50000:50000 \
29 --name myjenkin \
30 -v jenkins-data:/var/jenkins_home \
31 -v /var/run/docker.sock:/var/run/docker.sock \
32 jenkinsci/blueocean
33
34 ## get the administrator password
35 docker exec -it myjenkin bash
36 cat /var/jenkins_home/secrets/initialAdminPassword
37 exit
38
39 #or from docker logs
40
41 docker logs myjenkin
Using Yum:

1 ###############################################
2 ## Jenkins installation on AWS EC2 using YUM ##
3 ###############################################
4
5 ## Add Jenkins repo to your yum repository
6 sudo wget -O /etc/yum.repos.d/jenkins.repo https://pkg.jenkins.io/redhat/jenkins.repo
7
8 ## Import a key file from Jenkins-CI to enable installation from the package
9 sudo rpm --import https://pkg.jenkins.io/redhat/jenkins.io.key
10
11 ## Install Jenkins
12 sudo yum install jenkins -y
13
14 ## Start and enable Jenkins service
15 sudo systemctl start jenkins
16 sudo systemctl enable jenkins
17 sudo systemctl status jenkins
18
19 ## Get the initial administrative password
20 sudo cat /var/lib/jenkins/secrets/initialAdminPassword
Using a WAR file:

1 ###################################################
2 ## Jenkins installation on AWS EC2 using war file##
3 ###################################################
4
5 ## Get the latest jenkins war file
6 wget http://mirrors.jenkins.io/war-stable/latest/jenkins.war
7
8 ## Install jenkins using war file
9 java -jar jenkins.war
10
11 ## Get the initial administartive password from the output of the above command
12 # Jenkins initial setup is required. An admin user has been created and a password generated.
13 # Please use the following password to proceed to installation:
14 # 2871112a4e704e3bbc9c2e0b2963c8c0
15 # This may also be found at: /home/ec2-user/.jenkins/secrets/initialAdminPassword

Jenkins Configuration:
Step 1: Open your EC2 instance public DNS or public IP
(http://<PUBLIC_DNS/PUBLIC_IP>:8080/) along with port 8080 in your
favorite browser. And provide the administrative password obtained during
the installation.

Step 2: Click ‘Install suggested plugins’.

Step 3: Create an administrative user.

Step 4: Our Jenkins configuration completed, click ‘Start using Jenkins’ to

open the Jenkins UI.
Hope you have enjoyed this article. To install Jenkins on another platform, you can refer
below Jenkins documentation
Docker Installation: Click Here

Jenkins setup on AWS

Java 8 should be present,if not please use the below command

Install JAVA 8

yum install wget ( in case wget is not found)

$ wget --header "Cookie: oraclelicense=accept-securebackup-cookie" http://download.oracle.com/otn-pub

/java/jdk/8u161-b12/2f38c3b165be4555a1fa6e98c45e0808/jdk-8u161-linux-x64.rpm
$ sudo yum localinstall jdk-8u161-linux-x64.rpm
export JAVA_HOME=/usr/java/jdk1.8.0_161/
export JRE_HOME=/usr/java/jdk1.8.0_161/jre

PATH=$PATH:$HOME/bin:$JAVA_HOME/bin

export PATH
sudo alternatives --config java

Install Jenkins
sudo yum update
sudo wget -O /etc/yum.repos.d/jenkins.repo http://pkg.jenkins-ci.org/redhat-stable/jenkins.repo
sudo rpm --import http://pkg.jenkins-ci.org/redhat-stable/jenkins-ci.org.key
sudo yum install jenkins
sudo service jenkins start

Install Git on Jenkins server

yum install git

Open the url in the browser. Default port is 8080

http://localhost:8080/

It will ask for the initial password, please run the below command

cat /var/lib/jenkins/secrets/initialAdminPassword

Configure Jenkins with JAVA_HOME,MAVEN_HOME

Go to Manage Jenkins ->Global Tool Configuration

Add JDK -> JAVA_HOME -> put the java_home path of the machine

Add MAVEN -> Select Install automatically.

Create a freestyle project in Jenkins

In source code management put this url as shown below

“https://github.com/dhuriti/AWSDemo.git”

In Build Section,select maven version and goal which you want to execute as shown
below. This is very important, you have to select invoke Artifactory maven 3 only not the
invoke maven top level targets.

Trigger the Build

Output of this job

[INFO] Packaging webapp
[INFO] Assembling webapp [LoginWebApp] in [/var/lib/jenkins/workspace/Test_Maven/target/LoginWebA
pp]
[INFO] Processing war project
[INFO] Copying webapp resources [/var/lib/jenkins/workspace/Test_Maven/src/main/webapp]
[INFO] Webapp assembled in [732 msecs]
[INFO] Building war: /var/lib/jenkins/workspace/Test_Maven/target/LoginWebApp.war
[INFO] WEB-INF/web.xml already added, skipping
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:36 min
[INFO] Finished at: 2018-03-27T20:59:52Z
[INFO] ------------------------------------------------------------------------
Finished: SUCCESS

Install Jfrog Artifactory.

For linux

Copy software from your machine to linux

Java should be present

Install JAVA 8

yum install wget ( in case wget is not found)

$ wget --header "Cookie: oraclelicense=accept-securebackup-cookie"
http://download.oracle.com/otn-pub/java/jdk/8u161-
b12/2f38c3b165be4555a1fa6e98c45e0808/jdk-8u161-linux-x64.rpm$ sudo yum localinstall
jdk-8u161-linux-x64.rpmexport JAVA_HOME=/usr/java/jdk1.8.0_161/
export
JRE_HOME=/usr/java/jdk1.8.0_161/jrePATH=$PATH:$HOME/bin:$JAVA_HOME/binexport
PATH
sudo alternatives --config java

Start artifactory.bat/.sh,present in bin folder

Go to your browser and visit IP_ADDRESS:8081 to visit

Artifactory in the browser:

Login — default username password — admin/password

Create a Local repository to store package files created by the

Jenkins project:

Click on Admin -> Repositories ->Local

Click on New:

Select Maven as a Project Type:

Give a Repository Key “jenkins-release” and click on Save and
Finish:
Similarly, create a Jenkins-snapshot repository:

Verify all repositories in the list:

Create a user that you can utilize from Jenkins to access
Artifactory:

Click on New:
Provide user details and Save:

Verify the list of users:

Provide the newly created user with permissions to the
repositories:

Security -> Pemissions

Create a new permission “jenkins-permission”

Select Repositories and click on Save & Finish:

Check the Permissions Management section in Artifactory for
recent changes:

Edit the permissions and assign the user: If user is already admin
then it has all permissions
Click on Save & Finish:
Check the Permissions Management section in Artifactory for
recent changes:

Jfrog setup is complete

Now storing the artifacts to the jfrog artifactory

Integrate Artifactory with Jenkins

Go to Manage Jenkins -> Manage Plugins

Once plugin installation is successful, you can configure Artifactory-related settings in

Jenkins:
Configure Artifactory in Jenkins:

Go to Manage Jenkins ->Configure System

 1. Go to Section “Build Environment”

3. Select Maven3-Artifactory Integration

4. Click on Refresh Repositories and select the repository in the release and snapshot
field from the lists:

Add Build Step as shown below

1. Save and click on Build now and verify logs in the Console Output. Jar files are
resolved from the local repository or Artifactory:
2. Once the package is created, it is stored in Artifactory too:

[main] INFO org.apache.maven.plugin.war.WarMojo - Webapp assembled in [85 msecs]

[main] INFO org.codehaus.plexus.archiver.war.WarArchiver - Building war: C:\Program Files (x86)\Jenkin
s\workspace\Test_Maven_MyProject\target\LoginWebApp-1.0-SNAPSHOT.war
[main] INFO org.jfrog.build.extractor.maven.BuildDeploymentHelper - Artifactory Build Info Recorder: Savi
ng Build Info to 'C:\Program Files (x86)\Jenkins\workspace\Test_Maven_MyProject\target\build-info.json'
[main] INFO org.jfrog.build.extractor.maven.BuildInfoClientBuilder - Deploying artifact: http://localhost:808
1/artifactory/jenkins-snapshot/com/javawebtutor/LoginWebApp/1.0-SNAPSHOT/LoginWebApp-1.0-SNAP
SHOT.war
[main] INFO org.jfrog.build.extractor.maven.BuildDeploymentHelper - Artifactory Build Info Recorder: Depl
oying build info ...
[main] INFO org.jfrog.build.extractor.maven.BuildInfoClientBuilder - Deploying build descriptor to: http://lo
calhost:8081/artifactory/api/build
[main] INFO org.jfrog.build.extractor.maven.BuildInfoClientBuilder - Build successfully deployed. Browse it
in Artifactory under http://localhost:8081/artifactory/webapp/builds/Test_Maven_MyProject/10
[main] INFO org.apache.maven.cli.event.ExecutionEventLogger - -----------------------------------------------------
-------------------
[main] INFO org.apache.maven.cli.event.ExecutionEventLogger - BUILD SUCCESS
[main] INFO org.apache.maven.cli.event.ExecutionEventLogger - -----------------------------------------------------
-------------------
[main] INFO org.apache.maven.cli.event.ExecutionEventLogger - Total time: 5.419 s
[main] INFO org.apache.maven.cli.event.ExecutionEventLogger - Finished at: 2018-03-30T16:55:43-04:0
0
[main] INFO org.apache.maven.cli.event.ExecutionEventLogger - -----------------------------------------------------
-------------------
Finished: SUCCESS

1. Go to Artifactory and verify the package:

Artifactory setup using maven is completed successfully

Sonarqube Setup

1. Download SonarQube from https://www.sonarqube.org/downloads/ and extract it

in the system:
2. Execute StartSonar.bat/.sh as per OS

3. Once SonarQube is up and running, open the browser at http://localhost:9000 to visit

the SonarQube dashboard

Integrate Jenkins with sonar

1. Go to the Jenkins dashboard and click on Manage Jenkins. Go to Manage

Plugins and in the Available tab find the SonarQube plugin.
2. Click on Install without restart:
1. Go to the Jenkins dashboard and click on Manage Jenkins.
2. Click on Configure system and find the SonarQube section.
3. Now, let’s go to SonarQube to get the token to integrate Jenkins and SonarQube.
4. Once SonarQube is up and running, open the browser at http://localhost:9000 to
visit the SonarQube dashboard:

1. Click on Login and give the default username and password as admin and
default to log in as an administrator.
2. Click on Login:
1. As of now, there is no project available in the SonarQube dashboard.
2. Click on the Administration tab and in the Security menu click on Users:

1. Initially, there are no tokens issued; there is a 0 token for Administrator:

1. Click on Tokens:

1. Give a name in the Generate Tokens section and click on Generate:

1. Copy the newly created token. Click on Done:

1. Verify the number of Tokens for the Administrator user:

1. Now we have all the required parameters to integrate Jenkins and SonarQube:
2. Go to the Jenkins dashboard and click on Manage Jenkins.
3. Click on Configure system and find the SonarQube section.
4. Click on Add SonarQube.
5. Provide the Name, Server URL, and Server version.
6. Paste the token value in Jenkins and save it:
1. Go to Global Tool Configuration and configure Add SonarQube Scanner:

1. Now, you are ready for the static code analysis of the project.
2. Go to the Build section and select Execute SonarQube Scanner:
 1. You can provide the location of sonar-project.properties or provide details directly
for static code analysis.

# Required metadata
sonar.projectKey=SonarHTMLCSSJS
sonar.projectName=Simple HTML CSS JS project analyzed with the SonarQube
sonar.projectVersion=1.0
# Comma-separated paths to directories with sources (required)
sonar.sources=.
# Encoding of the source files
sonar.sourceEncoding=UTF-8
 sonar.java.binaries=.

1. sonar.sources is the main property for static code analysis. With this property,
you inform SonarQube which directory needs to be analyzed:

1. Click on Save.
2. Go to Jenkins Project and click on Build now.
3. Go to Console output to check the logs.

Integrate Jacoco plugin with Maven

1. Install Jacoco plugin

2. Manage Jenkins -> Manage Plugin -> Search for Jacoco
3. Create a freestyle project in Jenkins
4. Use this link in source control amangement
“https://github.com/pkainulainen/maven-examples.git”
5. You have to add all configuration in pom.xml as present in the code

6. Add post build actions

7 Select “Record Jacoco Coverage report”

8. Jenkins build logs as shown below

Configure Jenkins with sample spring boot project using Gradle/Maven

Please refer this link to generate a sample project http://start.spring.io/

All executable(.sh) file should have this permission.

For Gradle, any .sh file should have the below permission else while executing
permission denied error comes

git update-index — chmod=+x gradlew

Pipeline Project Using Gradle

Create a Pipeline project in Jenkins and put the below code in the pipeline script and
trigger the build

pipeline {
agent any
stages {
stage(“Checkout”) {
steps {
git url: ‘https://github.com/dhutifile/calculator.git’
}
}
stage(“Compile”) {
steps {
sh “./gradlew compileJava”
}
}
stage(“Unit test”) {
steps {
sh “./gradlew test”
}
}
}
}

After build is successful, you could see the below output view

./gradlew bootRun

We have created the pipeline script directly in the Jenkins job.

Now we will see how to create the Jenkinsfile and commit it with the source code into
the git repository.

Jenkinsfile

Let’s create a file called Jenkinsfile in the root directory of our project

pipeline {
agent any
stages {
stage(“Compile”) {
steps {
sh “./gradlew compileJava”
}
}
stage(“Unit test”) {
steps {
sh “./gradlew test”
}
}
}
}

$ git add .
$ git commit -m "Add sum Jenkinsfile"
$ git push

Running pipeline from Jenkinsfile

When Jenkinsfile is in the repository, then all we have to do is to open the pipeline
configuration and in the Pipeline section:

• Change Definition from Pipeline script to Pipeline script from SCM

• Select Git in SCM
• Put https://github.com/dhutifile/calculator.git in Repository URL

Trigger Build.

Code Coverage

Code coverage is a tool that runs all tests and verifies which parts of the code have
been executed. Then, it creates a report showing not-tested sections. Moreover, we can
make the build fail when there is too much untested code.

JACOCO

1. Add JaCoCo to the Gradle configuration.

2. Add the code coverage stage to the pipeline.
3. Optionally, publish JaCoCo reports in Jenkins.
In order to run JaCoCo from Gradle, we need to add the jacoco plugin to the
build.gradle file by adding the following line in the plugin section:

apply plugin: "jacoco"

Publishing report directly on Jenkins is not working.

No such DSL method 'publishHTML'

stage("Code coverage") {
steps {
sh "./gradlew jacocoTestReport"
publishHTML (target: [
reportDir: 'build/reports/jacoco/test/html',
reportFiles: 'index.html',
reportName: "JaCoCo Report"
])
sh "./gradlew jacocoTestCoverageVerification"
}
}

Acceptance test in pipeline

The process goes as follows:

1. The developer pushes a code change to GitHub.

2. Jenkins detects the change, triggers the build, and checks out the current code.
3. Jenkins executes the commit phase and builds the Docker image.
4. Jenkins pushes the image to Docker registry.
5. Jenkins runs the Docker container in the staging environment.
6. Staging the Docker host needs to pull the image from the Docker registry.
7. Jenkins runs the acceptance test suite against the application running in the
staging environment.

Adding a Dockerfile and commit in Git and and add docker build/push to the jenkins
pipeline(Jenkinsfile)

In the root directory of the project, let’s create the acceptance_test.sh file:

#!/bin/bash
test $(curl localhost:8765/sum?a=1\&b=2) -eq 3

Jenkinsfile

pipeline {
agent any
stages {
stage("Compile") {
steps {
sh "./gradlew compileJava"
}
}
stage("Unit test") {
steps {
sh "./gradlew test"
}
}

stage("Package") {
steps {
 sh "./gradlew build"
}
}
stage("Docker build") {
steps {

sh "docker build -t nikhilnidhi/calculator_1 ."

}
}
stage("Docker push") {
steps {
sh "docker login -u username -p password"
sh "docker push nikhilnidhi/calculator_1"
}
}
stage("Deploy to staging") {
steps {

sh "docker run -d --rm -p 8765:8080 --name calculator_1 nikhilnidhi/calculator_1"

}
}
stage("Acceptance test") {
steps {
sleep 60
sh "./acceptance_test.sh"
}
}
}
post {
always {
sh "docker stop calculator_1"
}
}
}
For Gradle

Create a freestyle project and use gradle

Configure Gradle -> Manage Jenkins->Global tools configuration->Gradle

and then in the job select Gradle version instead of default.

Setup using Docker-compose

Let’s start with an example and imagine that our calculator project uses the Redis
server for caching. In this case, we need an environment with two containers, calculator
and redis. let’s create the docker-compose.yml file at the same location.

version: "3"
services:
calculator:
image: calculator:latest
ports:
- 8080
redis:
image: redis:latest

References