- Here, we will introduce the deployment tool(Ansible) where we will use ansible playbook
- We are going to use Ansible as a deployment tool so that Jenkins does not need to do administrative kind of activities because Jenkins is more efficient in working as a build tool.
- Along with Ansible, we will be using Dockerhub help.
Table of Contents
Integrating Ansible in CI/CD Pipeline
- With the help of Ansible and Dockerhub, we will make this pipeline smoother.
- So in this case, Jenkins is going to take the code from GitHub, build artifacts, and copy that artifact onto the Ansible Server. Ansible is now in charge of creating images and deploying the containers.
- Ansible is going to take the artifact and with the help of Dockerfile, it creates a docker image. This docker image, we can commit it into Dockerhub because Dockerhub is a repository to store the docker images.
- Whenever we execute any Ansible playbook to deploy a container, Docker-host communicates With DockerHub to pull the Docker image(whatever we will mention in the playbook) and then create a container out of it.
NOTE:- we need a repository in case it is a non-containerized application(I mean to say which can deploy directly on VM) maybe we use ‘Nexus’ or ‘Artifactory’. But here already we have reached the containerization level that’s the reason we will use DockerHub.
What we will do here in Module 5?
1)First, we will set up the Ansible Server(EC2 instance.
2)Then we will create a DockerHub account.
3)Then we will integrate Ansible with Jenkins so that Jenkins can transfer the artifacts onto the Ansible server.
4)Then we are going to manage this docker host with the help of Ansible
Step 1:- Setup Ansible Server
We will do the following steps to prepare the Ansible server on AWS
1) Setup EC2 instance
First, Launch the EC2 instance and give the name as ansible-server.
If you want to know how to launch an EC2 instance then please go through the Module-2
2) setup Hostname:-
After connecting above EC2 instance(ansible-server), run below commands
sudo su -
vi /etc/hostname
init 6
Press R to restart the session
3) Create ‘ansadmin’ user:-
Please run the below command to create a user and setup the password as well to user ‘ansadmin’
sudo su -
useradd ansadmin
passwd ansadmin
4) Add the user to the ‘sudoers’ file :-so that the ‘ansadmin’ user will get Administrative Privileges
visudo
vi /etc/ssh/sshd_config
service sshd reload
Now, verify to log in as an ‘ansadmin’ user in a different session or duplicate session
5) generate ‘ssh-keys’:- Because in Ansible we are going to use ‘key-based authentication login as ‘ansadmin’ use ( you can say password less communication )
ssh-keygen
6) Enable ‘password-based’ login:- By default it is disabled in the EC2 instance and we are going to enable it for smoother configuration
7) Install Ansible:- Finally we will install the ansible now. for that please run the below command
sudo su -
yum install ansible
python –version
ansible –version
Video Recording
Still you have any doubt in any steps, please go through this video again
Step 2:- Integrate Docker with Ansible
Manage Docker-host with Ansible
- Here, We will add docker-host to Ansible as the ‘manager node’ so that our Ansible(control node) can manage our docker-host.
- We are going to write a playbook. And this playbook will tell docker-host how to create a container.
- For that,We need to add docker-host as a Slave(client).
- To enable, we need to do some activities on Docker-host as follows
On docker-host-
a) Create an ‘ansadmin’ user:- Please reun below commands
sudo su -
useradd ansadmin
b) Add ‘ansadmin’ to sudoers files
visudo
c) Enable password-based login
grep Password /etc/ssh/sshd_config
On Ansible Node-
First log in as root user with the help of below command
sudo su -
a) Add the docker host IP address into the host file(inventory file)
NOTE:- You can also find out the ip adress of docker-host with the help of ‘ifconfig‘ command on docker host
b) Copy SSH keys (public SSH keys to the docker host)
C) Test the connection:- please test test the conection(communication) with the help of below command.
ansible all -m ping
Now, we can say that our ansible server can communicate to docker-host without credentials(password-less authentication)
Video Recording
Still you have any doubt in any steps, please go through this video again
Step 3: - Integrate Ansible with Jenkins & copy artifacts onto the Ansible system.
Here, We are going to see how to integrate Ansible with Jenkins so that Jenkins can able to copy artifacts onto the Ansible system and people can able to create images or it can deploy the containers and Docker host.
Ansible is successfully integrated with Jenkins. And now, We will create a new Jenkins job to copy artifact onto Ansible. For that, we will do the following steps
Now, verify that the artifact is copied into ansible(/opt/docker) or not.
It successfully copied.
NOTE: – In this directory, we want to copy that’s why I have given the path in the remote directory section as “//opt//docker” above.
Video recording
Step 4: - Build an image and create a container on Ansible
- In the previous Step, we integrated our Ansible with Jenkins and copied artifacts by using Jenkins Job
- Now it’s time to create our Docker image with the help of the artifacts, which we have copied onto Ansible.
- This activity again, we have done previously on our Docker host. Same thing we are going to do for that.
- Of course, we need to install Docker on our Ansible system.
- We haven’t done that yet. So we are going to install Docker over here. For that please run the below commands
cd /opt/docker
sudo yum install docker
cat /etc/group
Now, we should add our ‘ansadmin’ in the docker group.
sudo usermod -aG docker ansadmin
Now. let’s start the docker services
service docker status
sudo service docker start
- It is time to build a docker image
- Of course, we need a Dockerfile to create a Docker image.
- And already we have created a docker file on our docker-host. So, we can get from there instead of creating a new Dockerfile which is available under ‘/opt/docker/‘
vi Dockerfile #(paste the content inside this)
docker build -t regapp:v1 . #(where regapp:v1 is image name)
chmod 777 /var/run/docker.sock #Give the execution permission
Now, we will create a container from this image-
docker run -t –name regapp-server -p 8081:8080 regapp:v1
Note: – If you use option -t, it won’t run in the background. And exit by ctrl+c
Now. we will verify whether our app is accessible or not.
browser>ip:8081/webapp
Whatever we have done here, we did it manually I mean to say we have run all commands manually to create an image and create containers. But we will do it automatically in the next step.
In the Next step, we will create an Ansible Playbook, which can do all these activities through the playbook.
Video Recodring
Step 5: - Ansible Playbook to create image and container
- So far, we have created an Ansible server and have integrated it with Jenkins, so Jenkins could able to copy artifacts.
- With those artifacts, we created a docker image using manual commands.
- Now this image is available on the Ansible system.
- Now how can I make This available to our target environment, which means that a docker host? And that is the reason Docker Hub comes into the picture.
- If we use a DockerHub, we can push our image onto Docker Hub, and this DockerHub can be accessible by any Docker system.
- Now from our Ansible, We can instruct this Docker host that, OK, we’re going to pull the image from DockerHub and create a container.
- This is how the Docker host can pull the image and create a container out of it.
- Now this image is customized by Ansible, which means the artifacts we created.
- So our application is running on Docker Host and our end users could be accessing this Docker Host.
Now, let’s go to our Ansible system to create Playbook to create a docker image
sudo su -
sudo su - ansadmin
cd /opt/docker
To run the Ansible playbook, we need to run the below command
ansible-playbook
cat /etc/ansible/hosts/ #(this is an inventory file)
ifconfig #(To find out Ip address)
sudo vi /etc/ansible/hosts
Note 1: –We need to add our client nodes (docker host or ansible client) along with ansible ip into the inventory file (/etc/ansible/hosts).
Note 2:- But to make work with Ansible, you should copy your SSH keys into the Ansible Server itself. Otherwise, you cannot be able to communicate with this system.
Now test it out with the help of below command
ansible all -a uptime
Now copy your SSH keys into the Ansible system
ifconfig
ssh-copy-id <destination ip> OR ssh-copy-id localhost
Now, it is time to write our Ansible playbook
vi regapp.yml
—
- hosts: ansible
tasks:
- name: create the docker image
command: docker run -t regapp:latest .
args:
chdir: /opt/docker
Run this playbook now with the help below command.
ansible-playbook regapp.yml –check #(To check only correct synext inside playbook)
ansible-playbook regapp.yml
Docker images
Video recording
Step 6: - Push (Copy) the docker image onto the Docker Hub
- So far, we have created ansible playbook, which can create your docker image.
- But for our purposes is, we would like to commit this Docker image to the DockerHub.
- For that, We need a DockerHub account. I have already created an account.
- If you haven’t created it, it is a quite simple step. You can create one so that we can commit our image or we can push our image under this picture.
- We are going to see how we can enable that by using the Ansible playbook.
To commit Docker images. First, we need to log in to our Docker hub account from our Ansible system.
Let’s go back to our Ansible server. And run the below commands.
docker login
Username: <your dockerhub username>
Password: <your dockerhub password>
docker images
docker tag <imageid> abhishek040/regapp:latest
docker push abhishek/regapp:latest
Now, verify whether its pushed to Dockerhub or not
NOTE: – But in the real world, we may not use Docker Hub. We can create our own Docker registries in our local system like JFrog Artifactory or Amazon docker registry or else azure docker registry or Anything we can use, that will just hold our image.
Video Recording
Step 7: - Jenkins Job to build an image onto Ansible
- In the previous lecture, we have seen that how to commit(push) a docker image to the docker hub.
- We have executed manual steps before executing.
- We have changed the Image’s name with the username, whatever we have created into the DockerHub. Then we used Docker push Command to push the image onto Dockerhub.
- Now I would like to incorporate these changes into the Ansible playbook so that I do not need to do it manually.
On ansible-
Here, I am going to add two more tasks in our Ansible Playbook “regapp.yml“
1) One task for tagging
– name: create a tag to push the image onto dockerhub
command: docker tag regapp:latest abhihsek040/regapp:latest
2) Another task for pushing image onto the docker hub
– name: push docker image
command: docker push abhihsek040/regapp:latest
sudo su -
sudo su - ansadmin
cd /opt/docker/
ll
vi regapp.yml
Run the below command now.
ansible-playbook regapp.yml –check
cat /etc/ansible/hosts
ansible-playbook regapp.yml –limit localhost
Now verify on the docker hub account whether it’s pushed or not.
Now, we will create a Jenkins job.
- Now I don’t want to execute this Ansible playbook as well.
- Then what we can do? we can give this command “ansible-playbook regapp.yml” to our Jenkins server.
- So Jenkins server will initiate this Ansible playbook whenever there is a new code commit.
- That is where we need to create a new image read. Otherwise, there is no point in creating the image.
- So now what we can do, is we will go to our Jenkins job and update and add this Ansible playbook to execute it whenever there is a change in the code. so that, this Ansible playbook gets executed and creates a new image.
let’s go back to our Jenkins and copy artifacts onto Ansible.
Now, I will update my source code so that this Jenkins job will be executed and run our ansible playbook. Once it’s done, we could able to see the docker image with the latest timestamp.
Now, I will commit these changes onto the GitHub branch as below
As soon as you commit and push to Github, our Jenkins job will triggered automatically because of POL SCM
Conclusion:- Once a code change happens, Jenkins triggers a job. Copy artifacts onto Ansible and will create the image.
Now, we are left with only one step which is creating a container out of this image.
For that, we need to write down an Ansible playbook.
In that Ansible playbook. We will tell our Docker host to go and connect to the Docker Hub. Pull this image and create a container out of it.
How to write that Ansible playbook will be seen in the next Step.
Video Recording
Step 8: - How to create a container on docker-host using ansible-playbook
- image and committing it to the Docker hub.
- Now we need to write one more Ansible playbook.
- With that Ansible playbook, we will create a new container under the Docker host.
- Then this Docker host can understand the instructions, and it’s going to pull the image from the DockerHub and create a container out of it.
Let’s create it and build a playbook.
On ansible system:- we will create ansible playbook here
sudo su -
sudo su - ansadmin
cd /opt/docker/
vi deploy_regapp.yml
—
- hosts: dockerhost
tasks:
- name: create container
command: docker run -d –name regapp-server -p 8082:8080 abhishek040/regapp:latest
ansible-playbook deploy_regapp.yml –check
Now, first, I will go to the docker host and delete all previous containers and images before executing the above ansible-playbook so that we can easily identify the newly created containers and images.
On docker host-
sudo su -
docker ps -a
docker rm -f <container id>
docker images
docker images prune
docker rmi <repository id with tag>
Now run ansible-playbook on Ansible system
ansible-playbook deploy_regapp.yml
It failed because of permission denied.
NOTE: – give Execution permission to /var/run/docker.sock on docker-host
Now, again run ansible-playbook
Now, verify that image and container created or not on docker-host.
NOTE: – But if we run this ansible-playbook again(second time), it will through an error.
ansible-playbook deploy_regapp.yml
If you see the above error, it will be that the container name is already used by another container.
So either you need to remove this container and create a new one. Or else change the name.
That is what it is suggesting how we can overcome this problem.
So we will overcome this issue in the next step
For Reference Doc, please click here
Video recording
Fixing the playbook error issue
We are trying to create an end playbook that can be able to deploy the container on the docker host. For that, we have already created the ansible-playbookplaybook.
But if we are using that one, it is throwing an error because that container exists on the Docker host. We cannot able to create a new container.
To overcome this problem, We need to follow the same steps that we have executed on our Jenkins in previous modules. That is
1) Remove the existing container,
2) then remove the existing image.
3)At last, Create a new container.
Note:- We need to remove the existing images because images exist in our local system.It won’t go to the Docker hub to pull the latest images. That is the reason we should remove the existing image.
These are the instructions we need to incorporate into our Ansible playbook so that the deployment process can be smooth.
Once this is done, we will add this deployment Ansible playbook to our Jenkins so that our Jenkins can able to take care of the entire ‘continuous integration and continuous deployment’ of our containers.
Let’s go and update our Ansible playbook now.
On ansible system-
sudo su -
sudo su - ansadmin
cd /opt/docker/
vi deploy_regapp.yml
—
- hosts: dockerhost
tasks:
- name: stop existing container
command: docker stop regapp-server
- name: remove existing container
command: docker rm regapp-server
- name: remove existing docker image
command: docker rmi abhishek040/regapp:latest
- name: create container
command: docker run -d –name regapp-server -p 8082:8080 abhishek040/regapp:latest
ansible-playbook deploy_regapp.yml –check
ansible-playbook deploy_regapp.yml
NOTE:- Remember, always stop the container first then remove the container and after that, we can remove the docker image
Video recording
Step 9:- Jenkins CI/CD to deploy on the container using Ansible
- Finally, we were able to manage to create the required Ansible playbooks, which can able to push the image and initialize our Docker host to create a container and the Docker host could, able to pull the image successfully and create a container.
- Now it’s time to make it automated.I don’t want to execute any command.
- Everything should be automated, so I will just make the changes and commit them to GitHub so that Jenkins could able to initiate all the required stuff.
- How it is going to happen? Let’s see How.
Go to Jenkins Dashboard>copy_artifact_onto_ansible>configure >exec command>
ansible-playbook /opt/docker/regapp.yml
sleep 10
ansible-playbook /opt/docker/deploy_regapp.yml
Now, change the code and commit that code into Git Hub. The Jenkins job will be triggered automatically now
We have configured our Jenkins job in such a way that if somebody modifies code, it should automatically build the code, create the image, and create a container.
And We were able to access those changes from the browser.
Note:- Problem:- But if you see the problem, whenever there are some changes, we terminate the existing container and create a new container.
During this time, any user cannot able to access the application.
Another thing is if our container is terminated, how we can come to know that it is not working out? How we can create a new container automatically?
Solution:- That is where the ‘container management system’ comes(Kubernetes) into the picture.
In the next Module 6, we are going to use the leverage of the container management system to run our containerized applications with higher availability and quality tolerance.