Docker Cheatsheet Part 3 – Create New Docker Image

In this post, we will see how to create docker images? However, if you would like to revisit some docker cheatsheets then go through these two posts Docker Cheatsheet 1and Cheatsheet 2.

Docker image gets build from dockerfile which contains all the instructions and commands. You can read more about dockerfile from here.

We will create a python program which prints the entered username and finally we will make the same as docker image. So let’s start-

docker run -it ubuntu #it will pull and run ubuntu image in your system.

Picture 1

Now as you are in ubuntu image, run below commands-

apt-get update

apt-get install -y python3

cd /home

Create a python file and write below program in it.

Picture 2

Run the program-

Picture 3

Now come back to host machine and create a new folder.

mkdir dockerprogram

cd dockerprogram

cat > my_pyhton_hello_app.py and copy beloe code

user_input = input(“Enter your username: “)

print(user_input)

Press ctrl+c to come out from editor.

Create dockerfile now.

cat > dockerfile and paste below instruction in it-

FROM ubuntu

RUN apt-get update

RUN apt-get install -y python3

COPY my_python_hello_app.py /home/my_python_hello_app.py

ENTRYPOINT python3 /home/my_python_hello_app.py

Press ctrl+c and come out from the editor.

Now run “docker build . -t ‘image_name'” command to build the image. Usually when you create new image you would like to push the same to docker hub as well but you need to have an account to push your created image. You can only push your image to your account once you associate your account name with your image name. So login to your docker hub account by running below account-

docker login #it will ask your username and password and once authentication is done, run the below commands once again

docker build . -t ‘account_name/image_name’

docker push ‘account_name/image_name’

docker pull ‘account_name/image_name’

docker run -it ‘account_name/image_name’

Picture 5

Docker Cheatsheet Part 2 – Volume Mapping in Docker

Volume mapping is a concept in Dockerization to persist the data generated and stored by docker. By default, all the data created inside the container gets vanished when container no longer exists.
Docker provides two options for container to store files in host machine and that are called – Volumes and Bind Mount

Volumes are stored in a part of the host filesystem which is managed by Docker. Non-Docker processes cannot modify this part of the filesystem.

Bind mounts may be stored anywhere on the host system. Non-Docker processes on the Docker host or a Docker container can modify them at any time.

Volumes can be created by using ‘docker volume create Volume1 . The created volume gets stored within a directory on the docker host. When you mount the volume into a container, this directory is what is mounted into the container. Volume can be mounted into multiple containers simultaneously. When no running container is using a volume, the volume is still available to Docker and is not removed automatically. You can remove unused volumes using ‘docker volume prune’.

Any directory full path on the host machine can be mounted to a container. Bind mounts and volumes can both be mounted into containers using the -v or –volume flag, but the syntax for each is slightly different.

Alright, let’s run Jenkins container with Volume.

Below command will create new Volume-

docker volume create Volume1

Now mount created volume to jenkins container by running below command-

docker run -it -p 8080:8080 -v Volume1:/var/jenkins_home jenkins

Note: Path is not required for create volume to mount, so just give volume name : and container directory path.

Once container is run and up. Lunch Jenkins and configure it.

Stop the Jenkins container by Ctrl+C and rerun same command-

docker run -it -p 8080:8080 -v Volume1:/var/jenkins_home jenkins

This time no need to configure it again as Jenkins data was mapped to Volume1 and it is reading all the configuration from there.

Same thing we will be doing with Bind Mount-

Create New directory anywhere in the system and run below command-

docker run -it -p 8080:8080 -v /Users/mayank.srivastava/Jenkins_Data:/var/jenkins_home jenkins

Note: In Bind Mound, we should give full path of directory of host machine to mount the volume.

Now configure Jenkins and shut down the Jenkins.

Run below command again-

docker run -it -p 8080:8080 -v /Users/mayank.srivastava/Jenkins_Data:/var/jenkins_home jenkins

Now go to web and run Jenkins. It is configured and up for action.

How to install Apache Airflow on Mac?

In this post, we are going to see how we can install and configure Apache Airflow on Mac. So, before starting it, let’s see what Apache Airflow is-

Apache Airflow is a workflow management system developed by Airbnb. Airflow helps to create and schedule their workflow programmatically. Airflow is written in Python language, but you can use any language to automate any tasks. Airflow is designed under the principle of ‘Configuration as code’ so all your configuration will be coded and versioned.

Intro is done and now we will see how we can install and configure it on Mac. Before anything, make sure you have installed python3 on your system.

Open terminal and execute below command-

cd ~

Now you are in root folder. Create a directory by executing below command-

mkdir airflow_leanring && cd airflow_learning

Run below command to install python virtual environment-

sudo pip3 install virtualenv

Now run below commands to create virtual environment and activate the virtual environment-

virtualenv -p python3 airflow_venv

source airflow_venv/bin/activate

you should be seeing (airflow_venv) before your root folder name in terminal like below-

Airflow_1

You can read more about virtual environment from here

Now it’s time to install airflow in this virtual environment by below command-

sudo pip3 install apache-airflow

Create new directory in this directory and give name airflow-

mkdir airflow && cd airflow

Now get the full path of airflow directory by supplying ‘pwd’ command in terminal and copy the same. Next, set the path of airflow directory-

export AIRFLOW_HOME=

In my case it was-

export AIRFLOW_HOME=/Users/My_Username/airflow_learning/airflow

Now come out from airflow directory by running ‘cd ..’ command.

Airflow keeps DAG metadata by default in sqlite db, so we need to initialise the db by below command-

airflow initdb

Now if you are following me then run ‘cd airflow’ command and do ‘ls’ and ‘tree’ commands. You should be seeing the below file structure-

Airflow_2.png

Time has come to run some services and check if above configurations are correct. So, press Ctrl+T and it will new terminal in same window.

Go to your airflow_learning folder in new terminal as well and activate the virtual environment by below command-

source airflow_venv/bin/activate

Set the path again with full airflow path-

export AIRFLOW_HOME=/Users/My_Username/airflow_learning/airflow

Alright, run airflow scheduler command in new window and you should be seeing below output-

Airflow_3.png

Go back to old window and run ‘airflow webserver’ command and you should be seeing below output-

Airflow_4.png

By default, airflow opens 8080 to port so, go to web and type localhost:8080 and get the Apache Airflow.

Airflow_5.png

That’s it for this post. Happy Learning!!

 

Docker Cheatsheet Part 1

Docker Hub:
Docker Hub is public repository for images.

Containers:
Containers are instance of docker image.

Host Machine:
A machine where docker is installed and running.

docker run “Image Name”
Above command run a container from given image. If image is not available in host machine, docker will pull the image from docker hub.

Note: Above command will run your container and stops until any process is running in your container.

docker run “Image Name” sleep 2000
Above command will run the image instance and will awake for 2000 seconds.

docker run –name “Name” “Image Name”
Above command will link a name with your running image instance

docker run -it “Image Name”
Above command will allocate interactive process in order to allocate a tty for the container process. So, with combination of ‘i’ and ‘t’, we attach a host terminal to docker containers for interactive user input and output.

docker run -d “Container Name”
Above command with parameter -d will run container in background. ‘-d’ it calls detach mode.

docker run -e environment_variable_name=variable_vaule “Image Name”

Above command will run the image with specified environment variable. We can find all the environment variable of any image by running docker inspect command.

docker attach “Container Name or ID”
Above command will attach the container.

docker ps
Above command will list down all running containers in host machine.

docker ps -a
Above command will list down all running and exited containers in host machine.

docker stop “Container Name or ID”
Above command will stop container. You can pass multiple containers as well to stop by adding a line space.

docker rm “Container Name or ID”
Above command remove the container from the system. You can pass multiple containers as well to remove by adding a line space.

Note: You can remove running status containers, so stop before removing them.

docker images
Above command will list down all the images available in host machine.

docker rmi “Image Name”
Above command will remove the image from the system. You can pass multiple image name by adding a line space.

Note: We cannot remove images if any instances are running therefore, we must stop and remove instances before removing images from host machine.

docker system prune
Above command will remove all stopped containers, all dangling images, and all unused networks.

docker container prune
Above command will remove all stop container.

docker image prune
Above command will remove all dangling and unused images from the host machine.

docker system prune --volumes
Above command will remove all unused volumes.

docker pull “Image Name”
Above command will pull image from docker hub.

docker run ubuntu
Above command will run/pull ubuntu image from docker hub. By default, docker pull latest version of package but if we require any specific version of package then we can tag them with command like below-
docker run ubuntu:16.04
This is called tagging. You can find any package tags from Docker Hub package page.

docker exec “Container name:myubuntu” cat /etc/*release*
Above exec parameter can use to execute command in running container in background.

docker inspect “Container Name or ID”
Above command will give all the details including Network information of the container. So, if you want to know the internal IP of container, just run above command.

docker logs “Container Name or ID”
Above command will list down the logs of containers, if you want to know.

Port Mapping

It is important concept in Docker and useful when we want to access running container from host machine or outside of host machine. Let’s take Jenkins example, you have to run Jenkins container in your host machine like below-

docker run jenkins

Above command will run the instance of jenkins and make the service up but you won’t be able to access this jenkins instance, though container can connect to outside world but outside world cannot connect. So, port mapping allows outside world to connect container and to do so we must map the container port to host port like below-

docker run -p 80:80 jenkins

You can run docker inspect to know the opened port of Jenkins and then you can map the same with opened port in host machine. Now you if you access jenkins by typing your host IP Address:8080 you will be able to see jenkins home page. Make sure port is open in your host machine to access it.

Install Apache2 on Ubuntu with Ansible

Here is playbook which you can use to Install, Start Server, Uninstall, and remove apache2 unwanted packages from Ubnutu 18.04-

---
- hosts: target1
become: yes
gather_facts: yes

tasks:
- name: install apache webserver in ubuntu server.
apt:
name: apache2
state: present
update_cache: yes
- name: start Apache2 server.
service:
name: apache2
state: started
- name: Get the install apache version
command: 'apache2 -v'
register: version
- debug:
var: version.stdout_lines
- name: Remove Apache server from Ubuntu server.
apt:
name: apache2
state: absent
update_cache: yes
- name: Remove unwanted Apache2 packages from the system.
apt:
autoremove: yes
purge: yes

Command module in Ansible

Many scenarios come while writing plays in Ansible, where we would to like to execute commands on terminal. In such cases, we can use command module in Ansible to run any command on target machine terminal. Most frequent case is to getting version of install any package like apache webserver.

Run below playbook to know more about command module-

---
- hosts: target1
become: yes
gather_facts: yes

tasks:
- name: install apache webserver in ubuntu server.
apt:
name: apache2
state: present
update_cache: yes
- name: Get the install apache version
command: 'apache2 -v'
register: version
- debug:
var: version.stdout_lines

Handlers in Ansible

In very layman’s term, Handlers in Ansible is like listener who listen to any defined task to perform another task. It is same as Ansible task; however, it gets execute once defined task complete which it was following/listening.

Usually we use handlers for second task that might be required after executing first task. It is useful in scenarios like starting/restarting the service after installing some package or updating the config file.

To define the Handlers, we have to use ‘notify’ variable.

In below playbook, I am running ‘docker –version’ command as Task1 in ubuntu and notifying to handlers through notify variable to run another command i.e. ‘ls -l’ –

---
- hosts: target1
become: true
gather_facts: true

tasks:
- name: Run Docker version command
command: 'docker --version'
register: changed1
notify:
- NotificationFromFirstTask

handlers:
- name: NotificationFromFirstTask
command: 'ls -l'

So above playbook is just for example but if I talk real use case then you can follow below task as well-

tasks:
- name: Install Nginx
apt: [‘pkg=nginx’ ‘state=installed’ ‘update_cache=true’]
notify:
- StartNginxService

handlers:
- name: StartNginxService
service: name=nginx state=started