Minimalistic docker: starting from scratch

Introduction
Next generation Linux distributions are out. Significant changes create special flavour by comparing to other unix-like derivatives.
In spite of implementation of similar features, the distance and the diversity
are growing. Sounds good, isn’t? Not all share this opinion.

Docker is one result of the actual virtualization trends on Linux. The control
groups (aka cgroups) realize features which make sense to use operating
system level virtualization, the one kernel many containers scenario.

Conditions, environment
X86 virtual machine deployed on VMware ESXi 5.5 on HP DL360Pg8.
Minimal Oracle Enterprise Linux 7 installed on 2 CPU cores and 4 GB.
No RAID on VM level, LVM and xfs for the general filesystems (default).
Extra softwares added: screen, wget and vim.
EL7 contains systemd and the Oracle kernel is used 3.8.13-44.el7uek.x86_64, dtrace is available.

First steps and look
Installing and configuring docker on OEL7 is quite easy.

1.) Addon repo has to be enabled:

# wget http://public-yum.oracle.com/public-yum-ol7.repo
# mv /etc/yum.repos.d/public-yum-ol7.repo /etc/yum.repos.d/public-yum-ol7.repo.orig
# mv public-yum-ol7.repo /etc/yum.repos.d/
# vim /etc/yum.repos.d/public-yum-ol7.repo

[ol7_addons]
name=Oracle Linux $releasever Add ons ($basearch)
baseurl=http://public-yum.oracle.com/repo/OracleLinux/OL7/addons/$basearch/
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
gpgcheck=1
enabled=1

2.) Install docker:

# yum install docker -y

3.) Start, check end enable rebbot persistent service:

# systemctl start docker.service
# systemctl status docker.service
docker.service – Docker Application Container Engine
Loaded: loaded (/usr/lib/systemd/system/docker.service; disabled)
Active: active (running) since Tue 2014-09-09 01:59:24 CEST; 17s ago
Docs: http://docs.docker.io
Main PID: 10332 (docker)
CGroup: /system.slice/docker.service
L¦10332 /usr/bin/docker -d –selinux-enabled

Sep 09 01:59:24 bone docker[10332]: [b9661965.init_networkdriver()] creating new bridge for docker0
Sep 09 01:59:24 bone docker[10332]: [b9661965.init_networkdriver()] getting iface addr
Sep 09 01:59:24 bone docker[10332]: [b9661965] -job init_networkdriver() = OK (0)
Sep 09 01:59:24 bone docker[10332]: Loading containers: : done.
Sep 09 01:59:24 bone docker[10332]: [b9661965.initserver()] Creating pidfile
Sep 09 01:59:24 bone docker[10332]: [b9661965.initserver()] Setting up signal traps
Sep 09 01:59:24 bone docker[10332]: [b9661965] -job initserver() = OK (0)
Sep 09 01:59:24 bone docker[10332]: [b9661965] +job acceptconnections()
Sep 09 01:59:24 bone docker[10332]: [b9661965] -job acceptconnections() = OK (0)
Sep 09 01:59:24 bone systemd[1]: Started Docker Application Container Engine.

# systemctl enable docker.service

Quick wins
1.) Login to the https://registry.hub.docker.com (modifications can be pushed back also)

# docker login
Username: miamxxx
Password:
Email: miamxxx@xxxxx.com
Login Succeeded

2.) Getting and starting the Ubuntu ecosystem incredible quickly, few hundreds MB of data:

# docker run -t -i ubuntu:14.04 /bin/bash
Unable to find image ‘ubuntu:14.04′ locally
Pulling repository ubuntu
826544226fdc: Download complete
511136ea3c5a: Download complete
b3553b91f79f: Download complete
ca63a3899a99: Download complete
ff01d67c9471: Download complete
7428bd008763: Download complete
c7c7108e0ad8: Download complete

Under the hood

1.) Release vs kernel:

root@5279d057869b:/# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 14.04.1 LTS
Release: 14.04
Codename: trusty
root@5279d057869b:/# uname -a
Linux 5279d057869b 3.8.13-44.el7uek.x86_64 #2 SMP Fri Aug 8 20:46:42 PDT 2014 x86_64 x86_64 x86_64 GNU/Linux
root@5279d057869b:/#

2.) Processes and mounts:

root@5279d057869b:/# ps auxw
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.3 0.0 18152 1844 ? Ss 01:34 0:00 /bin/bash
root 8 0.0 0.0 15568 1128 ? R+ 01:35 0:00 ps auxw

root@5279d057869b:/# mount
/dev/mapper/docker-252:1-467273-5279d057869b4bbd92b0a2fa90182efa59d88423e9c2d4a7d5487d042787e6c1 on / type ext4 (rw,relatime,context=”system_u:object_r:svirt_sandbox_file_t:s0:c73,c434″,discard,stripe=16,data=ordered)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
sysfs on /sys type sysfs (ro,relatime,seclabel)
tmpfs on /dev type tmpfs (rw,nosuid,context=”system_u:object_r:svirt_sandbox_file_t:s0:c73,c434″,mode=755)
shm on /dev/shm type tmpfs (rw,nosuid,nodev,noexec,relatime,context=”system_u:object_r:svirt_sandbox_file_t:s0:c73,c434″,size=65536k)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,context=”system_u:object_r:svirt_sandbox_file_t:s0:c73,c434″,gid=5,mode=620,ptmxmode=666)
/dev/mapper/ol-root on /.dockerinit type xfs (ro,relatime,seclabel,attr2,inode64,noquota)
/dev/mapper/ol-root on /etc/resolv.conf type xfs (ro,relatime,seclabel,attr2,inode64,noquota)
/dev/mapper/ol-root on /etc/hostname type xfs (ro,relatime,seclabel,attr2,inode64,noquota)
/dev/mapper/ol-root on /etc/hosts type xfs (ro,relatime,seclabel,attr2,inode64,noquota)
proc on /proc/sys type proc (ro,nosuid,nodev,noexec,relatime)
proc on /proc/sysrq-trigger type proc (ro,nosuid,nodev,noexec,relatime)
proc on /proc/irq type proc (ro,nosuid,nodev,noexec,relatime)
proc on /proc/bus type proc (ro,nosuid,nodev,noexec,relatime)
tmpfs on /proc/kcore type tmpfs (rw,nosuid,context=”system_u:object_r:svirt_sandbox_file_t:s0:c73,c434″,mode=755)

3.) From the host:

# mount|grep docker
/dev/mapper/ol-root on /var/lib/docker type xfs (rw,relatime,seclabel,attr2,inode64,noquota)
/dev/mapper/docker-252:1-467273-5279d057869b4bbd92b0a2fa90182efa59d88423e9c2d4a7d5487d042787e6c1 on /var/lib/docker/devicemapper/mnt/5279d057869b4bbd92b0a2fa90182efa59d88423e9c2d4a7d5487d042787e6c1 type ext4 (rw,relatime,context=”system_u:object_r:svirt_sandbox_file_t:s0:c255,c917″,discard,stripe=16,data=ordered)

# ls -l /var/lib/docker/devicemapper/mnt/5279d057869b4bbd92b0a2fa90182efa59d88423e9c2d4a7d5487d042787e6c1/rootfs/
total 76
drwxr-xr-x. 2 root root 4096 Sep 3 05:33 bin
drwxr-xr-x. 2 root root 4096 Apr 11 00:12 boot
drwxr-xr-x. 4 root root 4096 Sep 14 03:34 dev
drwxr-xr-x. 61 root root 4096 Sep 14 03:34 etc
drwxr-xr-x. 2 root root 4096 Apr 11 00:12 home
drwxr-xr-x. 12 root root 4096 Sep 3 05:33 lib
drwxr-xr-x. 2 root root 4096 Sep 3 05:33 lib64
drwxr-xr-x. 2 root root 4096 Sep 3 05:33 media
drwxr-xr-x. 2 root root 4096 Apr 11 00:12 mnt
drwxr-xr-x. 2 root root 4096 Sep 3 05:33 opt
drwxr-xr-x. 2 root root 4096 Apr 11 00:12 proc
drwx——. 2 root root 4096 Sep 3 05:33 root
drwxr-xr-x. 7 root root 4096 Sep 3 05:33 run
drwxr-xr-x. 2 root root 4096 Sep 4 20:41 sbin
drwxr-xr-x. 2 root root 4096 Sep 3 05:33 srv
drwxr-xr-x. 2 root root 4096 Mar 13 2014 sys
drwxrwxrwt. 2 root root 4096 Sep 4 20:41 tmp
drwxr-xr-x. 10 root root 4096 Sep 3 05:33 usr
drwxr-xr-x. 11 root root 4096 Sep 3 05:33 var

# ps auxw |grep docker
root 10332 0.0 0.6 753260 23660 ? Ssl Sep09 1:47 /usr/bin/docker -d –selinux-enabled
root 23908 0.0 0.1 283944 5488 pts/4 Sl+ 03:34 0:00 docker run -t -i ubuntu:14.04 /bin/bash
root 23983 0.0 0.0 112628 976 pts/1 S+ 03:43 0:00 grep –color=auto docker

# ip a s
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens32: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:6b:2e:30 brd ff:ff:ff:ff:ff:ff
inet 192.168.73.135/23 brd 192.168.73.255 scope global ens32
inet6 fe80::20c:29ff:fe6b:2e30/64 scope link
valid_lft forever preferred_lft forever
3: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
link/ether 56:84:7a:fe:97:99 brd ff:ff:ff:ff:ff:ff
inet 172.17.42.1/16 scope global docker0
inet6 fe80::5484:7aff:fefe:9799/64 scope link
valid_lft forever preferred_lft forever
7: veth1acf: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master docker0 state UP qlen 1000
link/ether 12:e2:f7:51:3a:f6 brd ff:ff:ff:ff:ff:ff
inet6 fe80::10e2:f7ff:fe51:3af6/64 scope link
valid_lft forever preferred_lft forever

From scratch

1.) Create plain docker – virtual size 0 byte:

# tar cv –files-from /dev/null | docker import – scratch
0336ff6e24b9e802deef2bd63e98fbe447c69065d18184a333530c7083018b5c
# docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
scratch latest 0336ff6e24b9 5 seconds ago 0 B

2.) Identifying linked libraries for /bin/sh (bash) and /bin/echo:

# ldd /bin/sh
linux-vdso.so.1 => (0x00007fff24bfe000)
libtinfo.so.5 => /lib64/libtinfo.so.5 (0x00007fd02916c000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007fd028f68000)
libc.so.6 => /lib64/libc.so.6 (0x00007fd028ba8000)
/lib64/ld-linux-x86-64.so.2 (0x00007fd02939c000)
# ldd /bin/echo
linux-vdso.so.1 => (0x00007fffbed81000)
libc.so.6 => /lib64/libc.so.6 (0x00007f74c8c26000)
/lib64/ld-linux-x86-64.so.2 (0x00007f74c8fec000)

3.) Prepare for building the docker image:

# mkdir scratch
# cp /bin/sh scratch/
# cp /bin/echo scratch/
# cp /lib64/libc.so.6 scratch/
# cp /lib64/ld-linux-x86-64.so.2 scratch/
# cp /lib64/libtinfo.so.5 scratch/
# cp /lib64/libdl.so.2 scratch/
# vim Dockerfile
FROM scratch
MAINTAINER Richard REFI <miamano@gmail.com>
ADD ./sh /bin/sh
ADD ./echo /bin/echo
ADD ./libc.so.6 /lib64/libc.so.6
ADD ./ld-linux-x86-64.so.2 /lib64/ld-linux-x86-64.so.2
ADD ./libtinfo.so.5 /lib64/libtinfo.so.5
ADD ./libdl.so.2 /lib64/libdl.so.2
CMD echo *

4.) Building image:

# docker build -t=”miam/scratch:echo” .
Uploading context 3.465 MB
Uploading context
Step 0 : FROM scratch
—> 0336ff6e24b9
Step 1 : MAINTAINER Richard REFI <miamano@gmail.com>
—> Using cache
—> e01bbb56f09e
Step 2 : ADD ./sh /bin/sh
—> Using cache
—> d0fe3890f4eb
Step 3 : ADD ./echo /bin/echo
—> 169f85d77b06
Removing intermediate container b3a7d4df1c19
Step 4 : ADD ./libc.so.6 /lib64/libc.so.6
—> f9bc468210dd
Removing intermediate container c48c609b0f4b
Step 5 : ADD ./ld-linux-x86-64.so.2 /lib64/ld-linux-x86-64.so.2
—> 80eceeab56de
Removing intermediate container 90ca7f4c1585
Step 6 : ADD ./libtinfo.so.5 /lib64/libtinfo.so.5
—> 5275b418bb2d
Removing intermediate container cbf546857862
Step 7 : ADD ./libdl.so.2 /lib64/libdl.so.2
—> 52831114f0f8
Removing intermediate container 9d8a2f302eba
Step 8 : CMD echo *
—> Running in c7de7a293d36
—> ae1f200c8a18
Removing intermediate container c7de7a293d36
Successfully built ae1f200c8a18

5.) Checking the build

# docker run -ti miam/scratch:echo /bin/sh
sh-4.2# echo *
bin dev etc lib64 proc sys

Conclusion

I really like the lightweight approach of the OS level virtualisations. I
designed and deployed many FreeBSD jail and Solaris zone solutions in the past.
Reading the headlines this year, I thougth that the wheel had been reinvented
again with docker. I was wrong. Docker and LXC invented a very fancy and modern way
working flexible with minimal overhead.
The enterprise usage of the docker is strongly depending on the connected ecosystem, how easily
can be deployed a container into differnet public or private cloud.
Openstack and/or Vagrant and/or Puppet etc. must be integrated.
At the end: Keep in mind that data migration can be a challange as in general
in case of docker.

References and useful links

http://en.wikipedia.org/wiki/Operating_system%E2%80%93level_virtualization

https://www.kernel.org/doc/Documentation/cgroups/cgroups.txt

https://linuxcontainers.org/

https://www.docker.io

http://stackoverflow.com/questions/17989306/what-does-docker-add-to-just-plain-lxc

https://github.com/docker/docker

https://blogs.oracle.com/linux/entry/ahoy_cast_off_with_docker

http://www.projectatomic.io/docs/docker-building-rhel-images/

https://coreos.com/

http://blog.xebia.com/2014/07/04/create-the-smallest-possible-docker-container/

http://www.scriptrock.com/articles/docker-vs-vagrant