Convert Any Server to a Docker Container (Updated)

zwischenzugs Uncategorized April 4, 2016August 9, 2017

1 Minute

How and Why?

Let’s say you have a server that has been lovingly hand-crafted that you want to containerize.

Figuring out exactly what software is required on there and what config files need adjustment would be quite a task, but fortunately blueprint exists as a solution to that.

What I’ve done here is automate that process down to a few simple steps. Here’s how it works:

You kick off a ShutIt script (as root) that automates the bash interactions required to get a blueprint copy of your server, then this in turn kicks off another ShutIt script which creates a Docker container that provisions the container with the right stuff, then commits it. Got it? Don’t worry, it’s automated and only a few lines of bash.

There are therefore 3 main steps to getting into your container:

– Install ShutIt on the server

– Run the ‘copyserver’ ShutIt script

– Run your copyserver Docker image as a container

Step 1

Install ShutIt as root:

sudo su -
pip install shutit

The pre-requisites are python-pip, git and docker. The exact names of these in your package manager may vary slightly (eg docker-io or docker.io) depending on your distro.

You may need to make sure the docker server is running too, eg with ‘systemctl start docker’ or ‘service docker start’.

Step 2

Check out the copyserver script:

git clone https://github.com/ianmiell/shutit_copyserver.git

Step 3

Run the copy_server script:

cd shutit_copyserver/bin
./copy_server.sh

There is a prompt to ask what docker base image you want to use. Make sure you use one as close to the original server as possible.

Step 4

Run the built image:

docker run -ti [username]/copyserver /bin/bash

You are now in a practical facsimile of your server within a docker container!

This post is based on material from Docker in Practice, available on Manning’s Early Access Program. Get 39% off with the code: 39miell2

Published by zwischenzugs

View all posts by zwischenzugs

Published April 4, 2016August 9, 2017

18 thoughts on “Convert Any Server to a Docker Container (Updated)”

Pingback: Convert Any Server to a Docker Container | zwischenzugs
Aly Saleh

August 3, 2016 at 7:56 pm

I am using amazon linux and I got this error:
Pause point on fail: Could not determine Linux distro information. Please inform ShutIt maintainers.

Reply
Aly Saleh

August 11, 2016 at 3:42 pm

which linux distro/version are tested and confirmed to work? -thanks.

Reply
Aly Saleh

August 13, 2016 at 2:36 pm

I tried it with centos 7 but got this error:

[root@ip-10-66-118-190 centos]# cd shutit_copyserver/bin
[root@ip-10-66-118-190 bin]# ./copy_server.sh
/home/centos/shutit_copyserver /home/centos/shutit_copyserver/bin
# ShutIt Started…
# Loading configs…
2016-08-13 14:34:12,551 INFO: ShutIt modules loaded
2016-08-13 14:34:12,552 INFO: Configuration loaded
2016-08-13 14:34:12,618 INFO: Connected to target

Please input a value for shutit.shutit_copyserver.shutit_copyserver.shutit_copyserver.base_image

Please input an appropriate docker base image, eg:
ubuntu
ubuntu:12.04
centos

You can use any base image you want, but be aware that non-apt or yum-based images are less likely to succeed.:
centos
2016-08-13 14:34:15,007 INFO: Considering whether to build: shutit.tk.setup
2016-08-13 14:34:15,087 INFO: Building ShutIt module: shutit.tk.setup with run order: 0
2016-08-13 14:34:15,090 INFO: Considering whether to build: shutit.shutit_copyserver.shutit_copyserver.shutit_copyserver
2016-08-13 14:34:15,177 INFO: Building ShutIt module: shutit.shutit_copyserver.shutit_copyserver.shutit_copyserver with run order: 1161086295
2016-08-13 14:34:15,177 INFO: Matching output from: “whoami” to one of these regexps:root
2016-08-13 14:34:15,179 INFO: Installing package: python-pip
2016-08-13 14:34:15,181 INFO: Sending: yum install -y python-pip
2016-08-13 14:34:18,776 INFO: Output (squashed): Loaded plugins: fastestmirrorL […]
2016-08-13 14:34:18,777 INFO: Installing package: git
2016-08-13 14:34:18,779 INFO: Sending: yum install -y git
2016-08-13 14:34:22,255 INFO: Output (squashed): Loaded plugins: fastestmirrorL […]
2016-08-13 14:34:22,256 INFO: Matching output from: “git config -l” to one of these regexps:user.email
2016-08-13 14:34:22,259 INFO: Matching output from: “git config -l” to one of these regexps:user.name
2016-08-13 14:34:22,261 INFO: Sending: rm -rf /tmp/shutit_copyserver
2016-08-13 14:34:22,263 INFO: Output (squashed): shutit.shutit_copyserver.shuti […]
2016-08-13 14:34:22,264 INFO: Sending: pip install blueprint
2016-08-13 14:34:22,539 INFO: Output (squashed): Requirement already satisfied […]
2016-08-13 14:34:22,540 INFO: Sending: echo ‘:package:apt/docker’ >> /etc/blueprintignore
2016-08-13 14:34:22,542 INFO: Output (squashed): shutit.shutit_copyserver.shuti […]
2016-08-13 14:34:22,543 INFO: Sending: echo ‘:package:apt/docker.io’ >> /etc/blueprintignore
2016-08-13 14:34:22,544 INFO: Output (squashed): shutit.shutit_copyserver.shuti […]
2016-08-13 14:34:22,545 INFO: Sending: echo ‘:package:apt/docker-engine’ >> /etc/blueprintignore
2016-08-13 14:34:22,547 INFO: Output (squashed): shutit.shutit_copyserver.shuti […]
2016-08-13 14:34:22,548 INFO: Sending: echo ‘:package:apt/virtualbox’ >> /etc/blueprintignore
2016-08-13 14:34:22,550 INFO: Output (squashed): shutit.shutit_copyserver.shuti […]
2016-08-13 14:34:22,551 INFO: Sending: echo ‘:package:apt/virtualbox-dkms’ >> /etc/blueprintignore
2016-08-13 14:34:22,553 INFO: Output (squashed): shutit.shutit_copyserver.shuti […]
2016-08-13 14:34:22,554 INFO: Sending: echo ‘:package:apt/virtualbox-qt’ >> /etc/blueprintignore
2016-08-13 14:34:22,556 INFO: Output (squashed): shutit.shutit_copyserver.shuti […]
Traceback (most recent call last):
File “/bin/shutit”, line 11, in
sys.exit(main())
File “/usr/lib/python2.7/site-packages/shutit_main.py”, line 609, in main
do_build()
File “/usr/lib/python2.7/site-packages/shutit_main.py”, line 421, in do_build
build_module(module)
File “/usr/lib/python2.7/site-packages/shutit_main.py”, line 362, in build_module
if not module.build(shutit):
File “/usr/lib/python2.7/site-packages/shutit_module.py”, line 58, in wrapper
ret = func(self, shutit)
File “/home/centos/shutit_copyserver/shutit_copyserver.py”, line 80, in build
shutit.send(shutit.cfg[‘host’][‘shutit_path’] + ‘/shutit skeleton –output_dir –base_image ‘ + shutit.cfg[self.module_id][‘base_image’] + ‘ –template_branch docker –module_directory /tmp/shutit_copyserver –module_name copied_server –domain shutit.copied_server –delivery docker’)
KeyError: ‘host’
/home/centos/shutit_copyserver/bin
[root@ip-10-66-118-190 bin]#

Reply
1. zwischenzugs
  
  June 17, 2017 at 2:29 pm
  
  After a busy few months I have some time to look at this. There are a couple of bugs I’m fixing, and testing now.
  
  Reply
zwischenzugs

August 13, 2016 at 3:10 pm

Thanks for this – I’ve pushed a change which should help.

Reply
1. Aly Saleh
  
  August 13, 2016 at 4:10 pm
  
  I pulled the update and retried again, but got this new error, please check:
  
  [root@ip-10-66-115-25 centos]# cd shutit_copyserver/bin
  [root@ip-10-66-115-25 bin]# ./copy_server.sh
  /home/centos/shutit_copyserver /home/centos/shutit_copyserver/bin
  # ShutIt Started…
  # Loading configs…
  2016-08-13 16:07:36,712 INFO: Correcting insecure file permissions on: configs/build.cnf
  2016-08-13 16:07:36,721 INFO: ShutIt modules loaded
  2016-08-13 16:07:36,722 INFO: Configuration loaded
  2016-08-13 16:07:36,787 INFO: Connected to target
  
  Please input a value for shutit.shutit_copyserver.shutit_copyserver.shutit_copyserver.base_image
  
  Please input an appropriate docker base image, eg:
  ubuntu
  ubuntu:12.04
  centos
  
  You can use any base image you want, but be aware that non-apt or yum-based images are less likely to succeed.:
  centos
  2016-08-13 16:07:39,115 INFO: Considering whether to build: shutit.tk.setup
  2016-08-13 16:07:39,196 INFO: Building ShutIt module: shutit.tk.setup with run order: 0
  2016-08-13 16:07:39,198 INFO: Considering whether to build: shutit.shutit_copyserver.shutit_copyserver.shutit_copyserver
  2016-08-13 16:07:39,284 INFO: Building ShutIt module: shutit.shutit_copyserver.shutit_copyserver.shutit_copyserver with run order: 1161086295
  2016-08-13 16:07:39,284 INFO: Matching output from: “whoami” to one of these regexps:root
  2016-08-13 16:07:39,288 INFO: Installing package: python-pip
  2016-08-13 16:07:39,289 INFO: Sending: yum install -y python-pip
  2016-08-13 16:07:42,803 INFO: Output (squashed): Loaded plugins: fastestmirrorL […]
  2016-08-13 16:07:42,804 INFO: Installing package: git
  2016-08-13 16:07:42,806 INFO: Sending: yum install -y git
  2016-08-13 16:07:46,258 INFO: Output (squashed): Loaded plugins: fastestmirrorL […]
  2016-08-13 16:07:46,259 INFO: Matching output from: “git config -l” to one of these regexps:user.email
  2016-08-13 16:07:46,261 INFO: Sending: git config –global user.email “[email protected]”
  2016-08-13 16:07:46,264 INFO: Output (squashed): shutit.shutit_copyserver.shuti […]
  2016-08-13 16:07:46,265 INFO: Matching output from: “git config -l” to one of these regexps:user.name
  2016-08-13 16:07:46,267 INFO: Sending: git config –global user.name “ShutIt”
  2016-08-13 16:07:46,270 INFO: Output (squashed): shutit.shutit_copyserver.shuti […]
  2016-08-13 16:07:46,271 INFO: Sending: rm -rf /tmp/shutit_copyserver
  2016-08-13 16:07:46,273 INFO: Output (squashed): shutit.shutit_copyserver.shuti […]
  2016-08-13 16:07:46,274 INFO: Sending: pip install blueprint
  2016-08-13 16:07:47,045 INFO: Output (squashed): Collecting blueprint Download […]
  2016-08-13 16:07:47,046 INFO: Sending: echo ‘:package:apt/docker’ >> /etc/blueprintignore
  2016-08-13 16:07:47,048 INFO: Output (squashed): shutit.shutit_copyserver.shuti […]
  2016-08-13 16:07:47,049 INFO: Sending: echo ‘:package:apt/docker.io’ >> /etc/blueprintignore
  2016-08-13 16:07:47,050 INFO: Output (squashed): shutit.shutit_copyserver.shuti […]
  2016-08-13 16:07:47,051 INFO: Sending: echo ‘:package:apt/docker-engine’ >> /etc/blueprintignore
  2016-08-13 16:07:47,053 INFO: Output (squashed): shutit.shutit_copyserver.shuti […]
  2016-08-13 16:07:47,054 INFO: Sending: echo ‘:package:apt/virtualbox’ >> /etc/blueprintignore
  2016-08-13 16:07:47,056 INFO: Output (squashed): shutit.shutit_copyserver.shuti […]
  2016-08-13 16:07:47,057 INFO: Sending: echo ‘:package:apt/virtualbox-dkms’ >> /etc/blueprintignore
  2016-08-13 16:07:47,059 INFO: Output (squashed): shutit.shutit_copyserver.shuti […]
  2016-08-13 16:07:47,060 INFO: Sending: echo ‘:package:apt/virtualbox-qt’ >> /etc/blueprintignore
  2016-08-13 16:07:47,062 INFO: Output (squashed): shutit.shutit_copyserver.shuti […]
  2016-08-13 16:07:47,063 INFO: Sending: shutit skeleton –output_dir –base_image centos –pattern docker –name /tmp/shutit_copyserver –domain shutit.copied_server –delivery docker
  2016-08-13 16:07:47,263 INFO: Output (squashed): # ShutIt Started… # […]
  2016-08-13 16:07:47,264 INFO: Sending: cd /tmp/shutit_copyserver
  2016-08-13 16:07:47,266 INFO: Output (squashed): shutit.shutit_copyserver.shuti […]
  2016-08-13 16:07:47,267 INFO: Sending: blueprint create shutit_copyserver
  2016-08-13 16:08:06,971 INFO: Output (squashed): # [blueprint] searching for AP […]
  2016-08-13 16:08:06,974 INFO: Fixing up your terminal, please wait…
  
  ================================================================================
  
  WARNING: command:
  blueprint create shutit_copyserver
  returned unaccepted exit code: 1
  If this is expected, pass in check_exit=False or an exit_values array into the send function call.
  
  Interactive, so not retrying.
  Pause point on exit_code != 0 (1). CTRL-C to quit
  ================================================================================
  
  You now have a standard shell. Hit CTRL and then ] at the same time to continue ShutIt run.
  /tmp/shutit_fixterm
  shutit.shutit_copyserver.shutit_copyserver.shutit_copyserver#cHAQmuDJ>
  
  Reply
  1. zwischenzugs
    
    August 13, 2016 at 9:37 pm
    
    OK,so this time can you post the output of the same command with -l debug (btw this is probably better pursued in a github issue)!
Aly Saleh

August 15, 2016 at 12:27 pm

Thanks a lot for the help, I created a new git issue and added the debug output here:
https://github.com/ianmiell/shutit_copyserver/issues/3

Reply
Pingback: Containerizing Legacy Applications :: qualimente
Toko

May 3, 2017 at 2:43 pm

Hi! I have tried to create the copy of my server (Ubuntu 16.04.2) but the script fails with the following output. Can you please help me?

root@sputnik:~/shutit_copyserver/bin# ./copy_server.sh
~/shutit_copyserver ~/shutit_copyserver/bin
# ShutIt Started…
# Loading configs…
2017-05-03 16:38:59,270 INFO: Correcting insecure file permissions on: configs/build.cnf
2017-05-03 16:38:59,282 INFO: ShutIt modules loaded
2017-05-03 16:38:59,283 INFO: Configuration loaded
2017-05-03 16:39:01,530 INFO: Connected to target

Please input a value for shutit.shutit_copyserver.shutit_copyserver.shutit_copyserver.base_image

Please input an appropriate docker base image, eg:
ubuntu
ubuntu:12.04
centos

You can use any base image you want, but be aware that non-apt or yum-based images are less likely to succeed.:
ubuntu
2017-05-03 16:39:16,880 INFO: Considering whether to build: shutit.tk.setup
2017-05-03 16:39:18,218 INFO: Building ShutIt module: shutit.tk.setup with run order: 0
2017-05-03 16:39:18,372 INFO: Considering whether to build: shutit.shutit_copyserver.shutit_copyserver.shutit_copyserver
2017-05-03 16:39:19,740 INFO: Building ShutIt module: shutit.shutit_copyserver.shutit_copyserver.shutit_copyserver with run order: 1161086295
2017-05-03 16:39:19,740 INFO: Matching output from: “whoami” to one of these regexps:root
2017-05-03 16:39:19,793 INFO: Installing package: python-pip
2017-05-03 16:39:19,896 INFO: python-pip already installed.
2017-05-03 16:39:19,896 INFO: Installing package: git
2017-05-03 16:39:19,995 INFO: git already installed.
2017-05-03 16:39:19,996 INFO: Matching output from: “git config -l” to one of these regexps:user.email
2017-05-03 16:39:20,049 INFO: Matching output from: “git config -l” to one of these regexps:user.name
2017-05-03 16:39:20,103 INFO: Sending: rm -rf /tmp/shutit_copyserver
2017-05-03 16:39:20,186 INFO: Output (squashed): sputnik:shutit.shutit_copyserver.shutit_copyserver.shutit_copyserver:Koy9EA0K#
2017-05-03 16:39:20,238 INFO: Sending: pip install blueprint
2017-05-03 16:39:20,784 INFO: Output (squashed): Requirement already satisfied: blueprint in /usr/local/lib/python2.7/dist-packagessputnik:shutit.shutit_copyserver.shutit_copyserver.shutit_copyserver:Koy9EA0K# […]
2017-05-03 16:39:20,836 INFO: Sending: echo ‘:package:apt/docker’ >> /etc/blueprintignore
2017-05-03 16:39:20,890 INFO: Output (squashed): sputnik:shutit.shutit_copyserver.shutit_copyserver.shutit_copyserver:Koy9EA0K#
2017-05-03 16:39:20,942 INFO: Sending: echo ‘:package:apt/docker.io’ >> /etc/blueprintignore
2017-05-03 16:39:20,996 INFO: Output (squashed): sputnik:shutit.shutit_copyserver.shutit_copyserver.shutit_copyserver:Koy9EA0K#
2017-05-03 16:39:21,048 INFO: Sending: echo ‘:package:apt/docker-engine’ >> /etc/blueprintignore
2017-05-03 16:39:21,103 INFO: Output (squashed): sputnik:shutit.shutit_copyserver.shutit_copyserver.shutit_copyserver:Koy9EA0K#
2017-05-03 16:39:21,155 INFO: Sending: echo ‘:package:apt/virtualbox’ >> /etc/blueprintignore
2017-05-03 16:39:21,209 INFO: Output (squashed): sputnik:shutit.shutit_copyserver.shutit_copyserver.shutit_copyserver:Koy9EA0K#
2017-05-03 16:39:21,261 INFO: Sending: echo ‘:package:apt/virtualbox-dkms’ >> /etc/blueprintignore
2017-05-03 16:39:21,316 INFO: Output (squashed): sputnik:shutit.shutit_copyserver.shutit_copyserver.shutit_copyserver:Koy9EA0K#
2017-05-03 16:39:21,368 INFO: Sending: echo ‘:package:apt/virtualbox-qt’ >> /etc/blueprintignore
2017-05-03 16:39:21,422 INFO: Output (squashed): sputnik:shutit.shutit_copyserver.shutit_copyserver.shutit_copyserver:Koy9EA0K#
2017-05-03 16:39:21,474 INFO: Sending: shutit skeleton –output_dir –base_image ubuntu –pattern docker –name /tmp/shutit_copyserver –domain shutit.copied_server –delivery docker
2017-05-03 16:39:21,768 INFO: Output (squashed): # ShutIt Started… # Loading configs…# Run:cd /tmp/shutit_copyserver && ./build.sh# to build.# And then:./run.sh# to run.sputnik:shutit.shutit_copyserver.shu […]
2017-05-03 16:39:21,820 INFO: Sending: cd /tmp/shutit_copyserver
2017-05-03 16:39:21,872 INFO: Output (squashed): sputnik:shutit.shutit_copyserver.shutit_copyserver.shutit_copyserver:Koy9EA0K#
2017-05-03 16:39:21,924 INFO: Sending: blueprint create shutit_copyserver
2017-05-03 16:41:17,936 INFO: Output (squashed): # [blueprint] parsing blueprintignore(5) rulesdpkg-query: Kein Paket gefunden, das auf python3:any passtdpkg-query: Kein Paket gefunden, das auf docker-engine p […]
2017-05-03 16:41:17,987 INFO: Sending: blueprint show -S shutit_copyserver
2017-05-03 16:41:19,067 INFO: Output (squashed): shutit_copyserver/bootstrap.shsputnik:shutit.shutit_copyserver.shutit_copyserver.shutit_copyserver:Koy9EA0K#
2017-05-03 16:41:19,118 INFO: Sending: sed -i ‘s/^…n “$SERVICE_.*$//g’ shutit_copyserver/bootstrap.sh
2017-05-03 16:41:19,212 INFO: Output (squashed): sputnik:shutit.shutit_copyserver.shutit_copyserver.shutit_copyserver:Koy9EA0K#
2017-05-03 16:41:26,349 INFO: ShutIt run finished

Reply
1. zwischenzugs
  
  June 17, 2017 at 2:29 pm
  
  After a busy few months I have some time to look at this. There are a couple of bugs I’m fixing, and testing now.
  
  Reply
passw0rd

June 7, 2017 at 3:26 pm

Just tried to run through this on one of our ec2 instances and I ran into the following problem. Any ideas?

Thanks in advance!

[root@ip-172-31-8-239 bin]# ./copy_server.sh
~/shutit_copyserver ~/shutit_copyserver/bin
/usr/lib/python2.6/dist-packages/shutit_pexpect.py:46: DeprecationWarning: the md5 module is deprecated; use hashlib instead
from md5 import md5
Traceback (most recent call last):
File “/usr/bin/shutit”, line 11, in
sys.exit(main())
File “/usr/lib/python2.6/dist-packages/shutit.py”, line 50, in main
if sys.version_info.major == 2:
AttributeError: ‘tuple’ object has no attribute ‘major’
~/shutit_copyserver/bin

Reply
1. zwischenzugs
  
  June 17, 2017 at 2:28 pm
  
  After a busy few months I have some time to look at this. At first glance though, shutit only supports 2.7+.
  
  Reply
Ajith

October 4, 2017 at 2:07 pm

Hi zwischenzugs,

I am trying to run ./copy_server.sh. But getting the below error. I am running it in an amazonlinux system (yum). Is there an issue due to this OS? Please help.

ShutIt version: 1.0.27
Linux ip-10-0-2-39 3.14.42-31.38.amzn1.x86_64 #1 SMP Wed May 13 20:33:05 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

2017-10-04 11:50:18,809 INFO: Installing package: python-pip
Traceback (most recent call last):
File “/usr/local/bin/shutit”, line 11, in
sys.exit(main())
File “/usr/local/lib/python2.7/site-packages/shutit.py”, line 67, in main
shutit.setup_shutit_obj()
File “/usr/local/lib/python2.7/site-packages/shutit_class.py”, line 3942, in setup_shutit_obj
self.do_build()
File “/usr/local/lib/python2.7/site-packages/shutit_class.py”, line 4181, in do_build
self.build_module(module)
File “/usr/local/lib/python2.7/site-packages/shutit_class.py”, line 4123, in build_module
if not module.build(self):
File “/usr/local/lib/python2.7/site-packages/shutit_module.py”, line 59, in wrapper
ret = func(self, shutit)
File “/opt/shutit_copyserver/shutit_copyserver.py”, line 66, in build
shutit.install(‘python-pip’)
File “/usr/local/lib/python2.7/site-packages/shutit_class.py”, line 1798, in install
loglevel=loglevel)
File “/usr/local/lib/python2.7/site-packages/shutit_pexpect.py”, line 1393, in install
package = package_map.map_packages(package, self.current_environment.install_type)
File “/usr/local/lib/python2.7/site-packages/package_map.py”, line 108, in map_packages
res = res + ‘ ‘ + map_package(package,install_type)
TypeError: cannot concatenate ‘str’ and ‘function’ objects

Reply
Pingback: Package atomic-registries requires python-pytoml error installing docker on CentOS 7 | Bits and Dragons

Convert Any Server to a Docker Container (Updated)

Convert Any Server to a Docker Container (Updated)

How and Why?

Step 1

Step 2

Step 3

Step 4

Published by zwischenzugs

Post navigation

18 thoughts on “Convert Any Server to a Docker Container (Updated)”

Leave a Reply Cancel reply

Recommend

Play With Kubernetes Quickly Using Docker (Updated)

Interactive Git Tutorials – Rebase and Bisect

ssh -R (reverse tunnel) man page hell

Power ‘git log’ graphing

A Quick Tour of Docker 1.12

A checklist for Docker in the Enterprise

Learn Kubernetes the Hard Way (the Easy and Cheap Way)

A Complete OpenShift Cluster on Vagrant, Step by Step

Migrating an OpenShift etcd Cluster

1-Minute Multi-Node VM Setup

About Joyk