10

Simple Continuous Deployment System with Jenkins and Github

 3 years ago
source link: http://www.linux-admins.net/2016/04/simple-continuous-deployment-system.html
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
neoserver,ios ssh client

Simple Continuous Deployment System with Jenkins and Github

In this post I'll demonstrate a simple CD system, that based on a merge to the Master branch for a project on Github will package, push and deploy a small RESTfull application.

The setup consists of a single Jenkins server, a load balancer node running HAProxy and two API servers running Apache with the libapache2-mod-wsgi module to drive the Python Bottle microframework.

First let's create the local git repository:

[workstation]$ mkdir simple_rest_ci_pipline && cd simple_rest_ci_pipline [workstation]$ echo "# Simple RESTfull service" >> README.md [workstation]$ touch .gitignore [workstation]$ mkdir project [workstation]$ cat << EOF > project/restfullapi.py from bottle import route

@route('/<command>') def ping(command=None): if command.lower() == "ping": return "pong!" else: return "Unknown command"

@route('/') def main(): return "Simple RESTfull API: curl -X GET $URL/ping" EOF

[workstation]$ cat << EOF > project/restfullapi.wsgi import sys

sys.path.insert(0, "/var/www/restfullapi")

import bottle import restfullapi application = bottle.default_app() EOF

[workstation]$ git init [workstation]$ git remote add origin [email protected]:someuser/simple_rest_ci_pipline.git

Create a new project on Github, and deploy your public key to it:

Github%2BDeploy%2BKeys.png


Next, test that you can connect using the deployed SSH key and push to the Master branch:

[workstation]$ ssh -T [email protected] [workstation]$ git ls-remote -h [email protected]:someuser/simple_rest_ci_pipline.git [workstation]$ git add * [workstation]$ git commit -m "Initializing the repository" [workstation]$ git push -u origin master

Let's build the Jenkins server:

[jenkins-n01]$ wget -q -O - https://jenkins-ci.org/debian/jenkins-ci.org.key | sudo apt-key add - [jenkins-n01]$ sh -c 'echo deb http://pkg.jenkins-ci.org/debian binary/ > /etc/apt/sources.list.d/jenkins.list' [jenkins-n01]$ apt-get update && apt-get -y install jenkins git [jenkins-n01]$ /etc/init.d/jenkins start [jenkins-n01]$ iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-port 8080

Once you login to Jenkins create a new project:
Jenkins%2BScreen%2B1.png
Jenkins%2BScreen%2B2.png
Jenkins%2BScreen%2B3.png
Or you can copy the following config file to /var/lib/jenkins/jobs/RESTfull-CI-Pipeline:

[jenkins-n01]$ cd ~/jobs/RESTfull-CI-Pipeline [jenkins-n01]$ cat config.xml

<?xml version='1.0' encoding='UTF-8'?> <project> <actions/> <description></description> <keepDependencies>false</keepDependencies> <properties> <com.coravy.hudson.plugins.github.GithubProjectProperty plugin="[email protected]"> <projectUrl>https://github.com/kaivanov/simple_rest_ci_pipline/</projectUrl> <displayName></displayName> </com.coravy.hudson.plugins.github.GithubProjectProperty> </properties> <scm class="hudson.plugins.git.GitSCM" plugin="[email protected]"> <configVersion>2</configVersion> <userRemoteConfigs> <hudson.plugins.git.UserRemoteConfig> <url>https://github.com/kaivanov/simple_rest_ci_pipline.git</url> </hudson.plugins.git.UserRemoteConfig> </userRemoteConfigs> <branches> <hudson.plugins.git.BranchSpec> <name>*/master</name> </hudson.plugins.git.BranchSpec> </branches> <doGenerateSubmoduleConfigurations>false</doGenerateSubmoduleConfigurations> <submoduleCfg class="list"/> <extensions/> </scm> <canRoam>true</canRoam> <disabled>false</disabled> <blockBuildWhenDownstreamBuilding>false</blockBuildWhenDownstreamBuilding> <blockBuildWhenUpstreamBuilding>false</blockBuildWhenUpstreamBuilding> <triggers> <com.cloudbees.jenkins.GitHubPushTrigger plugin="[email protected]"> <spec></spec> </com.cloudbees.jenkins.GitHubPushTrigger> </triggers> <concurrentBuild>false</concurrentBuild> <builders> <hudson.tasks.Shell> <command>PACKAGE_NAME="simple-restfull-api.deb" PACKAGE_VERSION=$BUILD_NUMBER API1="XXX.XXX.XXX.XXX" API2="XXX.XXX.XXX>XXX"

# Create the packe hierarchy and control file mkdir -p PACKAGE/DEBIAN mkdir -p PACKAGE/var/www/restfullapi

cat > PACKAGE/DEBIAN/control <<EOF Package: simple-restfull-api Version: $PACKAGE_VERSION Priority: optional Architecture: all Maintainer: Konstantin Ivanov Description: Simple RESTfull API EOF

cp $WORKSPACE/project/* $WORKSPACE/PACKAGE/var/www/restfullapi/

# Build the package dpkg-deb --build PACKAGE mv PACKAGE.deb $PACKAGE_NAME

# Deploy the package. rsync --rsync-path="sudo rsync" -vaz $PACKAGE_NAME $API1:/tmp ssh -o stricthostkeychecking=no -o UserKnownHostsFile=/dev/null $API1 "sudo dpkg --install /tmp/$PACKAGE_NAME && sudo service apache2 reload"

if [ "$?" -eq 0 ] then rsync --rsync-path="sudo rsync" -vaz $PACKAGE_NAME $API2:/tmp ssh -o stricthostkeychecking=no -o UserKnownHostsFile=/dev/null $API2 "sudo dpkg --install /tmp/$PACKAGE_NAME && sudo service apache2 reload"

if [ "$?" -ne 0 ] then echo "Error deploying to node2, aborting ..." exit 1 fi else echo "Error deploying to node1, aborting ..." exit 1 fi</command> </hudson.tasks.Shell> </builders> <publishers/> <buildWrappers> <hudson.plugins.ws__cleanup.PreBuildCleanup plugin="[email protected]"> <deleteDirs>false</deleteDirs> <cleanupParameter></cleanupParameter> <externalDelete></externalDelete> </hudson.plugins.ws__cleanup.PreBuildCleanup> </buildWrappers> </project>

Make sure you have the github plugin installed on Jenkins for this to work.

Lastly on the Github side, add a service that will call the Jenkins API when an event is triggered e.g. merge to Master:

Github%2BWebhook.png
Time to setup the API nodes:

[api-n01/02]$ useradd -m -s /bin/bash jenkins [api-n01/02]$ sh -c 'echo jenkins ALL=(ALL:ALL) NOPASSWD:ALL > /etc/sudoers.d/100-ci-users' [api-n01/02]$ apt-get update [api-n01/02]$ apt-get install apache2 [api-n01/02]$ apt-get install python-pip [api-n01/02]$ pip install bottle [api-n01/02]$ apt-get install libapache2-mod-wsgi [api-n01/02]$ cat << EOF > /etc/apache2/sites-available/000-default.conf <VirtualHost *:80> ServerName restfullapi-n01.example.net

WSGIDaemonProcess restfullapi user=www-data group=www-data processes=1 threads=1 WSGIScriptAlias / /var/www/restfullapi/restfullapi.wsgi

<Directory /var/www/restfullapi> WSGIProcessGroup restfullapi WSGIApplicationGroup %{GLOBAL} Order deny,allow Allow from all </Directory> </VirtualHost> EOF

And finally the load balancer node:

[lb-n01]$ apt-get update && apt-get install haproxy [lb-n01]$ cat /etc/haproxy/haproxy.cfg global log /dev/log local1 log /dev/log local1 notice user haproxy group haproxy daemon

defaults log global mode http option httplog option dontlognull contimeout 5000 clitimeout 50000 srvtimeout 50000 errorfile 400 /etc/haproxy/errors/400.http errorfile 403 /etc/haproxy/errors/403.http errorfile 408 /etc/haproxy/errors/408.http errorfile 500 /etc/haproxy/errors/500.http errorfile 502 /etc/haproxy/errors/502.http errorfile 503 /etc/haproxy/errors/503.http errorfile 504 /etc/haproxy/errors/504.http

frontend http bind :80 reqadd X-Forwarded-Proto:\ http default_backend http_nodes

backend http_nodes mode http balance leastconn option httpclose option forwardfor option redispatch option httpchk GET / cookie JSESSIONID prefix server api-n01 restfullapi-n01.example.net:80 check inter 1000 server api-n02 restfullapi-n02.example.net:80 check inter 1000

Now every time you merge a commit to Master, Github will POST to the Jenkins API, and Jenkins will execute the job, building a debian package, and deploying it to the API nodes.

To test the simple RESTfull API run:

[workstation]$ curl -X GET http://restfullapi.example.net/ping pong!


report

Recommend

About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK