5

Install Graphite and Graphite-Web on Ubuntu 20.04 using Docker

 2 years ago
source link: https://computingforgeeks.com/install-graphite-on-ubuntu-using-docker/
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
neoserver,ios ssh client
Install Graphite and Graphite-Web on Ubuntu 20.04 using Docker

Welcome to our guide that captures the installation of Graphite and Graphite-Web on Ubuntu 20.04. Graphite is an open source monitoring platform designed to be highly available and with support for data visualization. How a user will use Graphite is by writing an application that collects numeric time-series data to be graphed, and sending it to Graphite’s processing backend, carbon, which stores the data in Graphite’s specialized database.

Graphite will:

  1. Store numeric time-series data
  2. Render graphs of this data on demand

There are plenty of tools available that helps you collect and send data to graphite.  Even though it often requires a little code, sending data to Graphite is very simple. Graphite is designed for horizontal scalability, for both the frontend and the backend, meaning you can simply add more machines to the mix to get more throughput.

Graphite consists of 3 software components:

  1. carbon – a Twisted daemon that listens for time-series data
  2. whisper – a simple database library for storing time-series data (similar in design to RRD)
  3. graphite webapp – A Django webapp that renders graphs on-demand using Cairo

This article will show you how you can easily have a running instance of Graphite Server on Ubuntu 20.04 Server

Install Graphite and Graphite-Web on Ubuntu 20.04 using Docker

The easiest and quickest method of running Graphite on Ubuntu 20.04 is by using Docker containers. For this all that is required from you is installation of Docker, pulling container images and starting graphite containers.

Step 1: Update System

Start by updating and upgrading your system.

sudo apt update
sudo apt upgrade -y

After upgrade is done reboot the system.

sudo reboot

Step 2: Install Docker Container Engine

Once the system is rebooted login and install few dependencies.

sudo apt update
sudo apt -y install vim apt-transport-https ca-certificates curl gnupg-agent software-properties-common

Import Docker APT repository GPG Keys.

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

Add Docker CE repository to Ubuntu machine.

sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"

Install Docker CE on Ubuntu 20.04 Server.

sudo apt update
sudo apt install docker-ce docker-ce-cli containerd.io

Add your user account to docker group.

sudo usermod -aG docker $USER
newgrp docker

Verify installation by checking Docker version:

$ docker info
Client:
 Debug Mode: false

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 0
 Server Version: 19.03.13
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 8fba4e9a7d01810a393d5d25a3621dc101981175
 runc version: dc9208a3303feef5b3839f4323d9beb36df0a9dd
 init version: fec3683
 Security Options:
  apparmor
  seccomp
   Profile: default
 Kernel Version: 5.4.0-48-generic
 Operating System: Ubuntu 20.04.1 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 1
 Total Memory: 1.894GiB
 Name: ubuntu
 ID: ZJXE:T7RY:FSAV:CRBN:Q6Y7:YQQA:2ELU:VH25:P6PJ:6X6B:3WTC:EU2F
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Confirm that the service is running before you can proceed:

$ systemctl status docker
● docker.service - Docker Application Container Engine
     Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
     Active: active (running) since Sun 2020-10-11 03:02:10 CEST; 1min 12s ago
TriggeredBy: ● docker.socket
       Docs: https://docs.docker.com
   Main PID: 2663 (dockerd)
      Tasks: 8
     Memory: 35.7M
     CGroup: /system.slice/docker.service
             └─2663 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock

Oct 11 03:02:10 ubuntu dockerd[2663]: time="2020-10-11T03:02:10.495871164+02:00" level=warning msg="Your kernel does not support cgroup rt runtime"
Oct 11 03:02:10 ubuntu dockerd[2663]: time="2020-10-11T03:02:10.496038020+02:00" level=warning msg="Your kernel does not support cgroup blkio weight"
Oct 11 03:02:10 ubuntu dockerd[2663]: time="2020-10-11T03:02:10.496232070+02:00" level=warning msg="Your kernel does not support cgroup blkio weight_device"
Oct 11 03:02:10 ubuntu dockerd[2663]: time="2020-10-11T03:02:10.497045530+02:00" level=info msg="Loading containers: start."
Oct 11 03:02:10 ubuntu dockerd[2663]: time="2020-10-11T03:02:10.667666503+02:00" level=info msg="Default bridge (docker0) is assigned with an IP address 172.17.0>
Oct 11 03:02:10 ubuntu dockerd[2663]: time="2020-10-11T03:02:10.781078393+02:00" level=info msg="Loading containers: done."
Oct 11 03:02:10 ubuntu dockerd[2663]: time="2020-10-11T03:02:10.835604534+02:00" level=info msg="Docker daemon" commit=4484c46d9d graphdriver(s)=overlay2 version>
Oct 11 03:02:10 ubuntu dockerd[2663]: time="2020-10-11T03:02:10.836601759+02:00" level=info msg="Daemon has completed initialization"
Oct 11 03:02:10 ubuntu systemd[1]: Started Docker Application Container Engine.

Step 3: Run Graphite on Ubuntu 20.04 in Docker Container

Once the docker engine is installed and running, it is time to start docker container for Graphite and Statsd. We will use the official Docker repo for Graphite:

Download the latest docker image.

$ docker pull  graphiteapp/graphite-statsd
Using default tag: latest
latest: Pulling from graphiteapp/graphite-statsd
df20fa9351a1: Pull complete
f9a569415da5: Pull complete
8f0c7d0dc99e: Pull complete
700de820209a: Pull complete
Digest: sha256:04a0037cc2ae7cc189b81fb38fbdc914fe269b861108821bea1ed776878334de
Status: Downloaded newer image for graphiteapp/graphite-statsd:latest
docker.io/graphiteapp/graphite-statsd:latest

List available images:

$ docker images
REPOSITORY                    TAG                 IMAGE ID            CREATED             SIZE
graphiteapp/graphite-statsd   latest              875c7f22f4cd        2 months ago        628MB

Create directories on the host system for persistent data storage. This can be a mount point of a disk or partition dedicated to storage of Graphite data.

sudo mkdir -p /data/graphite/{data,logs,conf,statsd_config}

The image we downloaded includes the following components:

  • Nginx – reverse proxies the graphite dashboard
  • Graphite – front-end dashboard
  • Carbon – back-end
  • Statsd – UDP based back-end proxy

Mapped Ports

HostContainerService8080nginx20032003carbon receiver – plaintext20042004carbon receiver – pickle20232023carbon aggregator – plaintext20242024carbon aggregator – pickle80808080Graphite internal gunicorn port (without Nginx proxying).81258125statsd81268126statsd admin

Graphite & Statsd can be complex to set up. The docker image provided will have you running & collecting stats in just a few minutes.

Change timezone to your correct setting before running the commands.

docker run -d \
 --name graphite \
 --restart=always \
 -p 80:80 \
 -p 2003-2004:2003-2004 \
 -p 2023-2024:2023-2024 \
 -p 8125:8125/udp \
 -p 8126:8126 \
 -v /data/graphite/data:/opt/graphite/storage \
 -v /data/graphite/conf:/opt/graphite/conf \
 -v /data/graphite/statsd_config:/opt/statsd/config \
 -v /data/graphite/logs:/var/log \
 -e GRAPHITE_TIME_ZONE='Africa/Nairobi' \
 graphiteapp/graphite-statsd

List running containers to see if graphite container is running.

$ docker ps
CONTAINER ID        IMAGE                         COMMAND             CREATED             STATUS              PORTS                                                                                                                                                                       NAMES
4a26c4c0f746        graphiteapp/graphite-statsd   "/entrypoint"       31 seconds ago      Up 29 seconds       0.0.0.0:80->80/tcp, 0.0.0.0:2003-2004->2003-2004/tcp, 2013-2014/tcp, 8080/tcp, 0.0.0.0:2023-2024->2023-2024/tcp, 0.0.0.0:8126->8126/tcp, 8125/tcp, 0.0.0.0:8125->8125/udp   graphite

You can stream logs to check if there are any errors.

$ docker logs -f graphite

Sample output:

,,..
1/10/2020 01:23:58 :: [tagdb] Tagged carbon.aggregator.4a26c4c0f746-a.destinations.127_0_0_1:2004:None.attemptedRelays, carbon.aggregator.4a26c4c0f746-a.bufferedDatapoints in 0.021245479583740234
11/10/2020 01:24:06 :: [listener] MetricLineReceiver connection with 127.0.0.1:57472 established
11/10/2020 01:24:06 :: [listener] MetricLineReceiver connection with 127.0.0.1:57472 closed cleanly
11/10/2020 01:24:08 :: [tagdb] Tagging carbon.agents.4a26c4c0f746-a.errors
11/10/2020 01:24:08 :: [tagdb] Tagged carbon.agents.4a26c4c0f746-a.errors in 0.15246272087097168
11/10/2020 01:24:16 :: [listener] MetricLineReceiver connection with 127.0.0.1:57476 established
11/10/2020 01:24:16 :: [listener] MetricLineReceiver connection with 127.0.0.1:57476 closed cleanly
11/10/2020 01:24:26 :: [listener] MetricLineReceiver connection with 127.0.0.1:57478 established
11/10/2020 01:24:26 :: [listener] MetricLineReceiver connection with 127.0.0.1:57478 closed cleanly
11/10/2020 01:24:26 :: [tagdb] Tagging stats.statsd.graphiteStats.last_flush
11/10/2020 01:24:26 :: [tagdb] Tagged stats.statsd.graphiteStats.last_flush in 0.013707160949707031
11/10/2020 01:24:36 :: [listener] MetricLineReceiver connection with 127.0.0.1:57482 established
11/10/2020 01:24:36 :: [listener] MetricLineReceiver connection with 127.0.0.1:57482 closed cleanly
11/10/2020 01:24:36 :: [tagdb] Tagging stats.timers.view.graphite.tags.views.tagMultiSeries.POST.count_90
11/10/2020 01:24:36 :: [tagdb] Tagged stats.timers.view.graphite.tags.views.tagMultiSeries.POST.count_90 in 0.011429786682128906
11/10/2020 01:24:46 :: [listener] MetricLineReceiver connection with 127.0.0.1:57486 established
11/10/2020 01:24:46 :: [listener] MetricLineReceiver connection with 127.0.0.1:57486 closed cleanly
11/10/2020 01:24:47 :: [tagdb] Tagging carbon.aggregator.4a26c4c0f746-a.whitelistRejects
11/10/2020 01:24:47 :: [tagdb] Tagged carbon.aggregator.4a26c4c0f746-a.whitelistRejects in 0.015319108963012695
11/10/2020 01:24:56 :: [listener] MetricLineReceiver connection with 127.0.0.1:57490 established
11/10/2020 01:24:56 :: [listener] MetricLineReceiver connection with 127.0.0.1:57490 closed cleanly
11/10/2020 01:24:57 :: [tagdb] Tagging carbon.agents.4a26c4c0f746-a.memUsage
11/10/2020 01:24:57 :: [tagdb] Tagged carbon.agents.4a26c4c0f746-a.memUsage in 0.009841442108154297
11/10/2020 01:25:06 :: [listener] MetricLineReceiver connection with 127.0.0.1:57494 established
11/10/2020 01:25:06 :: [listener] MetricLineReceiver connection with 127.0.0.1:57494 closed cleanly
11/10/2020 01:25:07 :: [tagdb] Tagging carbon.agents.4a26c4c0f746-a.creates
11/10/2020 01:25:07 :: [tagdb] Tagged carbon.agents.4a26c4c0f746-a.creates in 0.019759416580200195

By default, statsd listens on the UDP port 8125. If you want it to listen on the TCP port 8125 instead, you can set the environment variable STATSD_INTERFACE to tcp when running the container.

Step 4: Accessing the Graphite Web interface

You can then access Graphite web console using the server IP address or port number and the port mapped on the host to container’s port 80:

The default logins creds are:

Username: root
Password: root

Change this login credentials after the first login at http://ip-address/admin/auth/user/1/. Click “Change Password” link to initiate root user password change.

Provide desired new root password and confirm.

You can now logout and back with the new password.

Step 5: Managing Graphite Container with Systemd

To ensure our container is started at system boot, let’s create new Systemd service unit file.

sudo tee /etc/systemd/system/graphite-docker.service<<EOF
[Unit]
Description=Graphite Docker Container
Documentation=https://github.com/graphite-project/docker-graphite-statsd
After=docker.service
Requires=docker.service

[Service]
Type=simple
TimeoutStartSec=0
Restart=on-failure
RestartSec=30s
ExecStartPre=-/usr/bin/docker kill graphite
ExecStartPre=-/usr/bin/docker rm graphite
ExecStartPre=/usr/bin/docker pull graphiteapp/graphite-statsd
ExecStart=/usr/bin/docker run \
            --name graphite \
            --restart=always \
            -p 80:80 \
            -p 2003-2004:2003-2004 \
            -p 2023-2024:2023-2024 \
            -p 8125:8125/udp \
            -p 8126:8126 \
            -v /data/graphite/data:/opt/graphite/storage \
            -v /data/graphite/conf:/opt/graphite/conf \
            -v /data/graphite/statsd_config:/opt/statsd/config \
            -v /data/graphite/logs:/var/log \
            graphiteapp/graphite-statsd

SyslogIdentifier=graphite
ExecStop=/usr/bin/docker stop graphite

[Install]
WantedBy=multi-user.target
EOF

Reload Systemd service.

sudo systemctl daemon-reload

The unit file we created should now be recognized.

$ sudo systemctl list-unit-files graphite-docker.service
UNIT FILE               STATE    VENDOR PRESET
graphite-docker.service disabled enabled

1 unit files listed.

Let’s enable the service to be started at system boot.

$ sudo systemctl enable graphite-docker
Created symlink /etc/systemd/system/multi-user.target.wants/graphite-docker.service → /etc/systemd/system/graphite-docker.service.

Kill the running container to confirm the service is working.

sudo docker rm -f graphite

If you don’t have other containers the ps list should return empty.

$ docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES

Start Systemd service.

sudo systemctl start graphite-docker.service

Check service status:

$ systemctl status graphite-docker.service
● graphite-docker.service - Graphite Docker Container
     Loaded: loaded (/etc/systemd/system/graphite-docker.service; enabled; vendor preset: enabled)
     Active: active (running) since Sun 2020-10-11 03:53:48 CEST; 10s ago
       Docs: https://github.com/graphite-project/docker-graphite-statsd
    Process: 16394 ExecStartPre=/usr/bin/docker kill graphite (code=exited, status=1/FAILURE)
    Process: 16410 ExecStartPre=/usr/bin/docker rm graphite (code=exited, status=1/FAILURE)
    Process: 16416 ExecStartPre=/usr/bin/docker pull graphiteapp/graphite-statsd (code=exited, status=0/SUCCESS)
   Main PID: 16428 (docker)
      Tasks: 7 (limit: 2286)
     Memory: 25.1M
     CGroup: /system.slice/graphite-docker.service
             └─16428 /usr/bin/docker run --name graphite --restart=always -p 80:80 -p 2003-2004:2003-2004 -p 2023-2024:2023-2024 -p 8125:8125/udp -p 8126:8126 -v>

Oct 11 03:53:53 ubuntu graphite[16428]: run: carbon: (pid 73) 5s; run: log: (pid 72) 5s
Oct 11 03:53:53 ubuntu graphite[16428]: run: carbon-aggregator: (pid 66) 5s; run: log: (pid 65) 5s
Oct 11 03:53:53 ubuntu graphite[16428]: down: carbon-relay: 1s, normally up, want up; run: log: (pid 59) 5s
Oct 11 03:53:53 ubuntu graphite[16428]: down: collectd: 1s, normally up, want up
Oct 11 03:53:53 ubuntu graphite[16428]: run: cron: (pid 58) 5s
Oct 11 03:53:53 ubuntu graphite[16428]: down: go-carbon: 1s, normally up, want up
Oct 11 03:53:53 ubuntu graphite[16428]: run: graphite: (pid 70) 5s
Oct 11 03:53:53 ubuntu graphite[16428]: run: nginx: (pid 63) 5s
Oct 11 03:53:53 ubuntu graphite[16428]: down: redis: 1s, normally up, want up
Oct 11 03:53:53 ubuntu graphite[16428]: run: statsd: (pid 71) 5s; run: log: (pid 67) 5s

Use docker CLI to list running containers.

$ docker ps
CONTAINER ID        IMAGE                         COMMAND             CREATED              STATUS              PORTS                                                                                                                                                                       NAMES
e5647ba93e4d        graphiteapp/graphite-statsd   "/entrypoint"       About a minute ago   Up About a minute   0.0.0.0:80->80/tcp, 0.0.0.0:2003-2004->2003-2004/tcp, 2013-2014/tcp, 8080/tcp, 0.0.0.0:2023-2024->2023-2024/tcp, 0.0.0.0:8126->8126/tcp, 8125/tcp, 0.0.0.0:8125->8125/udp   graphite

Perform system reboot to ensure container is started if server is ever rebooted.

sudo reboot

We can confirm the container is started with different ID:

$ docker ps
CONTAINER ID        IMAGE                         COMMAND             CREATED              STATUS              PORTS                                                                                                                                                                       NAMES
11e596067be9        graphiteapp/graphite-statsd   "/entrypoint"       About a minute ago   Up About a minute   0.0.0.0:80->80/tcp, 0.0.0.0:2003-2004->2003-2004/tcp, 2013-2014/tcp, 8080/tcp, 0.0.0.0:2023-2024->2023-2024/tcp, 0.0.0.0:8126->8126/tcp, 8125/tcp, 0.0.0.0:8125->8125/udp   graphite

Read through Graphite documentation to start sending metrics from your applications to Graphite.


report

Recommend

About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK