Running Zenoss on iSCSI and LVM remote Storage

Zenoss is the leading provider of hybrid IT monitoring, infrastructure monitoring and analytics software for physical, virtual and cloud-based IT infrastructures. In this blog post, I’m going to guide you through the steps to prepare iscsi storage for Zenoss Sever installation. Since zenoss needs huge amount of space for its data, docker, internal services and backups, It’s not ideal to use local storage, especially if you are working with a Virtual Machine.

In my setup, I have a physical server with 4 disks, each having 500GB. The four hard drives will be configured in raid 0 to deliver a total of 2TB raw storage. On top of raid 0, I’m going to create a physical volume on it then create volume group called data. The logical volumes created on data volume group will serve as basis for iscsi backstore.

Zenoss Master host block devices to create on iscsi will have the following specifications:

Storage typeSizeTypeMount pointDocker data storage50 GBDevice mapper thin pool/var/lib/dockerInternal services storage50 GBXFS file system/opt/serviced/var/isvcsApplication data300GBDevice mapper thin poolNoneApplication data backups150GBXFS file system/opt/serviced/var/backups

With that few notes to help you understand the configurations to follow, you should be ready to start the hands-on stuff.

Prepare physical storage devices

• Prepare raid devices:

parted --script /dev/sdc "mklabel gpt"
parted --script /dev/sdc "mkpart primary 0% 100%"
parted --script /dev/sdc "set 1 raid on" 

parted --script /dev/sdd "mklabel gpt"
parted --script /dev/sdd "mkpart primary 0% 100%"
parted --script /dev/sdd "set 1 raid on"

parted --script /dev/sde "mklabel gpt"
parted --script /dev/sde "mkpart primary 0% 100%"
parted --script /dev/sde "set 1 raid on"

parted --script /dev/sdf "mklabel gpt"
parted --script /dev/sdf "mkpart primary 0% 100%"
parted --script /dev/sdf "set 1 raid on"

• Create raid 0 for data storage

mdadm --create /dev/md0 --level=raid0 --raid-devices=4 /dev/sdc1  /dev/sdd1 /dev/sde1  /dev/sdf1 

• Prepare /dev/md0 for physical volume

pvcreate /dev/md0

• Create volume group called data.

vgcreate data /dev/md0

• Create logical volumes for zenoss

lvcreate -L 50G -n zenoss_docker data
lvcreate -L 50G -n zenoss_cc_internal data
lvcreate -L 200G -n zenoss_application_data data
lvcreate -L 150G -n zenoss_data_backups data

Install and configure iscsi on lvm

yum -y install targetcli

• Set iSCSI initiator name

# echo "InitiatorName=iqn.2017-05.co.angani-iscsi:9add7c1d8360"  > /etc/iscsi/initiatorname.iscsi
# systemctl restart iscsid
# /sbin/iscsi-iname

• Create and export storage object backed by logical volume

targetcli /backstores/block create dev=/dev/mapper/data-zenoss_docker name=zenoss_docker
targetcli /backstores/block create dev=/dev/mapper/data-zenoss_application_data  name=zenoss_application_data
targetcli /backstores/block create dev=/dev/mapper/data-zenoss_cc_internal  name=zenoss_cc_internal
targetcli /backstores/block create dev=/dev/mapper/data-zenoss_data_backups   name=zenoss_data_backups
targetcli /backstores/block ls

• Create an IQN for the iSCSI target

targetcli /iscsi create iqn.2017-05.co.angani-iscsi:target00
targetcli /iscsi ls

• Configure ACLs for the TPG
• This ACL allows zenoss-01 server to access the target’s IQN

targetcli /iscsi/iqn.2017-05.co.angani-iscsi:target00/tpg1/acls create iqn.2017-05.co.angani-iscsi:zenoss-01

• Configure CHAP authentication by creating initiator users to allow access to backend storage

targetcli /iscsi/iqn.2017-05.co.angani-iscsi:target00/tpg1/acls/iqn.2017-05.co.angani-iscsi:zenoss-01 set auth userid=zenoss password=Oopequaiquieng5

• Create the LUNs needed to associate a block device with a specific TPG.

targetcli /iscsi/iqn.2017-05.co.angani-iscsi:target00/tpg1/luns create  /backstores/block/zenoss_docker
targetcli /iscsi/iqn.2017-05.co.angani-iscsi:target00/tpg1/luns create /backstores/block/zenoss_application_data/
targetcli /iscsi/iqn.2017-05.co.angani-iscsi:target00/tpg1/luns create /backstores/block/zenoss_cc_internal/
targetcli /iscsi/iqn.2017-05.co.angani-iscsi:target00/tpg1/luns create /backstores/block/zenoss_data_backups/

• Configure a target to offer services on specific ip (optional)
• Default is 0.0.0.0:3260

targetcli /iscsi/iqn.2017-05.co.angani-iscsi:target00/tpg1/portals/ crate 192.168.70.80

• Save configs to /etc/target/saveconfig.json

targetcli saveconfig
ss -na | grep 3260

• Open firewall port for iscsi

firewall-cmd --add-port 3260/tcp --permanent
firewall-cmd --reload

For iptables:

iptables -I INPUT -p tcp -m tcp --dport 3260 -j ACCEPT
iptables-save /etc/sysconfig/iptables

• Start and enable target service

systemctl enable target
systemctl start target

Setup iSCSI Initiator

This part is done on the host to run Zenoss control center, either master or delegate server.

# yum install -y iscsi-initiator-utils

• Set InitiatorName

echo "InitiatorName=iqn.2017-05.co.angani-iscsi:zenoss-01"  > /etc/iscsi/initiatorname.iscsi
systemctl restart iscsid

• Configure initiator authentication

sed -i 's/#node.session.auth.username = username/node.session.auth.username = zenoss/g' /etc/iscsi/iscsid.conf
sed -i 's/#node.session.auth.password = password/node.session.auth.password = Oopequaiquieng5/g' /etc/iscsi/iscsid.conf

• Discover targets

iscsiadm -m discovery -t sendtargets -p 192.168.70.80 --discover

192.168.70.80:3260,1 iqn.2017-05.co.angani-iscsi:target00

• Login to target

iscsiadm -m node --login 

• Once the connection is established, both session and node details can be checked as follows.

iscsiadm -m session -o show
iscsiadm --mode node -P 1

• Before mounting the iSCSI Devices. First list the available iSCSI devices

# lsscsi

[0:0:0:0]    cd/dvd  QEMU     QEMU DVD-ROM     1.5.  /dev/sr0 
[2:0:0:0]    disk    LIO-ORG  zenoss_docker    4.0   /dev/sda 
[2:0:0:1]    disk    LIO-ORG  zenoss_applicat  4.0   /dev/sdb 
[2:0:0:2]    disk    LIO-ORG  zenoss_cc_inter  4.0   /dev/sdc 
[2:0:0:3]    disk    LIO-ORG  zenoss_data_bac  4.0   /dev/sdd 

• Do the same with lsblk command just to double-check:

# lsblk
NAME                   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda                      8:0    0   50G  0 disk 
sdb                      8:16   0  200G  0 disk 
sdc                      8:32   0   50G  0 disk 
sdd                      8:48   0  150G  0 disk 

You can now proceed to partitioning the devices for your Zenoss installation.

To logout do – That is if you want to, but why should you do so anyway! :

iscsiadm --mode node --logout

On our next tutoria about Zenoss server setup, I’ll cover installation steps for the control center and zenoss core system setup. Hope this was helpful.