5

Taming SAP HANA Express database Docker edition with MacOS – prepare

 1 year ago
source link: https://blogs.sap.com/2021/07/21/taming-sap-hana-express-database-docker-edition-with-macos-prepare/
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
neoserver,ios ssh client
42-27270734_sRGB-scaled.jpg

2.00.061.00.20220519.1

Taming HANA Express database Docker edition with MacOS and SAP Analytics Cloud is to illustrate some basic steps of SAP HANA express installation and configuration as a backend database for either SAP Analytics Cloud reporting or as an SQL database for some edge device.
However, I need to start with a disclaimer:
  • albeit the SAP HANA, express edition (database services) is not officially supported with Docker Desktop engine for MacOS, I will explain how one can use SAP HANA express database docker edition with the docker desktop engine on MacOS.

  • this blog post is neither an installation manual nor a tutorial. Please always refer to official SAP resources, namely:

Another way of using the supported docker installations with MacOS is to have a Virtual Machine with one of the supported Linux operating system and then proceed with the setup instructions available here.

However, the biggest advantage of running a docker container natively on the host operating system is that the container is always available including the fact it can survive the OS updates and reboots!

(my laptop had restarted several times during the write-up of this blog so indeed this does work…)

Before you start…

You will need a MacOS laptop with at least 16GB of memory and around 100GB of spare disk space together with:
  • docker desktop engine up and running with a recommended 100GB for docker disk image size
  • docker hub account to pull hana database image into your local docker repository
  • internet connection
  • a good understanding of what you are trying to get done here.
    • Please get familiar with the following HANA Express documentation.
    • Additionally, this SAP tutorial offers a good summary of what to expect from this exercise.
  • last but not least you will need HANA Studio or Eclipse with HANA Studio plugins — however, this will be covered in the 2nd instalment

The dry-run is described in the appendix.

After successful dry-run it makes sense to start our container in the background and expose the necessary ports to the localhost MacOS machine;

Good to know:

  • as we will need to be using the hana express host name (myhana) rather and not localhost we will need to get access to the docker container ip address with the socks5 proxy (as described in the appendix).
 
$ docker run -d --restart=always -p 8090:8090 -p 4390:4390 -p 39013:39013 -p 39017:39017 -p 39041-39045:39041-39045 -p 1128-1129:1128-1129 -p 59013-59014:59013-59014 -h myhana --privileged=true --ulimit nofile=1048576:1048576 --sysctl kernel.shmmax=1073741824 --sysctl net.ipv4.ip_local_port_range='40000 60999' --sysctl kernel.shmmni=32 --sysctl kernel.shmall=8388608 --name myhana_express saplabs/hanaexpress:2.00.061.00.20220519.1 --passwords-url https://*********/db.json --agree-to-sap-license
5c2984e67e1126b64b3b280129571c1c7ce96c342e687ec1862452f77329cca8

Please note that the docker run command will have returned almost immediately and the execution continues in the background. After a couple of minutes the installation is complete.

You can monitor the initial run and logs using the following docker logs command:

 
docker logs myhana_express
In the aftermath of a successful setup, you can get access to the running myhana_express container as the <sid>adm user as follows: 
$ docker exec -it myhana_express bash

hxeadm@myhana:/usr/sap/HXE/HDB90> HDB info
USER          PID     PPID  %CPU        VSZ        RSS COMMAND
hxeadm       2936        0   0.3      21084       4072 bash
hxeadm       2991     2936   0.0      20284       3352  \_ /bin/sh /usr/sap/HXE/HDB90/HDB info
hxeadm       3022     2991   0.0      41424       3312      \_ ps fx -U hxeadm -o user:8,pid:8,ppid:8,pcpu:5,vsz:10,rss:10,args
hxeadm          1        0   0.0      21064       3952 /bin/bash /run_hana --passwords-url https://*********/db.json --agree-to-sap-license
hxeadm        921        1   0.0     369160      57440 hdbdaemon
hxeadm        927      921  18.3    3973212    2635920  \_ hdbnameserver
hxeadm       1243      921   1.0     778336     130148  \_ hdbcompileserver
hxeadm       1244      921   1.0     861048     159604  \_ hdbpreprocessor
hxeadm       1291      921   1.2    1692252     397872  \_ hdbwebdispatcher
hxeadm       1423      921  18.8    3930976    2627024  \_ hdbindexserver -port 39040
hxeadm       1812      921   1.1    1402104     322420  \_ hdbdiserver
hxeadm       1070        1   0.0     620932      51344 hdbrsutil  --start --port 39001 --volume 1 --volumesuffix mnt00001/hdb00001 --identifier 1626125198
hxeadm       1523        1   0.0     621060      51464 hdbrsutil  --start --port 39040 --volume 2 --volumesuffix mnt00001/hdb00002.00003 --identifier 1626125259
hxeadm@myhana:/usr/sap/HXE/HDB90> 
hxeadm@myhana:/usr/sap/HXE/HDB90> hdbsql -i 90 -d SystemDB -u SYSTEM -p ***pwd***

Welcome to the SAP HANA Database interactive terminal.
                                           
Type:  \h for help with commands          
       \q to quit                         

hdbsql SYSTEMDB=> \mu
Multiline mode switched ON
hdbsql SYSTEMDB=> \al
Aligned output mode switched ON
hdbsql SYSTEMDB=> select * from "SYS"."M_DATABASES";
| DATABASE | DESCRIPTION     | ACT | ACTIVE_S | OS_USER  | OS_GROUP | RESTART | F |
| -------- | --------------- | --- | -------- | -------- | -------- | ------- | - |
| SYSTEMDB | SystemDB-HXE-90 | YES |          |          |          | DEFAULT | ? |
| HXE      |                 | YES |          |          |          | DEFAULT | ? |
2 rows selected (overall time 17.166 msec; server time 286 usec)

hdbsql SYSTEMDB=> SELECT * FROM SYS_DATABASES.M_SERVICES;
| DATABASE | HOST   | PORT        | SERVICE_NAME  | PROCESS_ID  | DETAIL | ACT | SQL_PORT    | COORDI | IS_DA |
| -------- | ------ | ----------- | ------------- | ----------- | ------ | --- | ----------- | ------ | ----- |
|          | myhana |       39000 | daemon        |         921 |        | YES |           0 | NONE   | FALSE |
| SYSTEMDB | myhana |       39001 | nameserver    |         927 | master | YES |       39017 | MASTER | TRUE  |
|          | myhana |       39002 | preprocessor  |        1244 |        | YES |           0 | NONE   | FALSE |
| SYSTEMDB | myhana |       39005 | diserver      |        1812 |        | YES |           0 | NONE   | TRUE  |
|          | myhana |       39006 | webdispatcher |        1291 |        | YES |           0 | NONE   | FALSE |
|          | myhana |       39010 | compileserver |        1243 |        | YES |           0 | NONE   | FALSE |
| HXE      | myhana |       39040 | indexserver   |        1423 | master | YES |       39041 | MASTER | TRUE  |
7 rows selected (overall time 16.668 msec; server time 286 usec)

hdbsql SYSTEMDB=> \q
hxeadm@myhana:/usr/sap/HXE/HDB90>

hdbupdrep is a very useful tool. Typically its main use is to re-install/ repair failed delivery units

but it also features several other options for instance:

 
myhana:/hana/shared/HXE/global/hdb/install/bin # ./hdbupdrep --list_systems
SAP HANA Database Installations:

HXE /hana/shared/HXE HDB_ALONE
	HDB90
	version: 2.00.054.00.1611906357
	host: myhana (worker)
	edition: SAP HANA Express Edition
	plugins: afl,epmmds

Already used instance number: 90
Next available instance number: 00
myhana:/hana/shared/HXE/global/hdb/install/bin #

content / HXEDownloadManager

You may also notice that the docker image has been stripped down to the very minimum and there is strictly no content (no delivery units packages) there.

After the inspection, the content directory, namely /hana/shared/HXE/global/hdb/content is initially empty.

hxeadm@myhana:/usr/sap/HXE/HDB90> cd /hana/shared/HXE/global/hdb/content
hxeadm@myhana:/hana/shared/HXE/global/hdb/content> ls -l
total 0

Let’s download the required content locally first.

$ mkdir 2.00.054.00.20210603.1
$ java -jar HXEDownloadManager.jar
docker01-1.png
$ ls -l 2.00.054.00.20210603.1

-rw-r--r--  1 toto  staff    57864815 xx Jul 13:45 apl.tgz
-rw-r--r--  1 toto  staff    36472170 xx Jul 13:45 eml.tgz
-rw-r--r--  1 toto  staff  1963468936 xx Jul 13:44 hxe.tgz
-rw-r--r--  1 toto  staff      423013 xx Jul 13:45 sdi.tgz
-rw-r--r--  1 toto  staff   322980246 xx Jul 13:45 shine.tgz

Prepare content

$ cd 2.00.054.00.20210603.1

$ tar -xvf sdi.tgz
..........
x HANA_EXPRESS_20/DATA_UNITS/SAP_HANA_SDI_20/HANA_IM_DP-2.5.30.tgz
..........

$ tar -xvf hxe.tgz 
..........
x HANA_EXPRESS_20/DATA_UNITS/HDB_SERVER_LINUX_X86_64/server/CONTENT.TGZ
...............

$ mkdir packages/tmp
$ mv HANA_EXPRESS_20/DATA_UNITS/SAP_HANA_SDI_20/HANA_IM_DP-2.5.30.tgz packages/
$ mv HANA_EXPRESS_20/DATA_UNITS/HDB_SERVER_LINUX_X86_64/server/CONTENT.TGZ packages/tmp

$ ls -l packages/
total 526096
-r--r--r--  1 toto  staff       3228 29 Jan 09:20 AHCO_INA_SERVICE.tgz
-r--r--r--  1 toto  staff     236114 29 Jan 09:20 HANA_DT_BASE.tgz
-r--r--r--  1 toto  staff       1366 29 Jan 09:20 HANA_DXC.tgz
-r--r--r--  1 toto  staff   18342067 29 Jan 09:20 HANA_IDE_CORE.tgz
........................(truncated)..............................................
drwxr-xr-x  5 toto  staff        160 13 Jul 14:40 systemdb

Copy local content to docker container

$ docker ps -a
CONTAINER ID   IMAGE                                              COMMAND                  CREATED         STATUS                     PORTS                                                                                                                                                                                                                                                                                                                                                                                             NAMES
5c2984e67e11   store/saplabs/hanaexpress:2.00.054.00.20210603.1   "/run_hana --passwor…"   22 hours ago    Up 22 hours (healthy)      0.0.0.0:1128-1129->1128-1129/tcp, :::1128-1129->1128-1129/tcp, 0.0.0.0:4390->4390/tcp, :::4390->4390/tcp, 0.0.0.0:8090->8090/tcp, :::8090->8090/tcp, 0.0.0.0:39013->39013/tcp, :::39013->39013/tcp, 0.0.0.0:39017->39017/tcp, :::39017->39017/tcp, 0.0.0.0:39041-39045->39041-39045/tcp, :::39041-39045->39041-39045/tcp, 0.0.0.0:59013-59014->59013-59014/tcp, :::59013-59014->59013-59014/tcp   myhana_express

// copy into container
$ docker cp . myhana_express:/hana/shared/HXE/global/hdb/content


hxeadm@myhana:/hana/shared/HXE/global/hdb/content> ls -l
total 263052
-r--r--r-- 1 501 cdrom      3228 Jan 29 09:20 AHCO_INA_SERVICE.tgz
-rwxr-xr-x 1 501 cdrom    413190 Mar 11 05:05 HANA_IM_DP-2.5.30.tgz
.....................(truncated)....................................
-r--r--r-- 1 501 cdrom      2032 Jan 29 09:20 ZHANA_WKLD_ANLZ.tgz
drwxr-xr-x 2 501 cdrom      4096 Jul 13 13:40 systemdb
hxeadm@myhana:/hana/shared/HXE/global/hdb/content>

Import delivery units

systemdb:

/usr/sap/HXE/HDB90/exe/regi import HANA_XS_BASE.tgz HDC_ADMIN.tgz HANA_TA_CONFIG.tgz HANA_XS_DBUTILS.tgz  HANA_XS_EDITOR.tgz  HDC_IDE_CORE.tgz SAP_WATT.tgz SAPUI5_1.tgz  --host=myhana:39017 --user=SYSTEM --password=******
cd hxeadm@myhana:/hana/shared/HXE/global/hdb/content> 

/usr/sap/HXE/HDB90/exe/regi import HANA_UI_INTEGRATION_2_CONTENT.tgz    --host=myhana:39041 --user=SYSTEM --password=********
/usr/sap/HXE/HDB90/exe/regi import HANA_UI_INTEGRATION_1_SVC.tgz    --host=myhana:39041 --user=SYSTEM --password=*******
/usr/sap/HXE/HDB90/exe/regi import ZHANA_WKLD_ANLZ.tgz   --host=myhana:39041 --user=SYSTEM --password=********
/usr/sap/HXE/HDB90/exe/regi import XHANA_SEC_CP.tgz YHANA_BACKUP.tgz YHANA_SEC_BASE.tgz  --host=myhana:39041 --user=SYSTEM --password=*****
/usr/sap/HXE/HDB90/exe/regi import AHCO_INA_SERVICE.tgz  --host=myhana:39041 --user=SYSTEM --password=*******
/usr/sap/HXE/HDB90/exe/regi import AHCO_INA_SERVICE.tgz  --host=myhana:39041 --user=SYSTEM --password=*******
/usr/sap/HXE/HDB90/exe/regi import AHCO_INA_SERVICE.tgz  --host=myhana:39041 --user=SYSTEM --password=*******
/usr/sap/HXE/HDB90/exe/regi import HANA_XS_BASE.tgz HDC_ADMIN.tgz HANA_TA_CONFIG.tgz HANA_XS_DBUTILS.tgz  HANA_XS_EDITOR.tgz  HDC_IDE_CORE.tgz SAP_WATT.tgz SAPUI5_1.tgz  --host=myhana:39041 --user=SYSTEM --password=*******
/usr/sap/HXE/HDB90/exe/regi import HANA_DT_BASE.tgz  --host=myhana:39041 --user=SYSTEM --password=******
/usr/sap/HXE/HDB90/exe/regi import HANA_IDE_CORE.tgz  --host=myhana:39041 --user=SYSTEM --password=*******
/usr/sap/HXE/HDB90/exe/regi import HANA_XS_IDE.tgz  --host=myhana:39041 --user=SYSTEM --password=********
/usr/sap/HXE/HDB90/exe/regi import HANA_XS_LM.tgz  --host=myhana:39041 --user=SYSTEM --password=********

shine

(Please refer to this github repo for alternative installation instructions.)

$ tar -xvf shine.tgz 
x HANA_EXPRESS_20/install_shine.sh
x HANA_EXPRESS_20/DATA_UNITS/HCO_HANA_SHINE/
x HANA_EXPRESS_20/DATA_UNITS/HCO_HANA_SHINE/HCODEMOCONTENT203_0.ZIP
x HANA_EXPRESS_20/DATA_UNITS/HCO_HANA_SHINE/LABEL.ASC
x HANA_EXPRESS_20/DATA_UNITS/HCO_HANA_SHINE/SIGNATURE.SMF
x HANA_EXPRESS_20/DATA_UNITS/XSA_CONTENT_10/
x HANA_EXPRESS_20/DATA_UNITS/XSA_CONTENT_10/sap-xsac-shine-1.8.4.mtaext
x HANA_EXPRESS_20/DATA_UNITS/XSA_CONTENT_10/XSACSHINE08_4.ZIP

$ unzip HCODEMOCONTENT203_0.ZIP
Archive:  HCODEMOCONTENT203_0.ZIP
  inflating: HCO_DEMOCONTENT.tgz     
  inflating: META-INF/SL_MANIFEST.XML  
  inflating: SIGNATURE.SMF  

         
$ docker cp HCO_DEMOCONTENT.tgz myhana_express:/hana/shared/HXE/global/hdb/content
 
/usr/sap/HXE/HDB90/exe/regi import HCO_DEMOCONTENT.tgz  --host=myhana:39041 --user=SYSTEM --password=*****

Next steps.

That culminates the prepare phase. You have a running hana express database instance with systemdb and hxe tenant db and some demo data (shine package).

In the next instalment we shall see how to reconfigure the webdispatcher and the xs engine. And, last but not least, how to connect SAP Analytics Cloud over a tunnel connection via SAP Cloud Connector.

__________

Appendix

Enable socks5 proxy support for Docker Desktop

$ cat ~/Library/Group\ Containers/group.com.docker/settings.json | grep socks
  "socksProxyPort": 8888,

add socks proxy in MacOS network settings: localhost:8888

you will need to check the ip address of your running myhana_express container...

then add myhana_express container ip address to /etc/hosts, for instance

172.1x.0.x    myhana.sap.corp myhana



please note:

myhana:39041 designates HXE, the tenant database
myhana:39017 designates sytemdb, the system database

SAP HANA, express edition (database services)

$ docker pull store/saplabs/hanaexpress:2.00.054.00.20210603.1
2.00.054.00.20210603.1: Pulling from store/saplabs/hanaexpress
f631eb8a0078: Pull complete 
d04f82e55126: Pull complete 
0b1212f566e8: Pull complete 
8e7d076cd7f0: Pull complete 
62aa9a741295: Pull complete 
c6d7482915a3: Pull complete 
Digest: sha256:fdf1a59bd0e434a56669fe0b933d75c5b802361c0374c4a8079053a2090d2d7d
Status: Downloaded newer image for store/saplabs/hanaexpress:2.00.054.00.20210603.1
docker.io/store/saplabs/hanaexpress:2.00.054.00.20210603.1

Dry run…

I strongly recommend that you take it as a first step before going any further…it will only take a couple of minutes to complete and if the run is successful just stop and remove the container.

$ docker run -h myhana --privileged=true --ulimit nofile=1048576:1048576 --sysctl kernel.shmmax=1073741824 --sysctl net.ipv4.ip_local_port_range='40000 60999' --sysctl kernel.shmmni=4096 --sysctl kernel.shmall=8388608 --name myhana_express2 e3de774167e2 --passwords-url https://raw.githubusercontent.com/********/db.json --agree-to-sap-license

======== Starting HANA container run script ========
Started at: Mon Jul 12 20:47:59 UTC 2021
Script parameters: --passwords-url https://********/db.json --agree-to-sap-license
HANA version: 2.00.054.00.1611906357
WARNING: Unsupported Linux kernel version 5.10. Supported are 3.12 and 4.4+.
New host: 'myhana'
Setting HANAs host name to 'myhana' ...
Checking allowed mountpoints ...
Checking system requirements ...
Checking /proc/sys prerequisites and limits...

...................(truncated)...........................

SAP HANA Lifecycle Management - HdbContainer 2.5.49
***************************************************


Checking container...
Checking system requirements...
Checking linux settings...
SAP HANA instance executed.
Log file written to '/var/tmp/HdbContainer_check_container_2021-07-12_20.47.59.log' on host 'myhana'.
Log files copied to '/hana/mounts/trace/myhana' on host 'myhana'.
Check succeeded: /proc/sys prerequisites and limits
Checking syscalls ...
	ok: move_pages
	ok: mbind

...................(truncated)...........................

Ready at: Mon Jul 12 20:51:03 UTC 2021
Startup finished!

Troubleshooting

Error when issuing the docker run command

sudo docker run -p 39013:39013 -p 39015:39015 -p 39041-39045:39041-39045 -p 1128-1129:1128-1129 -p 59013-59014:59013-59014  -p 39030-39033:39030-39033 -p 51000-51060:51000-51060  -p 53075:53075 \
-h hxehost \
-v /data/hana:/hana/mounts \
--ulimit nofile=1048576:1048576 \
--sysctl kernel.shmmax=1073741824 \
--sysctl net.ipv4.ip_local_port_range='60000 65535' \
--sysctl kernel.shmmni=524288 \
--sysctl kernel.shmall=8388608 \
--name hana_express_edition \
store/saplabs/hanaexpressxsa:2.00.036.00.20190223.1 \
--agree-to-sap-license \
--passwords-url file:///hana/mounts/hxeadm.json

Updating Docker Image / HXEDownloadManager

The sapjvm has also been removed thus you would have to put it back if you wanted to run the database update. 
$ docker exec -it -u root myhana_express bash
myhana:/ # 
myhana:/ # ls -l /usr/sap/HXE/home/bin/
total 2396
-rwxr-xr-x 1 hxeadm sapsys 1181911 Oct 16  2019 HXECheckUpdate_linux.bin
-rwxr-xr-x 1 hxeadm sapsys 1247121 Oct 16  2019 HXEDownloadManager_linux.bin
-rwxr-xr-x 1 hxeadm sapsys    9606 Jun  3 15:13 change_key.sh
-rwxr-xr-x 1 hxeadm sapsys    5192 Jun  3 15:13 hxe_gc.sh


myhana:/ # /usr/sap/HXE/home/bin/HXEDownloadManager_linux.bin -h
which: no java in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin)
which: no java in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin)
/usr/java/sapjvm_8_latest/jre/lib/amd64/server/libjvm.so: cannot open shared object file: No such file or directory
Cannot find 64-bit Java Runtime Environment (JRE) 8 or newer; or the installed JRE is not 64-bit.
 
 
myhana:/ # find / -name java
/usr/share/java
/etc/java

Additional resources

SAP_Best_scrn_R_blk_pos-4.png

report

Recommend

About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK