Happy New Year!

Thomas Hammer last week wrote a blog introducing new features available with Q4 2022 release of SAP HANA Cloud. I would like to also mention some enhancements done for SQL query execution for 2022 overall.

If you have been using the recent SAP HANA Cloud versions, you may have noticed improvements in execution time while using less memory for the same SQL statements. The engineering team has worked hard to deliver such improvements for the last couple of years and recently we are gradually switching over to the newer execution engine as the coverage of functionality is nearly completed to replace the existing engines such as Join, OLAP and ESX. As you may already know, the enhanced engine is HEX which was initially introduced with the on-premise version SAP HANA 2.0 SPS02.

The benefit of the new execution engine is

• All functionalities are combined to be executed in a single engine – Previously, swithing between engines was unavoidable which required materialization when moving data from one engine to another. This required a lot of memory for the materialization. As all functionality is being combined, there is no longer a need for such materialization. Additional benefit is improved performance as such materialization is avoided.
• Just-In-Time code compilation – This generates a execution plan compiled just prior to the exection exactly suited for the query itself. Previous engines generates pre-defined operators that are plugged together during execition. The downside of JIT compiled plans are increased plan preparation time as it needs to generate the JIT compilation. For year 2023, we are focusing on improving the plan perparation time. To overcome the longer preparation time, we will be actively using Abstract SQL Plan (ASP) captured with SQL Plan Stability. Once the plan is cached, we will benefit from the compiled plan that is already cached. The ASP capturing will be turned on by default starting from Q1 2023. We will announce new features that will utilize this in the coming quarters.
• Executions are pipelined and parallelized – The executions are pipelined and each pipeline is also parallelized. For example, joining 2 tables as hashjoin, the logical plan is as on the left side of the diagram below. For the execution plan, each table is processed as different pipelines and each pipeline is parallelized for faster execution.

Data is processed in chunks – The data is read and processed in chunks through the pipeline consuming less memory as the entire data doesn’t needed to be read first which executing each operators. The SQL execution engine is now able to return data using lesser memory compared to existing engines.

Based on our internal testing, we have identified big improvements when queries are executed with HEX vs existing engines in terms of overall elapsed time and CPU time.

For HEX, as the data is streamlined in data chunks, the CPU time is the key factore for performance. In addition to performance, we also have tested that it requires much less memory for executing the same statements.

Reduction of memory

You may have noticed that some of your queries have been executing with the newer engine and with Q4 2022 release of SAP HANA Cloud, you will see more statements running by HEX engine. There may be a still few queries running by the existing engines as we don’t have 100% coverage yet but expect them to swith over in 2023.

I will have some followup blogs regarding SQL Plan Stability and SQL Analyzer for HEX.

Best Regards,
Tae Suk Son
SAP HANA Product Management