Enterprises are creating and using more data than ever before. IDC’s Global DataSphere Forecast currently estimates that in 2020, enterprises will create and capture 6.4 zettabytes of new data. In terms of what types of new data is being created, productivity data—or operational, customer, and sales data as well as embedded data—is the fastest-growing category, according to IDC. However, as application usage and data grows, enterprise data management system challenges continue to arise.

One of those key challenges is performance degradation. Modern digital applications have to contend with thousands—even millions—of users, increasingly complex data, and more and more data being required to run most applications. Between the need for improved end-user experiences and increased metrics and tracking, web applications are constantly bogged down with data. To add to this, modern BI and reporting calls for ever-increasing speed and more real-time updates. Most of this is, of course, impossible to achieve in standard data warehouse environments.

As workloads grow and new uses for data arise, the challenge of providing a simple and efficient method for delivering optimal latency and handling of both planned and unplanned peaks becomes even greater. In order to manage this, data architects and data engineers can employ database caching techniques to improve the performance of their applications and BI tools.

In this article, we’ll discuss what caching is, various use cases, and the best approach for each to accelerate data access and decrease costs. We’ll also present some of the more popular caching tools and their relative advantages.

Caching: What, Why, and Where

Caching is a technique used to manage frequently accessed data in-memory. The method facilitates access to data while also reducing the workload on the main data stores. Not only can caching enhance performance; it can also improve the scalability and availability of an application.

Caching has a proven track record of helping companies of all sizes improve application performance. And in fact, modern caching services provide much more than just a quick key-value cache. But in order to understand what caching is all about and where it is headed, let’s first review the basics.

Caching Pattern Types

Different caching solutions support different implementations, each with its pros and cons. Data architects and software engineers must therefore consider which solution supports these out of the box and the design that best suits the team’s needs prior to implementation.

Read-Through (Lazy Loading)

Applications using the read-through caching pattern essentially treat the cache as the main database. When looking for a specific ID, the application will hit the cache first; if the data is missing, then the cache will be responsible for pulling the data from the main database and updating it.

Write-Through

The write-through caching pattern is similar to the read-through in that instead of writing directly to the main database, the application writes to the cache as if it were the main database. The cache then writes to the main database. This write to the cache is done synchronously and in turn, impacts the write latency.

Write-Behind

Write-behind caching takes a similar approach to write-through, except that writing to the main database is done asynchronously. This means write latency is only limited to the cache latency. Because it is volatile, however, there is a risk of data loss if the cache is down. That being said, some advanced caching solutions provide both performance and no data loss in case of failure.

Eviction Policies

You can’t simply treat your cache as you would a normal database where you store all the data ever entered. This is due to the balance of cost versus performance, as the fastest data tier (RAM) is more expensive than slower tiers (SSDs and hard disks).

Instead, in such cases, an eviction policy can be used to prevent the cache size from going over the maximum limit. The cache achieves this by removing elements according to the rules set by the eviction policy. There are several methods you can implement depending on what the data architect believes will best serve your application.

These methods include:

• Least recently used (LRU): Evicts the data that was least recently used, first.
• Least frequently used (LFU): Instead of removing the data that was least recently used (as with LRU), it evicts data that was least frequently used. Meaning that the data that is set to be removed might have been used recently, but wasn’t used as often as the rest of the data. 
• Most recently used (MRU): In contrast to LRU, MRU removes data that was mostly recently used.
• First in first out (FIFO): This eviction policy removes data that was added in first or least recently. It is unrelated to usage and is based solely on when the data was entered into the cache.

Each approach has its advantages and disadvantages. Ultimately, the goal of all of these policies is to keep only the data you need in the fastest tier. But these policies are not always sufficient. To this end, some more advanced in-memory cache providers offer an additional option:

• Business-Policy Driven: This allows data architects to set an app’s eviction policy to a more nuanced set of rules- and logic-based approach as opposed to a straight forward LRU or FIFO one. This grants full control over which data is saved in cache and when it should be evicted to SSD or disk according to the logic of the business needs, and allows for SLA-driven cost and performance optimization.

Caching Solution Types

Caching solutions can take on many different formats. While the method that might first come to mind for data architects is the key-value cache, this is not the only option, with some cache systems providing developers with additional capabilities.

Key-Value Caching

Some caches are based on a key-value store. This essentially means you can provide the data store a unique key and it will provide a corresponding value back. Because this is all done in-memory, they are commonly used by developers as caching layers.

While some key-value stores are limited in terms of the data types they can store, there are some versions that allow for multiple data types, such as lists, arrays, and sets. 

Generally speaking, key-value caching layers tend to be the first technology developers turn to, but these are not always enough. For simple cases, the key-value solution is often fast and efficient. However as data grows and user queries increase, architects are faced with new challenges, such as the inability to index on more than one criteria (adding secondary indexes).

In such cases, key-value stores generally offer the option to duplicate the entire data store so it can be indexed according to the new key. But this process is hardware-consuming (more expensive) and inflexible. Further, more complex relations (e.g., hierarchical and many-to-many) cannot be implemented resulting in the solution being deadlocked.

Fully Indexed In-Memory Caching

Data architects are not limited to caching in key-value caching systems. They can also take advantage of in-memory computing solutions that offer SQL support. This includes fully advanced indexing capabilities, such as having multiple secondary indexes, as well as index by exact match, ranges, compound, texts and geo-spatial data fields.

Additionally, some caching solutions support full SQL APIs, which enable a more flexible slice and dice of the data and easy migration from the primary data store.

They also provide both advanced indexing and SQL support, allowing for easy on-demand analytics and BI reporting. Without SQL support or advanced indexing, aggregations must be predefined for satisfactory performance, which involves a great deal of coding.

Comparing Advanced Caching to Simple Caching Solutions

	Advanced Caching	Simple Caching
Key-Value Store Support	✔	✔
Cloud Deployment	✔	✔
Hybrid Deployment Support	✔	✔
Scale Out Solution	✔	✔
Enterprise Grade Security	✔	✔
Basic Data Tiering RAM-SSD-Disk (LRU)	✔	✔
Native Secondary Indexing	✔
On-the-fly Server Side Aggregations	✔
Full SQL Compliance	✔
Strong Consistency	✔
Advanced Business Driven Data Tiering	✔
RDBMS 1-Click Integration	✔
On-line Scale Up and Out	✔
1-click-deployment on-prem/cloud and hybrid environments	✔

Why Consider Database Caching

No matter the design pattern or database you rely on, implementing a caching tier may take time and add some complexity to your overall design.

If this is the case, why implement it at all?

Data architecture requires data caching because having data stored locally in memory can help reduce issues such as long latency times between requests and high concurrency of users. In-memory caching can also help reduce run times of requests and batch jobs. Jobs that previously took hours can be reduced to minutes, and jobs that took minutes cut to seconds. In this modern world, where 53% of users are likely to abandon a website that takes more than 3 seconds to load, speed is critical.

Further, some caching systems that are provided as services can not only reduce latency but also scale elastically. Thus as usage volume increases or, for example, you experience an unexpected usage peak, performance is not impacted, and your team is spared having to invest time and effort in manual scaling. With dynamic scaling, you can avoid overprovisioning on private or public clouds.

Summary of Caching Benefits

The right caching platform can help organizations:

• Accelerate digital applications
• Support multiple concurrent clients
• Handle peak loads
• Shorten the batch reporting window
• Accelerate BI and generate BI reports on fresh data
• Offload legacy infrastructure

Caching Case Studies 

Now that we know what caching is, let’s explore some real-life examples of how in-memory caching—and in this case smart in-memory caches—can improve business processes.

Real-Time Trading Risk Reporting System

Many banking and financial institutions still rely on overnight calculations to provide them with risk models, credit valuations, and potential market exposure. They often have hundreds of applications that must constantly be synced with each other to support their analysts, traders, and general employees.

But slow databases and data warehouses can bog down the data access, applications, and models running on these systems.

This is especially relevant for this use case, where a bank was struggling to manage all of its data needs. The bank needed to handle all its modern models, which were connected to hundreds of data sources—that were often impacted by changes happening every minute—in order to accurately manage risk. Updating systems nightly because their data warehouses couldn’t handle the influx of data therefore posed a serious risk.

To mitigate this risk, the bank opted for an in-memory cache that kept the last 24 hours of multiple sources of data loaded to the cache and that offered advanced indexing and full SQL support. This enabled the clients to perform real-time risk analysis calculation on live data.

This model offers significant benefits for financial institutions in particular, and has already been adopted by most of the major banks.

Batch Acceleration for a Leading Airline

Company mergers and partnerships frequently require integrating systems. Often, this process is managed by batch jobs that help manage data flows. However, with large companies in particular, the merging and integration of data can be slow, harming productivity and customer experience.

Two airlines recently merged and needed to integrate their crew management systems, which were responsible for handling crew members’ shifts and which flights they would be on. Adding to the complexity, airline union policies allowed crew members to swap flights.

Due to the many possible risks and stress that could be caused by crew members not knowing whether their requests to switch flights would be approved, the airline was contractually mandated to finish this batch process within four hours.

This was a problem: The batch jobs took nine hours on standard SQL databases and at best half that on NoSQL databases, thus failing to meet the four-hour SLA.

To address this, the companies moved to an in-memory caching system that offered a distributed in-memory smart cache platform. This platform turned their nine-hour batch job into a three-hour batch job, thus meeting their SLA and improving overall user experience.

CO2 Emissions Calculator: Legacy Modernization

Following the 2015 “dieselgate scandal,” the EU was forced to replace its outdated CO2 emissions test. With the new regulations in place, a major European automobile manufacturer was forced to provide a CO2 emissions report for each car.

To help manage this, the manufacturer created an online CO2 emissions calculator for their vehicles. The calculator ran on a production mainframe that could handle up to 200 requests per second. The demand, however, was expected to reach 3,000 queries per second.

They initially attempted to address this through the use of a key-value data store, but with all the different possible configurations permitted by the manufacturer, this resulted in billions of custom configurations. This proved difficult for the key-value store, because the key-value store was optimized for a single criteria query, but the custom configurations required the ability for multi-criteria queries. This made the key-value store inefficient and unusable, and their efforts failed.

The car manufacturer therefore switched to an in-memory smart cache solution that allowed them to offload requests from their mainframe by having an advanced indexing layer of all possible configurations.

This enabled more efficient management of all possible data that could be cached in memory. Moreover, by collecting data and business logic in the same memory space, they were able to dramatically decrease data network movement and overhead while enhancing performance.

Caching services have come a long way in the last decade, now offering features that expand their use cases and make them easier to support and implement.

Modern Caching Could Be the Solution You’re Looking For

Today’s digital application and BI users expect immediate responses. Systems need to be able to handle large amounts of data as well as millions of user requests. Traditional databases and data warehouses are often too slow for real-time advanced access to live data. This is where advanced caching solutions have proven a useful tool for developing systems that need to meet tight SLAs. Modern caching can help improve performance, scalability, and availability for both your applications and BI needs.

The next time your application or BI reporting is running slow or you need to scale your system, might be an opportunity to implement a caching layer and, in turn, help your team meet SLAs and customer expectations where other databases fail.

Need a more in-depth deep dive on in-memory cache? Check out our solution paper. 

Download the Smart Cache Solution Paper