In this article, we’re going to compare some essential metrics of web applications using two different Java stacks: Spring Boot and Eclipse Micro Profile. More precisely, we’ll implement the same web application in Spring Boot 3.0.2 and Eclipse Micro Profile 4.2. These releases are the most recent at the time of this writing. Since there are several implementations of Eclipse Micro Profile, we’ll be using one of the most famous: Quarkus. At the time of this writing, the most recent Quarkus release is 2.16.2.

This mention is important regarding Eclipse Micro Profile because, as opposed to Spring Boot, which isn’t based on any specification and, consequently, the question of the implementation doesn’t exist, Eclipse Micro Profile has largely been adopted by many editors who provide different implementations, among which Quarkus, Wildfly, Open Liberty and Payara are from the most evangelical.

In this article, we will implement the same web application using two different technologies, Spring Boot and Quarkus, suchthat to compare their respective two essential metrics: RSS (Resident Set Size) and TFR (Time to First Request).

The Use Case

The use case that we’ve chosen for the web application to be implemented is a quite standard one: the one of a micro-service responsible to manage press releases. A press release is an official statement delivered to members of the news media for the purpose of providing information, creating an official statement, or making a public announcement. In our simplified case, a press release consists in a set of data like a unique name describing its subject, an author, and a publisher.

The micro-service used to manage press releases is very straightforward. As with any micro-service, it exposes a REST API allowing for CRUD press releases. All the required layers, like domain, model, entities, DTOs, mapping, persistence, and service, are present as well. Our point here is not to discuss the micro-services structure and modus operandi but to propose a common use case to be implemented in the two similar technologies, Spring Boot and Quarkus, to be able to compare their respective performances through the mentioned metrics.

Resident Set Size (RSS)

RSS is the amount of RAM occupied by a process and consists of the sum of the following JVM spaces:

• Heap space
• Class metadata
• Thread stacks
• Compiled code
• Garbage collection

RSS is a very accurate metric, and comparing applications based on it is a very reliable way to measure their associated performances and footprints.

Time to First Request (TFR)

There is a common concern about measuring and comparing applications' startup times. However, logging it, which is how this is generally done, isn’t enough. The time you’re seeing in your log file as being the application startup time isn’t accurate because it represents the time your application or web server started, but not the one required that your application starts to receive requests.

Application and web servers, or servlet containers, might start in a couple of milliseconds, but this doesn’t mean your application can process requests. These platforms often delay work through the process and may give a false, lazy initialization indication about the TFR. Hence, to accurately determine the TFR, in this report, we’re using Clément Escofier’s script time.js, found here in the GitHub repository, which illustrates the excellent book Reactive Systems in Java by Clément Escoffier and Ken Finnigan.

Spring Boot Implementation

To compare the metrics presented above for the two implementations, you need to clone and run the two projects. Here are the steps required to experience the Spring Boot implementation:

$ git clone https://github.com/nicolasduminil/Comparing-Resident-Size-
Set-Between-Spring-Boot-and-Quarkus.git metrics
$ cd metrics
$ git checkout spring-boot
$ mvn package
$ java -jar target/metrics.jar

Here you start by cloning the GIT repository, and once this operation is finished, you go into the project’s root directory and do a Maven build. Then you start the Spring Boot application by running the über JAR created by the spring-boot-maven-plugin. Now you can test the application via its exposed Swagger UI interface by going here. Please take a moment to use the feature that tries it out that Swagger UI offers. The order of operations is as follows:

• First, the POST endpoint is to create a press release. Please use the editor to modify the JSON payload proposed by default. While doing this, you should leave the field pressReleaseId having a value of “0” as this is the primary key that will be generated by the insert operation. Below, you can see an example of how to customize this payload:

  {
    "pressReleaseId": 0,
    "name": "AWS Lambda",
    "author": "Nicolas DUMINIL",
    "publisher": "ENI"
  }

• Next, a GET /all is followed by a GET /id to check that the previous operation has successfully created a press release.
• A PUT to modify the current press release.
• A DELETE /id to clean-up.

Note: since the ID is automatically generated by a sequence, as explained, the first record will have the value of “1.” You can use this value in GET /id and DELETE /id requests. Notice that the press release name must be unique.

Now, once you have experienced your micro-service, let’s see its associated RSS. Proceed as follows:

$ ps aux | grep metrics
nicolas 31598 3.5 1.8 13035944 598940 pts/1 Sl+ 19:03 0:21 java -jar target/metrics.jar
nicolas 31771 0.0 0.0 9040 660 pts/2 S+ 19:13 0:00 grep --color=auto metrics
$ ps -o pid,rss,command -p 31598
PID RSS COMMAND
31598 639380 java -jar target/metrics.ja

Here, we get the PID of our micro-service by looking up its name, and once we have it, we can display its associated RSS. Notice that the command ps -o above will display the PID, the RSS, and the starting command associated with the process, which PID is passed as the -p argument. And as you may see, the RSS for our process is 624 MB (639380 KB). If you’re hesitating about how to calculate this value, you can use the following command:

$ echo 639380/1024 | bc
624

As for the TFR, all you need to do is to run the script time.js, as follows:

node time.js "java -jar target/metrics.jar" "http://localhost:8080/"
173 ms

To resume, our Spring Boot micro-service has a RSS of 624 MB and a TFR of 173 ms.

Quarkus Implementation

We need to perform these same operations to experience our Quarkus micro-service. Here are the required operations:

$ git checkout quarkus
$ mvn package quarkus:dev

Once our Quarkus micro-service has started, you may use the Swager UI interface here. And if you’re too tired to use the graphical interface, then you may use the curl scripts provided in the repository ( post.sh, get.sh, etc.) as shown below:

java -jar target/quarkus-ap/quarkus-run.jar &
./post.sh
./get.sh
./get-1.sh 1
./update.sh
...

Now, let’s see how we do concerning our RSS and TFR:

$ ps aux | grep quarkus-run
nicolas 24776 20.2 0.6 13808088 205004 pts/3 Sl+ 16:27 0:04 java -jar target/quarkus-app/quarkus-run.jar
nicolas 24840 0.0 0.0 9040 728 pts/5 S+ 16:28 0:00 grep --color=auto quarkus-run
$ ps -o pid,rss,command -p 24776
PID RSS COMMAND
24776 175480 java -jar target/quarkus-app/quarkus-run.jar
$ echo 175480/1024 | bc
168
$ node time.js "java -jar target/quarkus-app/quarkus-run.jar" "http://localhost:8081/q/swagger-ui"
121 ms

As you can see, our Quarkus micro-service uses an RSS of 168MB, i.e., almost 500MB less than the 624MB with Spring Boot. Also, the TFR is slightly inferior (121ms vs. 173ms).

Conclusion

Our exercise has compared the RSS and TFR metrics for the two micro-services executed with the HotSpot JVM (Oracle JDK 17). Spring Boot and Quarkus support the compilation into native executables through GraalVM. It would have been interesting to compare these same metrics of the native replica of the two micro-services, and if we didn’t do it here, that’s because Spring Boot heavily relies on Java introspection and, consequently, it’s significantly more difficult to generate Spring Boot native micro-services than Quarkus ones. But stay tuned; it will come soon.

The source code may be found here. The GIT repository has a master branch and two specific ones, labeled spring-boot and, respectively, quarkus.

Enjoy!