Cloud-Native Java vs Golang
Java once-famous motto: “Write once and run everywhere” is pretty much obsolete these days, the only place we want to run code is inside a container. Where a “Just in time” compiler does not make any sense.
For this reason, probably, the Java ecosystem is in the midst of its transformation in order to become better suited for the cloud. Oracle’s GraalVm allows compiling byte code to Linux executables (ELF) and Rad Heat’s Quarkus, and other frameworks, aspire to make it as easy as bootstrapping a react app. Quarkus also leverages Netty and Vertx.x at its core to build very efficient reactive web services.
Java compiled to executable binaries that launch in milliseconds, with a small memory footprint. That can leverage the Java ecosystem, and can even be written in other JVM languages like Scala and Kotlin!
sounds too good to be true…
If You don’t believe it, you can play with Quarkus using the online project generator or by generating a project locally using the maven plugin.
Golang on the other hand was born in the cloud and has no legacy to burden it when it comes to running in containers. It’s considered to be the programming language of the cloud. Small binaries, fast bootstrap, small memory footprint, all there from day one. And it’s wildly adopted. A serious challenge to the java world.
Does Java stand a chance? only time will tell. However, out of curiosity, I’d like to compare a java cloud-native service with a golang equivalent in terms of performance and development experience.
In this post, I will stress two services. Compare their CPU, RAM, latency, and uptime speed. The services will be launched in a container with the same resource allocation and apache benchmark will make them sweat.
Its a “good enough” benchmark for my case study, since I dont assume to find the best/worst benchmark results but a comparison of two benchmarks executed on the same environment.
The scenario
Both services will connect to a MySQL database that runs in another container, with one table and three rows.
Every service will fetch all three rows, transform them into a domain object, and then write a JSON array response.
Apache benchmark will run 10K requests with a concurrency level of 100, two times for the quarkus JVM version ( to also test a “cold” / “warm” JVM )
The Golang Service
Using a popular reactive web framework called gin that has great benchmarks.
When looking for a golang nonblocking MySQL driver I found nothing, the internet recommends go-sql-driver in a unison, so that’s what I’ll be using.
The golang style is very explicit. An In your face attitude. The main function launches the server, configures the request handler, and opens a DB connection.
Building the native go executable
The Kotlin Cloud Native Service — Quarkus
This is a Kotlin example that roughly follows the quarkus reactive MySql extension guide.
Compared to the go version there are implicit stuff going on, CDI Dependency Injection, Declarative routes using javax annotations, automatic configuration parsing, and datasource/connection creation/server bootstrap. But this is a cost of using a framework, it does the heavy lifting for you and dictates its way of doing things. However, it’s much shorter than the go version and as long as the black magic works I don’t mind!
Under the hood, there’s a Netty reactive web server, wrapped by Vert.x multi-event loop and a Vert.x reactive MySQL driver that can handle multiple DB connections with a single thread.
Also, I can use Kotlin’s amazing collection library to fold a list wherein the go version that has no generics yet ( but coming soon ), and no rich standard collections library I'd had to write or generate it manually.
Building the Java native executable
Basically what I’ve been able to figure out about whats happening in the container that builds our native executable is that SubstrateVM. An embeddable virtual machine designed to be compiled ahead-of-time is linked to our code and compiled as one unit. This is amazing, but not without a cost, SubstrateVM has fewer optimizations then HotSpot Vm and a simpler garbage collector, according to oracle.
The compiler that does this is called “Graal” it’s language agnostic and before it can be used the java bytecode needs to be translated into an intermediate representation, a Truffle language. This is very interesting, A great Graal and Truffle explanation can be found in this post.
Building a java native image looks much more complex, it’s slower, it produces a binary almost twice the size. But it works! And a 35M executable binary file is really nothing compared to a java Uber (fat) Jar that can be easily ten time times larger. 35MB is something you can even put in an aws lambda.
Stressing the services
I’m running all tests on my local machine with the following setup:
Ill be using a:
MacBook Pro (15-inch, 2017)
2.9 GHz Intel Core i7 ( 8 cores )
16 GB 2133 MHz LPDDR3
Ill be using a tool called cAdvisor to monitor the stats of my containers.
The scenarios
- quarkus jvm hotspot container
- quarkus java native container
- golang container
each with the following resources allocated
- 100MB / 0.5 CPU | 200MB / 1 CPU | 300MB / 2 CPU
I’m interested in
- cpu/ram utilization ( how well multi cores are utilized )
- cpu/ram spikes
- cpu/ram idle
- bootstrap time
- response latency avg/max
- throughput ( requests per second )
Now I’m going to run a lot of benchmark tests and collect many data points for each. Feel free to jump to the summary at the end if that’s too much information
github repo with all the code of this experiment can be found here
quarkus jvm hotspot — 100MB / 0.5 CPU
- idle cpu usage 0.25%
- idle ram usage 66MB
- bootstrap time 6s
Stress Test Round 1 ( Cold JVM )
Surprisingly there were no failed requests.
Stress Test Round 2 ( warm JVM )
quarkus jvm hotspot — 200MB / 1 CPU
- idle cpu usage 0.13%
- idle ram usage 66MB
- bootstrap time 3s
Stress Test Round 1 ( Cold JVM )
Stress Test Round 2 ( warm JVM )
quarkus jvm hotspot — 300MB / 2 CPU
- idle cpu/ram same as previous scenario
- bootstrap time 1.1s ( NICE )
Stress Test Round 1 ( Cold JVM )
Stress Test Round 2 ( warm JVM )
Now lets see how the native image will perform.
quarkus java Native — 100MB / 0.5 CPU
- bootstrap time: 0.125s. ( !!! )
- no cpu spike on startup
Stress Test Results
quarkus java Native — 200MB / 1 CPU
- instant bootstrap (0.0125s)
- 4 idle ram usage
- 19 ram usage under stress
- 100% cpu utilization
- no cpu spike on startup
Test results
quarkus java Native — 300MB / 2 CPU
no improvement.
golang — 100MB / 0.5 CPU
- idle cpu 0
- idle ram 2.3MB (nice)
- bootstrap time: fraction of a second
- no cpu spike on startup
The results are a bit skewed. For some reason a tiny fraction of requests take ~7s to complete.
When attempting to run the test again to see if the skewd results reproduce ive the test actualy crushed !
runtime error: invalid memory address or nil pointer dereference. mmm… May be im doing something wrong? It seems theres a bug in the go-sql library. The code that reads from a table is 100% as the documentation says, and works 99% of the time. This should not happen.
golang — 200MB / 1 CPU
I keep getting the runntime error. suspiciously always at the end of the test. However, the correctnes of the go-mysql driver is not a main concern, so im terminating manually the test after it completes 90% of the requests.
CPU / RAM usage under stress
golang — 300MB/ 2 CPU
no significant improvement, all stats are pretty much the same. CPU utilization is under 1.0. I wonder why go wasn’t able to utilize well more cores, Interesting… may be because the process is IO bound, Or may be gin needs to be manually configured to better utilize multiple cores.
Summary
It seems Quarkus is production-ready, it allows an easy JVM / native release/dev modes and allows running native tests locally. And as long as you don’t use reflection or JNI , you are safe with regards to GraalVM configuration. Else you will have to configure the graal compiler yourself, and there are existing solutions for that as well.
Latency and throughput
Both the golang and cloud-native java produced similar results, though slightly in favor of the golang service on average. However, the java native results were more stable. The golang service sometimes responded within 1.25µs and very rarely within 7s.
The JVM after “warming up” produced good results, but worse than the native or go version.
CPU utilization
Both go and native-java seem to under-perform under load when given less than a single-core and displayed no significant improvement when launched with 2 cores. May be because the workload is IO bound. Or because the default configuration of gin/Netty does not account for multiple cores.
JVM on the other hand utilized all cores given to it, and improved its performance in all dimensions.
RAM usage
under stress, 40MB for the java native, and 24MB for the golang service. Not bad in both cases, although the golang version used almost twice less ram.
JVM used 140MB under stress. Exactly as the official quarkus stats. Not bad at all for a JVM but almost 6 times more than the golang version.
Bootstrap time
Both golang and cloud-native java start instantly, the JVM version on the other hand takes a few seconds ( depending on the allocated CPU ) and produces a CPU spike on startup. Which can cause k8s HPA to freak out and spin up twice the amount of pods you want if not properly configured, as described in my other post here.
Development expirience
This is more a religious than a practical question. So ill answer it cautiously. Quarkus creates abstractions well familiar in the java world ( like annotation-based DI ). It starts the service for you and creates the connection pool. It’s possible to use the rich collections standard library and generics. However, it can feel a bit like black magic, and once something stops working you can find yourself feeling helpless. In addition, compiling java code to a native binary is not that simple, there are limitations and caveats that you must be aware of, and not every java library will be compatible with the native compilation, though red hat does a great progress with their extensions. ( java libraries pre-configured for native compilation ). Using a library that is not compatible with the native compilation ( like Guice for example ) will require you to configure Graal VM manually. Which is possible, but is not straight forward like just using the jar. Quarkus and Graal VM are also “relatively” new. So there are many adventures waiting ahead. But because of the dual-mode ( JVM or native ). There is always a fallback in case something with the native version stops working, which is a great workaround to any emerging problem.
Golang on the other hand only now ( after 10 years of existence ) admitted it needs generics. And it certainly does not like implicit stuff going on. This is both good and bad in many ways. In addition, there are fewer tools and libraries available ( for example only one popular blocking MySQL-driver ), although the go community does a really good job catching up. On the other hand, its compilation and build process is so much faster / simpler. And every golang package will work for you without the limitations the java-native platform introduces.
Conclusion
Its great that java is becoming cloud native and Golang does not vastly over performs it like it does to the JVM . I’m sure its going to be wildly used in the future. But golang can definitely give a fight.
So choose carefully !
And don’t forget to water the cactus
