I found claude and GPT very helpful on this, because java have a very sofisticated monitoring harness. Just ask the agent to connect to the running application (on kubernetes or whatever) on prod and do a java flight recording then analyze allocations.
I managed to improve some applications of ours from several garbage collections per second to several minutes between collections. That _really_ improves p99.
How about not using Java? Then you can have low latency.
Average go, rust, c++ and c will outperform amazing java programs, and the former will also be way way more easy to run, troubleshoot, interpret logs from.
Had to write a router software once in Java back in 2014. The past experience in me believed that its not a good choice to use something with stop the world GCs for routers. The goal was to have everything under 1ms (max 2ms in rare cases). It has http server on head and websocket connections on the tail end. Turns out the class instantiations and JSON parsing were a bottleneck. Thread safe resources were valuable, the libraries which did not had thread safe resources were re-written to support what we needed. The http server was re-written barebones, the JSON parsing was re-written with barebones. And it worked.
Chronicle has a lot of great resources. I’ve been out of Java for a few years but when I had to write high perf code, the libraries and blog insights were invaluable.
A lot of the advice is good in general - keeping things simple, generating fewer objects, etc. Profiling with Yourkit or JMH to find and improve slow spots.
This let us build high performance software that had a vast difference in say p90 input size and p99. It involved rewriting third party libraries to let us 3x speed and substantially reduce GC. I think in the end we were around a p99 of 7ms and p90 well under that (the data spread was kbs to megabytes) across hundreds of millions of inputs per day.
8 comments
[ 2.9 ms ] story [ 31.1 ms ] threadI managed to improve some applications of ours from several garbage collections per second to several minutes between collections. That _really_ improves p99.
Average go, rust, c++ and c will outperform amazing java programs, and the former will also be way way more easy to run, troubleshoot, interpret logs from.
Java is usch garbage in every stack.
A lot of the advice is good in general - keeping things simple, generating fewer objects, etc. Profiling with Yourkit or JMH to find and improve slow spots.
This let us build high performance software that had a vast difference in say p90 input size and p99. It involved rewriting third party libraries to let us 3x speed and substantially reduce GC. I think in the end we were around a p99 of 7ms and p90 well under that (the data spread was kbs to megabytes) across hundreds of millions of inputs per day.