Cool! As someone who has extensively used Log4j, it's nice to see STF investing in the technology. Log4j has been a bedrock in the Java-based software world for a long time now. The security vulnerabilities that sent everyone scrambling back December 2021 highlighted the need for more sustained efforts to enhance its security and functionality.
I would have assumed someone would build an alternative in the style of openssl alternatives. Log4j does some very complex things and has huge attack surface area. Does the 400k lines of code reflect actual, inherent complexity?
Sort of. At its core is a logging interface, but there is a factory that gives you a configured local instance based on requesting a logger by name or by hierarchal class name.
You have log levels but also markers which act as hierarchal filters. Based on the supplied configuration, something like debug-level log statements may become a no-op for a section of code.
The appenders are an extension point and rather diverse - you can go to a file, a REST server, a database, etc.
Writing a log message to a database or REST server may take a while, so there is a system to dispatch logging events to separate worker threads.
Logging messages may be localized. This means there needs to be a parameterization system to inject data values into localized messages.
You also may want to collect data from various parts of the system as available and log them, basically accumulating information as you go. There are thread contexts maps and stacks to accumulate such data, which can also be parameterized into messages.
It's a highly flexible system without clear security boundaries. Maybe a log message comes from user input rather than being a parameter on a formatted log message. Maybe that user-supplied message contains a parameter itself. And maybe the parameters are written such that they are fulfilled by making system calls.
400K LoC is still far too huge for that kind of flexibility. Most of lines are probably attributed to the "batteries" included in Log4j. For example `log4j-core` contains too many plugins by default, including the notorious `JndiLookup`. It may have a modular interface, but its packaging is much less modular than it should be.
They’ve stated several times that version 3.x (in development) largely modularizes the system. In fact, the JNDI functionality has already been split out into its own repository for other JakartaEE/JavaEE integrations.
I’ve done this before, don’t remember regretting it and always wonder why more don’t do it. I’ve even avoided any logging configuration files by configuring programmatically during application startup.
I like minimal application configuration and find the “configuration surface” offered by most logging libraries much too large and also see it as a burden for users to learn/google the configuration file format of the chosen implementation.
I prefer to offer minimal and ergonomic logging configuration options. Analogous to basic “-q”, “-v”, “-d” options typical of command line programs, which my code translates to whatever logging framework.
And you don’t end up spending ages fiddling with slf4f and other exclusions and version range stuff in maven.
The most annoying part of logging frameworks is figuring out their logging format file.
A very very very close second is figuring out where to put the fucking config file so the app will pick it up in preference to whatever defaults or packaged settings are.
There's an article on the feed today about events that was hitchikers something and then threw about 10 more layers of sarcasm and meme on the subject, and one of the frustrations I have with current logging frameworks is that a truly mature logging framework needs a LOT of interesting things:
- how about a per-user log?
- how about a per-core or per-node log?
- how about a per-request log?
- event extraction and publishing
And of course the issue of cross-system-boundary tracking/tracing.
Sure, log aggregation and mining can help, but it seems like a very very big expensive hammer for an intricate peg and hole.
Anyway, Log4j was written from the age of computer science where computers were single machine monoliths with a database (two-tier or three-tier: that's right systems architect in two or three boxes rather than https://www.youtube.com/watch?v=y8OnoxKotPQ
why using SLF4J with Log4j, when you can use it over LogBack, Log4J2 or JUL ?
I would always stick with SLF4J because gives that flexibility of not being married to a single logging library/implementation
2) you don't control what your library dependencies use. This can be a mix of log4j, log4j2, java util logging, and commons-logging. These are all commonly used in libraries. If you don't use slf4j, you end up with a console full of messages that are formatted slightly different from each other.
So even if you use JUL logging, you should use slf4j to ensure some uniformity. And if you do that, you might as well take the next step and format your logs properly using log4j2 or logback. It's not that hard with slf4j, you can use any of the common logging APIs and it will all end up in the same place.
If you use most common server frameworks (e.g. Spring), you have no choice. Slf4j is going to be on the classpath and it will likely come in via any number of other dependencies as well.
People still write tutorials about the obsolete HttpUrlConnection
in 2023, when HttpClient exists.
In fact, almost every API in the Java standard library is bad and once they realize the necessity to replace it, they add a second actually usable API that nobody uses.
The system logger is intentionally designed for use by standard Java classes, not by user code. It’s to help unify the various ways JDK functionality configures logging so that it can be delegated to a proper library like Log4j.
Yes, it has a few design flaws that were never addressed that make it a bit slow and inefficient. Also it is a bit lacking in features.
Separating how logs are formatted and published from how you log from your code is a good idea. This is essentially what slf4j does. It will happily intercept java util logging, log4j, and other common logging APIs using API compatible stubs for each of these and redirect them to your preferred logging framework. What things like log4j2 and logback offer is a lot of control over how the logs are processed, formatted, and published.
Logback has plugins, appenders for different logging backends, etc. I actually have my own plugin for logback that writes logs straight to a remote elasticsearch cluster. It does things like add MDC (mapped diagnostic context) fields and selected environment variables to the json blob that it sends to elasticsearch and a few more useful things. That makes it easy to make logging dashboards. Log4j2 has similar features. Java util logging is a bit barebones. It's what you use when you really don't care about your logs.
I have used logback for almost 15 years now instead of log4j.
util logging and commons logging weren't nearly as good, although I don't recall specifically why. jdk logging was fine-finer-finest and some other weird things
Ultimately what is important in javaland is adopting SLF4J and then delegating to whatever you want to use beyond that for the actual logging.
My gut reaction was "what a waste of money, just move on to literally any other logging framework". Then I remembered that reality is still a thing, and that there are thousands of companies running oodles of unmaintained applications. Applications to which they probably don't have the source code, and if they do, are unable to build new versions because of bit rot. And that, maybe, these people can get away with replacing the Log4J JAR file in their installation. Maybe.
The state of software security seems truly abysmal. Which is no surprise, given that many companies outsource development, receive a working product, and then the development team moves on and no one ever seems to bother with security maintenance.
29 comments
[ 2.8 ms ] story [ 65.9 ms ] threadYou have log levels but also markers which act as hierarchal filters. Based on the supplied configuration, something like debug-level log statements may become a no-op for a section of code.
The appenders are an extension point and rather diverse - you can go to a file, a REST server, a database, etc.
Writing a log message to a database or REST server may take a while, so there is a system to dispatch logging events to separate worker threads.
Logging messages may be localized. This means there needs to be a parameterization system to inject data values into localized messages.
You also may want to collect data from various parts of the system as available and log them, basically accumulating information as you go. There are thread contexts maps and stacks to accumulate such data, which can also be parameterized into messages.
It's a highly flexible system without clear security boundaries. Maybe a log message comes from user input rather than being a parameter on a formatted log message. Maybe that user-supplied message contains a parameter itself. And maybe the parameters are written such that they are fulfilled by making system calls.
If that's not a factory which makes factories which makes factories that make logging objects, we've got work to do!
I like minimal application configuration and find the “configuration surface” offered by most logging libraries much too large and also see it as a burden for users to learn/google the configuration file format of the chosen implementation.
I prefer to offer minimal and ergonomic logging configuration options. Analogous to basic “-q”, “-v”, “-d” options typical of command line programs, which my code translates to whatever logging framework.
And you don’t end up spending ages fiddling with slf4f and other exclusions and version range stuff in maven.
A very very very close second is figuring out where to put the fucking config file so the app will pick it up in preference to whatever defaults or packaged settings are.
There's an article on the feed today about events that was hitchikers something and then threw about 10 more layers of sarcasm and meme on the subject, and one of the frustrations I have with current logging frameworks is that a truly mature logging framework needs a LOT of interesting things:
- how about a per-user log? - how about a per-core or per-node log? - how about a per-request log? - event extraction and publishing
And of course the issue of cross-system-boundary tracking/tracing.
Sure, log aggregation and mining can help, but it seems like a very very big expensive hammer for an intricate peg and hole.
Anyway, Log4j was written from the age of computer science where computers were single machine monoliths with a database (two-tier or three-tier: that's right systems architect in two or three boxes rather than https://www.youtube.com/watch?v=y8OnoxKotPQ
1) It's a decent API. Simple, easy to use.
2) you don't control what your library dependencies use. This can be a mix of log4j, log4j2, java util logging, and commons-logging. These are all commonly used in libraries. If you don't use slf4j, you end up with a console full of messages that are formatted slightly different from each other.
So even if you use JUL logging, you should use slf4j to ensure some uniformity. And if you do that, you might as well take the next step and format your logs properly using log4j2 or logback. It's not that hard with slf4j, you can use any of the common logging APIs and it will all end up in the same place.
If you use most common server frameworks (e.g. Spring), you have no choice. Slf4j is going to be on the classpath and it will likely come in via any number of other dependencies as well.
It's an old and tired API, inefficient, slow... you can do better by using the newer System.Logger which fixes those issues, mostly: https://docs.oracle.com/en/java/javase/17/docs/api/java.base...
And of course the python logging library was inspired on it. Sigh...
One part of the std python lib that needed a big overhaul
People still write tutorials about the obsolete HttpUrlConnection in 2023, when HttpClient exists.
In fact, almost every API in the Java standard library is bad and once they realize the necessity to replace it, they add a second actually usable API that nobody uses.
Separating how logs are formatted and published from how you log from your code is a good idea. This is essentially what slf4j does. It will happily intercept java util logging, log4j, and other common logging APIs using API compatible stubs for each of these and redirect them to your preferred logging framework. What things like log4j2 and logback offer is a lot of control over how the logs are processed, formatted, and published.
Logback has plugins, appenders for different logging backends, etc. I actually have my own plugin for logback that writes logs straight to a remote elasticsearch cluster. It does things like add MDC (mapped diagnostic context) fields and selected environment variables to the json blob that it sends to elasticsearch and a few more useful things. That makes it easy to make logging dashboards. Log4j2 has similar features. Java util logging is a bit barebones. It's what you use when you really don't care about your logs.
util logging and commons logging weren't nearly as good, although I don't recall specifically why. jdk logging was fine-finer-finest and some other weird things
Ultimately what is important in javaland is adopting SLF4J and then delegating to whatever you want to use beyond that for the actual logging.
The state of software security seems truly abysmal. Which is no surprise, given that many companies outsource development, receive a working product, and then the development team moves on and no one ever seems to bother with security maintenance.