Log4j and Other Bloatware

I tire of AWS Lambda cold starts in Java. I’m sure there are further optimisations to be made, but today’s efforts yielded a new culprit for cold start delays.

In general the cold start is proportional to the package size. We have a package that we want to serve requests on a REST API linked to the UI and the original cold start time of 8 seconds was deeply unsatisfactory.

Step one was to replace MySQL connector the latest version of which is 2.3Mb. One must assume Oracle can ONLY bloat thing. The MariaDB equivalent is 0.6Mb. This is perfectly compatible with AWS Aurora.

But Log4j…?

Between log4j-core and log4j-api there’s a WHOPPING 2Mb of classes. What the hell’s it doing?

For very sound technical reasons, we’re also using sl4j in this project, which is a more sedate library kilobytes, not megabytes.

But how do we live without Log4J, especially when Amazon has specialised Log4j support which allows its logger to put request context information to the logs in a neat manner. But what does it do?

Amazon’s Lambda Log with Log4j ain’t all that…

It doesn’t do that much. In fact there’s a replacement for it that uses Sl4j with Logback.

All that the actual aws lambda logger does is:

  • turn on a magic feature in the Lambda runtime where the core runtime tells log4j’s MDC about the request id
  • uses Logger to log to console, which itself just maps to System.out

So the above replacement does that, but using sl4j and logback.

But that still adds a couple more hundred k of bloat to the project.

Enter TinyLog

TinyLog is another logging alternative. It has a bridge from sl4j and it is pretty configurable. It’s also a couple of hundred k in size all in.

The process of making this work with a lambda:

  • Import the tiny log dependencies – core and sl4j
  • Create a properties file to configure it
  • Add the log4j-over-sl4j project, which allows log4j’s MDC to be forwarded to sl4j’s equivalent, which is forwarded, in turn, to tinylog’s
  • Find a way to tell Lambda to turn on the feature hidden inside itself to inform the MDC of things

Configuration

My TinyLog properties file:

writer       = console
writer.level = info
exception    = unpack, strip: jdk.internal
writer.format = {date} {context:AWSRequestId} {level} {class}.{method}() - {message}

That gives me a log4j style output, and the context:AWSRequestId will include the request ID as provided by Lambda to the Log4j MDC class being faked by sl4j and forwarding through… it’s a bit complex, but it’s SMALLER than doing it natively!

Finally, we need to find a place to execute this code:

// turn on MDC mapping
LambdaRuntimeInternal.setUseLog4jAppender(true);

This needs to be activated as early as possible in the app start up. Ideally it should be in a static constructor. I made my lambda handler classes inherit a base class with this in the static initialization.

It’s Faster

Package size down from 6.8Mb to 5.2Mb – cold-startup a couple of seconds faster.

Nuff said!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s