How to configure access logging in Tomcat

24 Oct 2018 20:29

Introduction
Sometimes we need to log usage activity in tomcat. It could be that tomcat is the main web server for the site and we want to record site activity, (hits, page views, errors). It could be that tomcat is the application server and we want to see if there are any test systems hitting production or it could be a desire to correlate resource requests to exceptions. This HowTo is meant to illustrate the steps necessary to set up access loging in tomcat. At time of this writing, tomcat 6 is still the mainstream version in use, so this document will be using tomcat 6 for examples but I don't expect there to be too many differences that could not be applied to tomcat 5.5 or tomcat 7.

Enabling the Tomcat Access Logger
Tomcat access logging is enabled by modifying the server.xml file and uncommenting the Access Log Valve. In a default tomcat implementation, the access log valve section is located within the Host element. Uncommenting the entry will enable an access log that contains fields equivalent to a "common" log file format from Apache. The defaults for the valve will result in a file named "localhost_access_log" followed by the date, followed by a ".txt" file extension. IP addresses will be logged, not hostnames and log file will be written into the ${tomcat.home}/logs directory. The fields present in the log file using a common format are:

Client host name (recorded as an IP if the default resolveHosts is not changed to "true").

Remote logical username (which always prints a "-").

Remote authenticated user ID (if one exists)

Date and Time of the request

HTTP Method and URI requested

HTTP Response Status Code

Size, in bytes, of the response (excluding https response headers)

Below is a snippet of the relevants parts of a server.xml displaying the newly enabled access logging defaults:

Code: Select all

<Host name="localhost"  appBase="webapps"
	            unpackWARs="true" autoDeploy="true"
	            xmlValidation="false" xmlNamespaceAware="false">

	        <!-- SingleSignOn valve, share authentication between web applications
	             Documentation at: /docs/config/valve.html -->
	        <!--
	        <Valve className="org.apache.catalina.authenticator.SingleSignOn" />
	        -->

	        <!-- Access log processes all example.
	             Documentation at: /docs/config/valve.html -->

	        <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs" 

	               prefix="localhost_access_log." suffix=".txt" pattern="common" resolveHosts="false"/>
</Host>

Customizing the Access Log
The common log format is ok but changing the pattern to combined adds the User-Agent (browser or robot type) and the referring web site and URI. Tomcat also provides additional options to log things like the request protocol, the local port that received the request, user session ID's, incoming or outgoing request headers, etc. A full list is documented at the Tomcat Configuration Reference Valve Component page.

If you are running a version of tomcat greater than version 6.0.21 or tomcat 7, you can take advantage of the new Remote IP Valve. For access logging, the nice thing about this valve is that it will swap the client IP with an IP address passed with the X-Forwarded-For header—automatically—if an IP address is passed in the X-Forwarded-For header. Loading it is pretty easy. Just add the org.apache.catalina.valves.RemoteIpValve to your server.xml before your AccessLogValve declaration. For example:

Code: Select all

<Host name="localhost"  appBase="webapps"
	    unpackWARs="true" autoDeploy="true"
	    xmlValidation="false" xmlNamespaceAware="false">

	  <!-- SingleSignOn valve, share authentication between web applications
	    Documentation at: /docs/config/valve.html -->
	  <!--
	    <Valve className="org.apache.catalina.authenticator.SingleSignOn" />
	  -->

	  <!-- Remote IP Valve -->
	    <Valve className="org.apache.catalina.valves.RemoteIpValve" />

	  <!-- Access log processes all example.
	    Documentation at: /docs/config/valve.html -->

	  <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs" 
	    prefix="localhost_access_log." suffix=".txt"
	    pattern="combined" resolveHosts="false"/>
	  -->
</Host>

This is enough to get you started with the RemoteIP Valve but you're going to want to add some additional settings to customize it so that it is specific to your environment. For example, if there are some F5 BigIP's load-balancing your servers, you will want to add the IP address(es) of the SNAT IP to RemoteIP Valve's internalProxies property.

If you are using a version of tomcat 6 older than 6.0.21 and you want to store the X-Forwarded-For IP address instead, then you could modify the pattern property of your AccessLogValve. You'll need to remove the "common" or "combined" pattern and replace it with one of the following patterns:

Code: Select all

Common Log Format: %{X-Forwarded-For}i %l %u %t "%r" %s %b
Combined Log Format: %{X-Forwarded-For}i %l %u %t %r %s %b %{User-Agent}i %{Referer}i

The main problem here, that RemoteIP Valve does take care of, is that you'll only get the X-Forwarded-For address in the logs. If you hit the app server directly, bypassing the device that is inserting the X-Forwarded-For header in the request, you won't get an IP address logged. You will still log a request—you just will not know where it came from.