Make your Radiator log data searchable

This is the first part of blog series that helps you to use log data that Radiator generates. Jump to second part.

Radiator exports AAA (authentication, authorisation, accounting) data to various formats. You can process the log data further by other log collection systems, such as Splunk and Elasticsearch. In this article, we briefly describe how to export data in JSON format. The common use case is to record the metrics that best describe your environment, for example, authentication, and authorisation messages.

The image below shows you an example of visualised Radiator worker statistics. The graphics were created with Grafana. Click the image for a larger view.

Adding a new field

With Radiator, it is possible to export log data in JSON format (for more information, see JSON.org). Basically, JSON is a set of name-value pairs. The values can also be ordered lists and it is possible to nest lists inside other lists. This makes it possible to express complex data structures in an universal manner with JSON.

Usually, the hardest part in modifying configuration is to figure out how to synchronise modifications everywhere, especially if the logs are centrally collected and parsed. For example, if Client-Identifier or some other RADIUS attribute is added to a log message when authentication fails, you have to ensure the log parser engine understands the new field.

This is an example of AuthLog FILE, which has date, username, and result.

Wed May 18 15:48:44 2016:mikem:FAIL

If you add a new field, the log entry looks like this:

Wed May 18 15:48:44 2016:mikem:client-1:FAIL

Here is the same information as a default JSON message without the new field:

{"timestamp":"2016-05-18T15:48:44Z","result":"reject","source_host":"osc-dev-3","username":"mikem","type":"authentication"}

Here is the JSON message with the new field: {"timestamp":"2016-05-18T15:48:44Z","result":"reject","source_host":"osc-dev-3","username":"mikem","type":"authentication", “client”:”client-1”}

With JSON, it is easy to add the new field to Radiator log message. Usually, there is no need to modify the parser configuration since the fields are just a group of name-value pairs and not fixed together in any way.

Configuring Radiator

The configuration process is straightforward: add Log <FILE, SYSLOG, ...> clause and use it in the same way as existing ones to your Radiator config and you are done. With Radiator, you can customise your own LogFormatHook and add, remove, or modify the fields. This is how Radiator extends the log usage possibilities even further.

Note: The following configuration example needs Radiator 4.16 with latest patches. You must have JSON module installed. JSON::XS module is recommended (see https://metacpan.org/pod/JSON and https://metacpan.org/pod/JSON::XS).

Configuration example: JSON output to radius.cfg (source goodies/logformat.cfg):

---

# This logger logs events in JSON format. It requires the Perl JSON

# module. Note the specific requirement for loading the logger module.

<Log FILE>

       Identifier mylogger-json

       Trace 4

       Filename %L/logfile.json

       LogFormatHook sub { Radius::LogFormat::format_log_json(@_); }

</Log>

# This auth logger logs both successes and failures to a JSON file.

<AuthLog FILE>

       Identifier myauthlogger-json

       Filename %L/authlog.json

       LogFormatHook sub { Radius::LogFormat::format_authlog_json(@_); }

       LogSuccess 1

       LogFailure 1

</AuthLog>

# This is the Handler-clause.

<Handler>

   <AuthBy FILE>

       Filename %D/users

   </AuthBy>

   AuthLog myauthlogger-json

   # Log accounting messages in JSON format.

   AcctLogFileName %L/acctlog.json

   AcctLogFileFormatHook sub { Radius::LogFormat::format_acctlog_json(@_); }

</Handler>

---

In this example configuration, all log data is saved into a single file. This may cause problems in the real configuration because of increasing log data file size. You can avoid this by using log rotation tools, for example, logrotate in Unix-based systems. Rotating log files can safely be done without restarting Radiator. Radiator also supports the special characters in the file names.

Do you want to know more?

Your JSON files are now ready, the next step is to use them efficiently. In the next part of the series, we will introduce the more detailed use cases, which will help you get the most out of Radiator logging.