From time to time, our customers reach out to us and ask guidance on how to speed up RADIUS authentication processing during peak hours. Performance of this core functionality of Radiator is stable and predictable, and with increasing traffic load, most of the bottlenecks often emerge from the performance of the backend services that Radiator is utilising. In this blog, we give some helpful information to make improvements.
What causes bottlenecks?
In many cases, authentication backends that Radiator is configured to use may be too slow to process the number of requests that are arriving. For example SQL databases may be slow to respond, LDAP lookups can take time, or HTTP requests be delayed. Writing accounting data or logging to a slow database can slow down Radiator.
When this happens, the host system's UDP traffic buffer on the listening port fills up to the point that it overflows and new RADIUS request packets arriving at the server get dropped. Because the requests are dropped by the host operating system, they cannot even be logged as failed requests by Radiator.
To get a better understanding of what is happening during peak hours, set `Trace 4` debug log level with `LogMicroseconds` in the configuration file (see [How to enable debug logging?]. The debug log will show how long each processing step is taking, and from there you can determine how many requests per second the configuration can handle.
How to tune Radiator to speed up authentication processing?
There are several ways to tune Radiator to speed up processing:
- Split configuration into two separate files, one for authentication and one for accounting, and run Radiator as two separate running instances. This way there are two processes handling traffic with their own UDP buffers and backend dependencies that may even out the congestion. On Linux systems with systemd there are mechanisms readily available to help with that. More information: [How do I manage Radiator instances on a single host?].
- Use a proxy instance to forward authentication and/or accounting requests to multiple worker instances (see first bullet). The requests can be spread out to worker instances so that request state is preserved (see [methods for load distribution and balancing] in documentation).
- Within instances, use [FarmSize] to create a farm of server processes to add parallel processing of requests. Note: This is not compatible with many EAP protocols, such as EAP-TLS, EAP-TTLS, PEAP etc. This is because such protocols rely on authentication state being held within each server process, and it is necessary for all the requests for such protocols to go to the same Radiator process.
- Incoming requests can be distributed from one or two proxying or load-balancing hosts to multiple hosts for processing.
- Depending on use case, any combination of FarmSize, multiple instances per host and multiple hosts can be used together to enable more throughput. Separating processing on the Radiator side will create more separate connections to backends that may help even out backend load.
- If the host is running other processes, make sure it has enough resources available for Radiator. The best would be to use dedicated hosts for each function.
- If a virtualisation platform is used, make sure that it can provide the configured resources to Radiator virtual machines even during the peak hours when other vm's are running their peak loads.
- Logging to external servers may cause some unneeded delays for Radiator. This can be mitigated by configuring Radiator to log locally, and then process and ship logs with a separate agent.
- Outside of Radiator, performance of backends may be tuned depending on the backend itself. For example a database can be scaled up or replicated to multiple-server cluster. On the other hand, speeding up the database might be possible even without adding more hardware if queries can be optimised or suitable indexes created. For optimising backends, please refer to their respective documentation.
- Running configuration or Radiator may contain some parts that can be optimised. At the end of this blog post, we tell a bit about our consultation services.
