syslog-ng Open Source Edition 3.2: real-time log message correlation in the syslog daemon

The latest version of syslog-ng Open Source Edition (OSE), a replacement for the syslog logging daemon, improves its message classification and identification engine and enables system administrators to correlate log messages in real-time.
By: Róbert Fekete
Spread the Word
Listed Under

Open Source



Dec. 17, 2010 - PRLog -- The syslog-ng pattern database, which was introduced almost two years ago, allows for real-time message identification and classification by comparing the incoming log messages to a set of message patterns. The classification engine of syslog-ng is much faster and scalable than using regular expressions to identify messages, and also permits the administrator to extract relevant information from the message body or to add custom metadata (for example, tags) to the log messages. The new message correlation feature extends the syslog-ng pattern database and makes it possible to associate related log messages, and to treat the information from these messages as if they were a single event.

Another addition to the message classification is the possibility to trigger new messages for identified or correlated messages, creating a base for a flexible alerting framework.

To ease the task of creating message patterns to identify log messages, syslog-ng provides a separate application called pdbtool that uses clustering techniques to group identical events and automatically recognize the changing parts (for example, IP addresses) of the log messages. With the pdbtool application it is also possible to process existing log files to classify and correlate the already stored log messages, and extract and format relevant information from them, which can be real handy for example in forensic situations.

Perhaps the most important, albeit less technical change in syslog-ng OSE 3.2 is its new licensing model. In the recent years, syslog-ng has been licensed under a dual license – the Open Source Edition published under GPL, while a commercial version called Premium Edition was available under a proprietary license. This model hindered community contributions to syslog-ng, because required a contributory agreement from developers working on the syslog-ng codebase. To make syslog-ng more open and accessible to developers and contributors, syslog-ng OSE 3.2 is licensed under an LGPL+GPL combo, with the core of syslog-ng being LGPL, and its main functionality released as plugins under GPL.

Another effort is to be able to collect nonstandard and non-syslog messages centrally like normal log messages. As a first step of this development, syslog-ng Open Source Edition 3.2 can collect the process accounting (pacct) logs of Linux systems.

Version 3.2 of syslog-ng also offers refinements and improvements of several existing features, including the ability to modify a log message when a certain condition is met, to dynamically create its own configuration files to adapt to a particular environment, or the improved performance when storing log messages in SQL databases.

"This version has the largest list of features ever since the syslog-ng project was born. But the development of syslog-ng does not stop here, we have already started work on 3.3, which will focus on improved support for multicore and multithreaded operations to further increase the performance of syslog-ng." - says Balazs Scheidler, lead developer of syslog-ng and CEO of BalaBit.

Further information
* BalaBit IT Security Ltd. -
* The syslog-ng homepage -
* The syslog-ng Open Source Edition 3.2 Administrator Guide -
* Download syslog-ng -
* Changelog of syslog-ng Open Source Edition 3.2 -

# # #

BalaBit IT Security is a developer of unique network security solutions like:
- the syslog-ng system logging software,
- and the BalaBit Shell Control Box, an appliance that can transparently control, audit, and replay SSH, RDP, and Telnet traffic.
Email:*** Email Verified
Tags:Syslog-ng, Log Message, Open Source, Pattern Database, Framework
Industry:Software, Technology, Security
Account Email Address Verified     Disclaimer     Report Abuse
BalaBit IT Security PRs
Trending News
Most Viewed
Top Daily News

Like PRLog?
Click to Share