Directly Streaming Application Logs to Fluentd

In the context of 12-factor applications, it is essential for apps to write logs to stdout and stderr. This approach allows for better log management and aggregation. However, you might wonder how to send these log streams directly to Fluentd without relying on traditional logging systems like rsyslog or syslog.

Why Use Fluentd?

Fluentd is a versatile log collector that can aggregate logs from various sources, making it an excellent choice for cross-platform applications. By bypassing rsyslog/syslog, you ensure that your application remains compliant with the 12-factor methodology, which emphasizes simplicity and portability.

Configuration Steps

To achieve direct logging to Fluentd, you can use the following approach:

1. Install Fluentd: Ensure that Fluentd is installed and configured to receive logs on a specific port.
2. Configure Fluentd Input: Set up an input source in your Fluentd configuration to capture logs from your application.
3. Modify Application Logging: Adjust your application to write logs to stdout/stderr, which Fluentd will then capture.

Example Fluentd Configuration

Here’s a sample Fluentd configuration that listens for logs on port 24224:

<source>
  @type forward
  port 24224
</source>

<match **>
  @type stdout
</match>

Application Logging Example

In your application, ensure that logs are written to stdout. For example, in a Node.js application:

console.log('This is an info log');
console.error('This is an error log');

Conclusion

By configuring your application to log directly to stdout and setting up Fluentd to capture these logs, you can maintain compliance with the 12-factor app principles while leveraging Fluentd's powerful log aggregation capabilities.