Fluentd Docker Compose Configuration

This article outlines how to configure Fluentd within a Docker Compose setup, ensuring that it communicates securely over an internal network without exposing ports to the host.

Docker Compose Configuration

In your docker-compose.yml file, you can define the services and their networking settings as follows:

version: '3.8'

services:
  fluentd:
    build: ./fluentd
    container_name: fluentd
    expose:
      - 24224
      - 24224/udp
    depends_on:
      - elasticsearch
    networks:
      - internal

  public-site:
    build: ./public-site
    container_name: public-site
    depends_on:
      - fluentd
    logging:
      driver: fluentd
      options:
        tag: public-site
    networks:
      - internal

networks:
  internal:

Explanation of the Configuration

• Fluentd Service: This service builds from the ./fluentd directory and exposes the necessary ports for Fluentd to receive logs. The expose directive makes these ports available to other containers in the same network but does not publish them to the host.
• Public Site Service: This service builds from the ./public-site directory and is configured to use Fluentd as its logging driver. It depends on the Fluentd service to be up and running before it starts.
• Internal Network: Both services are connected to an internal network named internal, which allows them to communicate securely without exposing ports to the outside world.

Common Issue

When running docker-compose up, you may encounter an error similar to:

ERROR: for public-site  Cannot start service public-site: failed to initialize logging driver: dial tcp 127.0.0.1:24224: connect: connection refused

This error occurs because the Fluentd service is not accessible on 127.0.0.1 when the ports are only exposed internally. To resolve this, ensure that both services are correctly connected to the same internal network, as shown in the configuration above.

Conclusion

By using the expose directive and an internal network, you can keep your Fluentd service secure while still allowing your applications to log effectively. This setup prevents unnecessary exposure of your logging service to the host network, maintaining a tighter security posture.