Configuring Falco Docker Container Logging to AWS CloudWatch

When running the Falco Docker container, you may want to send logs to AWS CloudWatch for better monitoring and analysis. Below are the configurations for running Falco in different modes and ensuring logs are captured correctly.

Running Falco in Interactive Mode

To run Falco in interactive mode and send logs to CloudWatch, use the following command:

docker run --rm -i -t --log-driver awslogs \
    --log-opt awslogs-region=us-east-1 \
    --log-opt awslogs-group=falcoint \
    --log-opt awslogs-create-group=true \
    --privileged \
    -v /dev:/host/dev \
    -v /proc:/host/proc:ro \
    -v /boot:/host/boot:ro \
    -v /lib/modules:/host/lib/modules:ro \
    -v /usr:/host/usr:ro \
    -v /etc:/host/etc:ro \
    falcosecurity/falco:latest

This command allows you to see logs in both the console and AWS CloudWatch Logs.

Running Falco in Detached Mode

When you run Falco in detached mode, you might encounter issues where logs do not appear in CloudWatch. Use the following command:

docker run --rm -d --log-driver awslogs \
    --log-opt awslogs-region=us-east-1 \
    --log-opt awslogs-group=falcoint \
    --log-opt awslogs-create-group=true \
    --privileged \
    -v /dev:/host/dev \
    -v /proc:/host/proc:ro \
    -v /boot:/host/boot:ro \
    -v /lib/modules:/host/lib/modules:ro \
    -v /usr:/host/usr:ro \
    -v /etc:/host/etc:ro \
    falcosecurity/falco:latest

If logs still do not appear in CloudWatch, it may be due to buffering. Logs are often sent only when the container stops. You can verify this by running the container without -d or -it flags:

docker run --rm --log-driver awslogs \
    --log-opt awslogs-region=us-east-1 \
    --log-opt awslogs-group=falcoint \
    --log-opt awslogs-create-group=true \
    --privileged \
    -v /dev:/host/dev \
    -v /proc:/host/proc:ro \
    -v /boot:/host/boot:ro \
    -v /lib/modules:/host/lib/modules:ro \
    -v /usr:/host/usr:ro \
    -v /etc:/host/etc:ro \
    falcosecurity/falco:latest

Troubleshooting Log Issues

If you notice that logs are not being sent to CloudWatch when running in detached or foreground mode, check the following:

• Ensure that the AWS permissions for the ECS task or EC2 instance allow writing to CloudWatch Logs.
• Verify that the log group specified in the awslogs-group option exists or is set to create automatically.

Example Log Output

When the Falco container is stopped, you might see logs like this:

2020-06-04T02:33:44+0000: SIGINT received, exiting...
Syscall event drop monitoring:
   - event drop detected: 0 occurrences
   - num times actions taken: 0
2020-06-04T02:32:32.495581404+0000: Notice A shell was spawned in a container...
2020-06-04T02:33:00.014981252+0000: Error File created below /dev by untrusted program...

These logs should ideally be sent to CloudWatch without needing to stop the container. If you find that using -d -t allows logs to be sent, it may be worth investigating the interaction between the logging driver and the container's execution mode.

Conclusion

By following the configurations and troubleshooting steps outlined above, you should be able to successfully send Falco logs to AWS CloudWatch, enhancing your monitoring capabilities.