Docker is a set of platform as a service products that deliver software in containers. This integration is a Docker container that uses Filebeat to collect logs from other Docker containers and forward them to your Layerlog account.

To use docker-collector-logs, you’ll set environment variables when you run the container. The Docker logs directory and docker.sock are mounted to the container, allowing Filebeat to collect the logs and metadata.

Upgrading to a newer version

  • Upgrading to a newer version of docker-collector-logs while it is already running will cause it to resend logs that are within the ignoreOlder timeframe. You can minimize log duplicates by setting the ignoreOlder parameter of the new docker to a lower value (for example, 20m).

  • Version 0.1.0 of docker-collector-logs includes breaking changes. Please see the project’s change log for further information.

Deploy the Docker collector

Pull the Docker image

Download the logzio/docker-collector-logs image.

docker pull logzio/docker-collector-logs
Run the Docker image

For a complete list of options, see the parameters below the code block.👇

Docker
docker run --name docker-collector-logs \
--env LOGZIO_TOKEN="<<LOG-SHIPPING-TOKEN>>" \
-v /var/run/docker.sock:/var/run/docker.sock:ro \
-v /var/lib/docker/containers:/var/lib/docker/containers \
logzio/docker-collector-logs
Docker Swarm
docker service create --name docker-collector-logs \
--env LOGZIO_TOKEN="<<LOG-SHIPPING-TOKEN>>" \
--mount type=bind,source=/var/run/docker.sock,target=/var/run/docker.sock \
--mount type=bind,source=/var/lib/docker/containers,target=/var/lib/docker/containers \
--mode global logzio/docker-collector-logs
Parameters
Parameter Description Required/Default
LOGZIO_TOKEN Your Layerlog account token. Replace <<LOG-SHIPPING-TOKEN>> with the token of the account you want to ship to. Required
LOGZIO_REGION Layerlog region code to ship the logs to. This region code changes depending on the region your account is hosted in. For example, accounts in the EU region have region code eu. If you don’t specify this parameter, the default value will be used. For more information, see Account region on the Layerlog Docs. (US region)
LOGZIO_TYPE The log type you’ll use with this Docker. Declare your log type for parsing purposes. Layerlog applies default parsing pipelines to the following list of built-in log types. If you declare another type, contact support for assistance with custom parsing. Can’t contain spaces. Docker image name
LOGZIO_CODEC Set to json if shipping JSON logs. Otherwise, set to plain for plain text format. plain
ignoreOlder Set a time limit on back shipping logs. Upgrading to a newer version of docker-collector-logs while it is already running will cause it to resend logs that are within the ignoreOlder timeframe. You can minimize log duplicates by setting the ignoreOlder parameter of the new docker to a lower value (for example, 20m). 3h
LOGZIO_URL URL for your account listener host. The URL changes depending on the region your account is hosted in. You can skip this parameter if you specify LOGZIO_REGION. If neither LOGZIO_URL nor LOGZIO_REGION is specified, the default value will be used. For more information, see Account region on the Layerlog Docs. listener.layerlog.com:5015
additionalFields Include additional fields with every message sent, formatted as "fieldName1=fieldValue1;fieldName2=fieldValue2". To use an environment variable, format as "fieldName1=fieldValue1;fieldName2=$ENV_VAR_NAME". In that case, the environment variable should be the only value in the field. If the environment variable can’t be resolved, the field is omitted. --
matchContainerName Comma-separated list of containers you want to collect the logs from. If a container’s name partially matches a name on the list, that container’s logs are shipped. Otherwise, its logs are ignored. Note: Can’t be used with skipContainerName --
skipContainerName Comma-separated list of containers you want to ignore. If a container’s name partially matches a name on the list, that container’s logs are ignored. Otherwise, its logs are shipped. Note: Can’t be used with matchContainerName --
includeLines Comma-separated list of regular expressions to match the lines that you want to include. Note: Regular expressions in this list should not contain commas. --
excludeLines Comma-separated list of regular expressions to match the lines that you want to exclude. Note: Regular expressions in this list should not contain commas. --
renameFields Rename fields with every message sent, formatted as "oldName,newName;oldName2,newName2". To use an environment variable, format as "oldName,newName;oldName2,$ENV_VAR_NAME". When using an environment variable, it should be the only value in the field. If the environment variable can’t be resolved, the field will be omitted. --
HOSTNAME Include your host name to display it for the field agent.name. If no value is entered, agent.namedisplays the container id. ''
multilinePattern Include your regex pattern. See Filebeat’s official documentation for more information. ''
multilineNegate Include 'true' to negate the pattern. Note: Cannot be used without multilinePattern. See Filebeat’s official documentation for more information. 'false'
multilineMatch Specifies how Filebeat combines matching lines into an event. The settings are after or before. The behavior of these settings depends on what you specify for negate. Note: Cannot be used without multilinePattern. See Filebeat’s official documentation for more information. 'after'

By default, logs from docker-collector-logs and docker-collector-metrics containers are ignored.

Check Layerlog for your logs

Spin up your Docker containers if you haven’t done so already. Give your logs some time to get from your system to ours, and then open Kibana.

If you still don’t see your logs, see log shipping troubleshooting.