Send docker logs to Stackdriver using gcplogs

 In Docker, Logging, Stackdriver

When it comes to logging solutions you can either choose a managed or an unmanaged solution. If you decide to go with the latter and you are in Google Cloud Platform the recommended option is Stackdriver Logging.

Google offers a custom fluentd based package to send logs to stackdriver (we’ll talk about it in an upcoming post) but if you are only concerned about docker logs and want to avoid installing stuff in the compute instances there is an easier option: gcplogs.

Requirements

To be able to send logs to Stackdriver the first you need to do is to create a service account and assign it to your compute instances. The following permissions must be set on the new service account:

  • role/metricWriter
  • role/loggingWriter

You might also need to set the following scope in the compute instance or the instance template that should be using the service account:

  • scope: cloud-platform

We won’t go over it step-by-step, I’m assuming that you already know how to do that.

Tell docker to use gcplogs driver

WARNING: Using gcplogs driver will prevent you from inspecting logs with the “docker logs” command, you will only be able to access them from Stackdriver Logging.

This driver is already shipped with docker, so you won’t need to install anything besides docker daemon. Depending on how we run the containers the usage of the driver might differ slightly.

Using docker run

docker run --log-driver=gcplogs ...

Using docker-compose

nginx:
    image: nginx
    container_name: nginx
    logging:
        driver: gcplogs
    ports:
        - 80:80

Can I send logs from external sources?

Sure! If you would like to send docker logs to Stackdriver from outside GCP environment (for example, from an AWS instance) you still can do it! You just need to set the environment variables with the credentials and start docker with a few additional parameters.

First, you will need to set the GOOGLE_APPLICATION_CREDENTIALS variable (there are official docs about this: https://docs.docker.com/machine/drivers/gce/). You should be able to use the same service account we created above.

Then you just need to run docker with the following parameters:

docker run --log-driver=gcplogs \
--log-opt gcp-project=test-project
--log-opt gcp-meta-zone=west1 \
--log-opt gcp-meta-name=`hostname` \
your/application

Visualizing logs

By default all the logs are grouped by instance. If you have an autoscaler and therefore multiple replicas of the container running you might want to filter the logs based on the container name. To do so you just need to write the desired query:

resource.type="gce_instance" "my_container_name"

Stackdriver offers you the option to store queries to access them anytime without writing them again. To do so just write your rule and click “Save Search”. Then, you’ll find it under “Show library”:

And if you just want to see ALL the logs sent by docker you can click on the dropdown menu “All logs” and select “docker-gcplogs-driver”. By setting this option you’ll be able to visualize all the logs that were sent using this driver.

To summarize

As you can see it was pretty easy to set everything up. Although it is usually recommended to use stackdriver logging agent in my opinion this is still a valid solution and might be of some use when you don’t want or don’t have time to pack a custom image or install additional stuff in the instances.

The most anoying part is the creation of the service account and setting the correct permissions and scopes, but you would have to deal with that also when using stackdriver logging agent.

I hope this post was of any use to some of you. See you in the next one!

Recent Posts

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.