Blog

A selection of musings from the tray.io team.

Small, simple and secure way of sending Docker container logs to logentries
By Luka

tl;dr;

Send all container logs on the host to logentries log with the token <token>:

$ docker run -d -v /var/lib/docker:/var/lib/docker:ro tray/heka-logentries:latest <token>

For the curious, head over to our GitHub repo and check the content.

How Stuff Works

To keep things relatively small we decided to use progrium/busybox image as our base. This gave us a starting size of just 5MB. We could use the official busybox image which is half the size, but in the end it wouldn't make much of a difference.

Heka adds another 37MB when extracted, but it's well worth it. We were already using it for a similar task and based on the positive experience decided to stick with it.

After configuring Heka to watch container logs and send data to logentries using TCP with TLS we were almost done.
Sending logs to logentries requires user to create a log which is defined by a token. This token must be present in every log line sent, otherwise logentries will reject it.

Deconstruction of the JSON logs generated by Docker and inserting the token is done by a simple script written in Lua, which Heka supports out of the box. Using built in JSON decoder, all we had to do was:

local ok, payload = pcall(cjson.decode, read_message("Payload"))  
if not ok then  
    return -1
end

inject_message(token .. " " .. payload.time .. " " .. read_message("Hostname") .. " " .. payload.log)  

On top of that, there's a small shell script (sed wrapper actually) set as an entry point that replaces a token placeholder in Heka's configuration with the one provided by user.

And that's it!

Drawbacks

Decision to use progrium/busybox as a base image wasn't the smartest one due to the scary notice about it probably being deprecated in the near future.

Heka does produce a small amount of logs when starting and those do end up in logentries, but it's only a couple of lines so not too bad.

Feedback

We would love to hear your thoughts. Please leave a comment here, on the Docker registry or GitHub.

Cheers!

What we're working on at tray.io
Author image By Rich

After many, many months of blood, sweat and code we're fast approaching an initial release of the tray.io platform. We'll be publishing plenty of details over the coming weeks, to kick things off I'll provide a quick recap on where we're at.

What exactly is tray.io?

We're working on a way to help build and run software integrations, our mission is to make connecting software services a breeze. Many software services we use every day are distinctly separate and difficult to use - we intend to fix that.

This is especially true when trying to build integrations beyond a single push between two services.

Intergra-what?

Imagine instructing a robot to take care of a bunch of tasks for you, like:

  • when someone subscribes to your mailing list then look up their profile on LinkedIn if the person works for a Fortune 500 company then send a summary of the profile on Slack else send an auto-generated email welcoming the subscriber
  • poll your appstore RSS feed then loop through new reviews if a review is under 3 stars then create a ticket in Zendesk.
  • when someone mentions a Trello card in a git commit then update the status of the Trello card to complete.
  • loop through Mailchimp signups lookup email address on AngelList if they are an Angel investor add to Pipedrive (as we did)

Writing these scripts is a slow process, aside from actually producing and testing the code, finding a home to deploy these projects and continually monitor them takes a lot of time and expense.

How can tray.io help?

Well we've been working on a way these integrations can be built and deployed quickly whilst still being fully extendable by anyone that has the technical ability.

New integrations can be built via our editor directly in the browser. We provide a visual way to create workflows from a list of community driven connectors and support a range of SaaS services as well as logic components, storage services and authentication.

alt

Each project is built against our framework Subvert, meaning that at any point you have full access to the code and can extend the existing integration however you wish.

Finally having built and tested your integration (which you can run and debug in the browser) you're able to push the project into the tray.io cloud. Each project is run within a Docker container and can therefore also be run in your own data centre.

I'd like to thank the (1000s) of people we've spoken with along the way. Stay tuned for more posts on the turns we've made along this path. We look forward to showing you more soon!