How Keen IO Builds Internal Integrations & Tracks Customer Engagement

We welcome Joe Wegner from Keen IO to the tray blog, Keen IO provide a powerful Analytics service - driven from a core set of developer friendly API's.

I've always been a bit of a hacker. By that I mean that I've always found a lot of joy in taking a set of disparate tools, tying them together, and producing serious value. This has always been a pretty important part of my job - having been both a marketing engineer and a support engineer in the past, I've had to work with a lot of different tools. The tray folks harp on this quite a bit - when you can take the data between two separated systems and combine them, you gain a lot of power. Unfortunately, from years of tying services together, I've learned (at least) one thing: pretty much all integrations suck.

It's super unfortunate, but I also understand why. Most companies are super focused on building their product (hint: this is good!). Pretend your company builds an intelligent gifmoji generation service. Your engineers are probably going to be heads down building out neural networks or something, but you probably also have a sales team that keeps hearing from customers that they want to integrate your GIFs with Braintree (or Chargify or Zendesk or ...). This is where most of the terrible integrations come from - the sales team begs the engineers for 20 minutes of their time so that they can put up an Integrations Page and list all these cool things they integrate with. Except the integration is crappy. And it's really just a placeholder.

alt

When I started at Keen I joined the open source team, and helped build this really cool thing called Pushpop. The idea was that none of the integrations out there were good, and they all required custom code to make them useful. Pushpop let you harness the power of integrations, but also write some code to do things in a logical way. I leveraged Pushpop heavily every day... but it also had its downsides. Pushpop harnessed the power of (crappy) integrations + my own (crappy) code, which means I had the burden of maintaining both. When integrations changed, I had to fix Pushpop. When my code had a bug, I had to fix Pushpop. When Heroku pooped on my dynos, I had to fix Pushpop.

Enter tray. tray has given me the power to integrate with all of my tools, do it in a logical way, but not have to worry about the messy details of actually building & hosting a service. The great thing about tray is that, seriously, it is their job to write good integrations. Integrations aren't a distraction from the stuff their engineers actually want to work on; it's their whole product. Let me show you a quick Slack chat that pretty clearly explains what I mean:

alt

It was a holiday that day, but first thing the next morning I got a message from tray:

alt

Seriously. It's their job to do that kind of stuff.

There's the other half, too - the maintenance half. We used to have this ridiculously crazy Pushpop job that tracked our customer engagement. It's wild, but essentially what it did was queried an internal Keen collection to see which customers had done a read or write API operation that day. It then did some reformatting, batched them up, did another call out to an internal Keen API to get metadata, and then hit Hubspot's API so that we could do marketing campaigns based on engagement. The code was ridiculous, but let's not get into that. That painful part was that this thing broke all the time. As it turns out, storing a list of all the active Keen customers in the memory provided by a hobby Heroku dyno is really hard. Pretty much every other week I'd hear from our marketing peeps that the engagement stats in Hubspot were broken. I'd then have to dive into Pushpop and try to think of yet another clever idea of how to make the thing suck up less memory. It was exhausting. And distracting. And not valuable. Here's that job translated to tray:

alt

I actually think I could clean that up with some of the new utility connectors.

The best part? I haven't opened up the logs for that workflow since the day I built it. It just works - I haven't had to tweak it; I haven't had to fix anything. I hate to be cliche, but it just works. tray has abstracted out the whole backend of my integration - I can write the logic and completely forget about the rest.

And that's why Keen is a customer of tray. It's mostly just so I can sleep at night, without worrying about all of my tools breaking. I want to take this power - the power of tying my tools together - and just trust it to work. tray has built that trust for me. I mean, there's a lot of other cool things (like, my coworkers can write workflows without me giving them a few hours briefing on how to not break Pushpop), but the biggest thing for me is getting a ton of power while maintaining my sanity. It's... powerful.

That, and they gave me my own slack emoji

alt