Marathon-Alerts: Alerting tool for Marathon Apps

Another month has passed by, and a bunch of open source contributions that has happened. I recently published on a blog post on Indix's blog titled - "Marathon-Alerts: A Tool for Keeping Your Apps on Marathon Under Radar". It is an official post for open sourcing the marathon-alerts tool to the outside world. Without further delay, here's the tl;dr version of the post.

Marathon-Alerts, is a tool to monitor apps running on marathon. Marathon while reaching it's 1.0.0 milestone (as of writing), it still doesn't have first class support for alerts for tasks running via it. If you're on all containers, then there are lot of tools out there in the market which can help you monitoring and alerting (Sysdig for example).

Unlike k8s, marathon doesn't limit to running only containers but arbitrary commands too. This is where specialised tools like marathon-alerts comes into play. Today we've notifier only for Slack, but it's quite easy to extend them to other sinks which can integrate to your existing infrastructure.

Above screenshot is an example alert that comes on our slack channels when one of our apps went down because of an error. 

This is one of my best open source contributions in recent times. Check out the project on https://github.com/ashwanthkumar/marathon-alerts, it's released under Apache 2 License. Fork away!

Introducing Matsya

I recently wrote a blog post on Indix's Engineering blog about Matsya. Do check it out at http://www.indix.com/blog/engineering/matsya-auto-scaling-availability-zone/. I'm going to summarise the blog post here in TLDR; version.

We have 100% of our Hadoop workloads running on AWS Spot infrastructure. One of the prominent issues we've have faced is "Spot Outages". In order to solve that, we went cross AZs. This in return resulted in huge data transfer costs especially for systems like Hadoop HDFS. We realised the only way to survive this madness is to be on a single AZ but intelligently move to another AZ when there's a Spot spike in the current one. That worked :) There were cases when we couldn't find any AZ within your bid budget. Typically our bid budget was 100% of OD price. During such times, it can fallback to OD instances until the surge ceases.

I wrote a tool to automate the above process, and that's Matsya. Matsya is the name of first incarnation of Lord Vishnu in this world. The main part of the story is he takes a form of a Fish (with a horn) and carries the Seven Sages (saptarishis) through safety on the Judgement day. The name was so apt since Matsya is able to carry the cluster from one Spot Market to another (read cheapest) during a Spot surge.

Bonus - Slides to my presentation I gave at Chennai Devops meetup on January 2016 can be found at http://j.mp/to-matsya.