Easily Analyzing Your S3 Buckets

Amazon S3 is a storage solution used by pretty much everyone these days. Due to this there are naturally a bunch of tools for doing almost everything you can think of with S3. This post is about a tool I wrote to retrieve metadata about S3 buckets. If you don't…

Filtering Duplicate Log Entries via Rust and Runiq

If you work on any kind of backend system, odds are you're familiar with the pains of trawling through files looking for something specific only to discover huge amounts of entries which are basically all the same thing. Perhaps you have a really cool API that everyone calls all the…

Automatic Bang! Function Generation in Elixir

Elixir developers typically conform to the standard of returning Tuple structures from functions which can either succeed or fail in order to effectively signal their status to the caller. Whilst this encourages the caller to directly handle the various cases which can arise, sometimes you just know that everything was…

Bulk Indexing with the Elasticsearch Java REST Client

With the introduction of Elasticsearch v5.0.0, Elastic began to recommend moving to their new REST client for server interactions rather than the transport client that was shipped previously. The intent was to make a HTTP client which would act with little to no interaction with the internal Elasticsearch…

A Year with Cachex in Production

A little over a year ago I began working on what would become my first real project in Elixir. Without going into too much detail, it was a re-imagination of a legacy system designed to handle many incoming data streams quickly and reliably. It seemed like a good fit for…

Introducing Vessel: A Bridge Between Elixir and Hadoop

Disclaimer: I should point out that I wrote this project for those who are already working with Elixir and have need of integrating their existing codebases with Hadoop. I'm going to try assume limited knowledge of Hadoop/Elixir within this post - but I apologise if it goes a little…