Easily Analyzing Your S3 Buckets

Amazon S3 is a storage solution used by pretty much everyone these days. Due to this there are naturally a bunch of tools for doing almost everything you can think of with S3. This post is about a tool I wrote to retrieve metadata about S3 buckets. If you don't…

Filtering Duplicate Log Entries via Rust and Runiq

If you work on any kind of backend system, odds are you're familiar with the pains of trawling through files looking for something specific only to discover huge amounts of entries which are basically all the same thing. Perhaps you have a really cool API that everyone calls all the…

Optimizing Docker Images for Rust Projects

This post will be the first of several addressing Docker image optimizations for different project types. It stems from my recent experiences with badly written Dockerfiles, which result in sitting around for 10 minutes every time you build, only to then need to upload images of over 1GB in size.…

Automatic Bang! Function Generation in Elixir

Elixir developers typically conform to the standard of returning Tuple structures from functions which can either succeed or fail in order to effectively signal their status to the caller. Whilst this encourages the caller to directly handle the various cases which can arise, sometimes you just know that everything was…

Achieving Multiple Routing Keys in RabbitMQ Exchanges

One of the key features inside RabbitMQ is the ability to easily route messages between exchanges and queues, using any number of provided routing patterns. Out of the box, RabbitMQ provides a variety of algorithms to use when routing (based on the AMQP). You can route using the following algorithms:…

Bulk Indexing with the Elasticsearch Java REST Client

With the introduction of Elasticsearch v5.0.0, Elastic began to recommend moving to their new REST client for server interactions rather than the transport client that was shipped previously. The intent was to make a HTTP client which would act with little to no interaction with the internal Elasticsearch…

A Year with Cachex in Production

A little over a year ago I began working on what would become my first real project in Elixir. Without going into too much detail, it was a re-imagination of a legacy system designed to handle many incoming data streams quickly and reliably. It seemed like a good fit for…

Introducing Vessel: A Bridge Between Elixir and Hadoop

Disclaimer: I should point out that I wrote this project for those who are already working with Elixir and have need of integrating their existing codebases with Hadoop. I'm going to try assume limited knowledge of Hadoop/Elixir within this post - but I apologise if it goes a little…

Stash: a user-friendly wrapper for ETS/DTS caching in Elixir

Erlang (and by extension, Elixir) comes with a sweet little memory storage library called ETS (Erlang Term Storage). This provides an awesome way to store a whole bunch of values in memory, without having to write your own implementation, or pull in large third-party libraries. The issue with ETS is…

Writing SipHash: my introduction to NIFs in Elixir

After recently discovering SipHash when searching for a solid algorithm to use in another project, I decided to take a shot at writing an implementation in Elixir as an exercise (having only started picking up Elixir in the last few weeks or so). The project is available on GitHub and…

Introducing Oryx; a convient scaffolding library for Node.js and Express

Say hello to Oryx. Oryx is a project I have been working on for a little over a month now, born from another project I'm currently working on (but haven't yet released). It's designed to be a fairly straightforward way to set up an Express app to use models and…

Java 8 Custom Collectors: Jackson ArrayNode

Streams are one of the niceties provided in JDK 8 and they provide extreme flexibility to the user. They make it extremely easy to transform Collections in some way, either serially or in parallel. One of the most awesome things about Streams is that they're extremely extensible. The example we're…

Using MongoDB with Elixir, Phoenix and Ecto

This is as much for my reference as anyone else's, so I hope everything is clear enough for newcomers. Update: Refer here for an example project following the method below. Due to having gotten a little bored with Node.js in recent times, and also a lack of engagement with…

Dot Notes: a simple module for parsing dot notation

Recently I have somehow found myself in need of using JavaScript dot notation to represent values a bit too often to bother constantly rewriting the code for it. Turns out, there's no real (and nice) way to do this in the way that I wanted. Perhaps I'm conforming to some…

Safely Creating Custom Handlebars Helpers For Ghost Blogs

I'm fairly new to Handlebars, the templating system that the blogging system Ghost uses, but my initial impressions are that it's a pretty sweet tool to use. However, it's somewhat limited when you're developing a Ghost blog, due to the fact that Ghost only implements certain definitions. Odds are that…

Neek - A faster way to remove duplicates in files

You may be aware of the *nix tool uniq, used to filter duplicate values from within files. Uniq is a simple way to remove duplicated lines within a file via a simple command. The downside to uniq is that it requires the input to be sorted, which can be a…

Asynchronous Test Loops With Mocha Using it-each

Nowadays people are using Mocha more and more for testing their JavaScript/NodeJS code. One major limitation of Mocha is that there is no real way to loop tests asynchronously and maintain the ability to keep track of what's happening (such as realising which iteration a test fails on). Due…

Redirecting An InputStream To An OutputStream In Java

After working with NodeJS, Streams have always been an annoyance for me in Java (especially when I was first learning), simply due to the fact that you had to wait for the input to buffer and be fed through the abstract read() method of the InputStream interface. Most examples around…

Migrating From Subversion To Git The Easy Way

Until recently, I was a big fan of using Subversion for most of my private projects. Now, however, I'd rather be using Git. This is all well and good for new projects, but how do I move the mature projects from my repository to Git (specifically GitHub)? Turns out, it's…

Sorting An Object Recursively By Key In Node.JS/JavaScript

Sorting an object is one of those little things which nobody ever considers useful to know (until it becomes so), but it's proven particularly useful to me in the last few months - not necessarily for the comparison of two objects at runtime (which, thankfully, there are much quicker things…

Handling Synchronous Asynchronous Loops In Javascript/Node.JS

One of the most lovable (yet frustrating) things about JavaScript and Node.js is the use of Asynchronous functions and the callback based system. Several times I've found myself wanting to halt for things to complete instead of sending off a bunch of processes all at once - an example…

How To Remap The Android Power Button Elsewhere

I figured this was probably something that people might find useful as when searching for a possible solution myself I came across a massive Google groups post with no good on how to remap the power button on an Android device! Basically, the other day my Nexus S Power button…