iboto
Amazon EC2 shell for managing multiple accounts and regions easily
[Read More]
s3grep
Parallelized grep for Amazon S3
[Read More]
Sous-vide Teriyaki Chicken
This produces the most flavoursome, juiciest and succulent chicken. My mouth is still watering.
The recipe doesn’t involve much ‘working’ time, but it takes about 3 hours total start to finish. The brining process increases the water and salt content of the chicken through osmosis and denatures the proteins, trapping more water during the cooking process.
Brine 540g Chicken Breast (3 breasts) 500ml water mixed 40g salt (8% brine - dissolve in batches of hot water) 20g sugar Dissolve the salt in the water in batches of hot water.
[Read More]
Sous-vide Arduino
The project Sous-vide on a budget. Build a PID temperature controller for mains (230V) devices. Then any suitable device (slow cooker, rice cooker, hotplate, etc.) can then be plugged in and turned into a highly accurate temperature controlled cooker.
Data logging I also wanted remote data logging, so I could see what it was doing. My controller emits temperature readings emulating an Oregon Scientific THGR810 (so my existing home automation system can pick this up and graph it).
[Read More]
Introducing elasticsearch*
Just opensourced a starcluster plugin for provisioning elasticsearch clusters automatically in the cloud.
What is Starcluster? Starcluster is great for spinning up clusters of ec2 nodes quickly for some analytics / data chomping.
If you’re not familiar take a look at this very cool screencast:
Starcluster and Elasticsearch = best friends Elasticsearch is a great fit - it runs smoothly on ec2 and for ease of searching big data is second to none.
[Read More]
Migrating from GoDaddy to Amazon Route53
Due to the recent outage on GoDaddy a lot of people are reconsidering their DNS options. Amazon Route53 is a great option - cheap, flexible and well proven.
To migrate you first need to export a zone file for the domain from GoDaddy.
It’s been highlighted the zone files are slightly broken in CNAME records, so you may need run this fix over them:
$ perl -pe 's/(CNAME .+)(?!.)$/$1./i' broken.txt > fixed.
[Read More]
A duplicate classifier using elasticsearch
Frequently with search based, big data projects the problem of content duplication is an obstacle to having a clean data source. Here’s an approach to improving the data quality by training a classifier to spot duplicates.
The Problem The data set has about 470,000 non-unique hotel descriptions (e.g. name, metadata, images) provided from 11 bed bank services. These are received in various different feed formats, and in this data is hotel name, resort name, address, country, description and image data, among other data.
[Read More]
iboto screencast
Made a screencast (my first!) of iboto to give a demo of the Amazon EC2 multi-account coolness:
Incidentally, making screencasts on Linux was a bit of a slog until I found the right tools and workarounds, so that might make the subject of a blog post of its own.
A tweeting, speaking, IMing house
My house tweets. It speaks. It sends me jabber messages. This seems to be a constant source of amusement for guests. This ‘house of the future’ was built with a small amount of DIY electronics and some software hacking, on a shoestring budget.
Over time I’ve added more systems to the house and it is interesting experimenting with novel ways to surface this into a more interactive house.
The first system online was a weather station (an Oregon Scientific WMR100).
[Read More]
Route 53 latency based routing
Amazon have launched a neat new Route 53 feature: latency-based routing. The idea behind this is when someone hits www.yoursite.com this resolves to the closest server to them, cutting latency.
This DNS cleverness has been used by the big boys for some time, but not been available to us mortals without shelling out big bucks to someone like neustar/ultradns (shudder).
The ’location’ of your server is determined by multiple DNS records for a given lookup, each with an EC2 region attached to them (us-east-1, eu-west-1, etc.
[Read More]