On Splunk crashing every 18 minutes

Every 18 minutes, give or take ten seconds the Splunk heavy forwarder we run was spontaneously crashing. There had been no change in traffic or other conditions that we knew about on the ESX host the server runs on.

The operating system is Debian 7.2 x86_64, installed from the netinst ISO with a very minimally modified install, and the logs weren’t showing the OOM errors I’d been fighting with previously - 2Gb of memory for Splunk is never enough.

[Read More]

Using fail2ban to mitigate Apache POST flooding

Every so often a bot has a go at my poor little Linode, triggering a notification about CPU/memory/IO usage and I have to come up with a way of defending it. Today’s notification was about high CPU usage, as usual.

linode CPU graph

I toddled off to my handy (free) splunk instance and saw that this time it was an attack against my wordpress instance’s login page. I could see there were lots of 302 responses to POST requests, so it was an easy case for fail2ban to handle - something I’ve used before with great success.

[Read More]

Lego City Advent Calendar 2014 Day 13

Who’s to say if he’s naughty or nice? Maybe he’s here to share the wealth during the holiday season!

Because I got myself all confused, this was originally posted as day 12. Apologies!

A man with a bag of jewels

[Read More]

Lego Star Wars Advent Calendar 2014 Day 12

The X-34 Landspeeder‘s probably the first one we see in the series and was made famous to us by Luke Skywalker’s owning one 🙂

Edit: Somehow I originally posted this as day 11, because I got up early and rushed. Apologies!

An X-34 LandspeederAn X-34 Landspeeder

[Read More]