Latest articles


CouchDB compaction - big impacts

CouchDB needs to have it’s databases compacted regularly. It’s quite easy to do but the ease of doing so may lead you into thinking that it’s not worthy of serious consideration. You need to be aware of a few things.

Here at the beeb we have many databases of very differing sizes, with very different “busy times” and to be honest we don’t really know nor care what data is in them. Some documents are very simple, some we know are serialised PHP objects, some are JSON base64 encodings of images (we do not allow attachments …

➟ Read more

Problems with replication maps

For a long set of reasons that I must sometime write about, I have a set of CouchDB databases which replicate with each other. Each database replicates with two others: one in the same datacentre, one in the other datacentre (we’re only running with two datacentres at the moment). One is to do with resiliance the other with response capability.

Anyway - having two or more replications is not a problem with CouchDB (if you’re careful) - even with our version 0.9x.

Lacking symmetry

However - we may not have been as careful as we’d have wished: look at …

➟ Read more

CouchDB 0.9x - 1st read from v large views serially

On a server, we run 4 different CouchDB nodes, each with 30 or so databases. We can therefore have over 100 databases - and if you’re reading from large views - or view over large databases - you will need to do so serially.

We have 4 as “normally” CouchDB is kind re it’s use of memory, and kind in terms of CPU usage (we have 8-core machines here). Yes - there is a headache re being disk-IO bound, but it’s not normally a problem.

But views are now needed (to identify conflicting docs) as we have many applications using our …

➟ Read more

Uncertain overnight behaviour

It's getting to be a wonder what's happening overnight. Here are some data points to ponder:

  • 27th: 6.5@3:22, 3.6@7:02: is this down to one glass of wine last night?
  • 26th: 19.6@3:35: was that me overdoing the recovery from a 4.7 the evening before?
  • 25th: 13.0@9:01: no clue
  • 24th: 14.0@8:37: no clue
  • 23rd: 14.9@4:51: recovery from a 2.3 at 20:54 the previous

It seems clear that the jury is out on whether or not the overnights are working: Is there …

➟ Read more

Accu-Chek Combo

The Accu-Chek Combo is an insulin pump that has the potential to make really important changes to how I manage my diabetes and therefore my live. It'll allow me to change how much insulin I get when and so better match my body's needs.

Let's do the downsides first:

  • It's a machine that I have to wear - always
  • There are a lot of "fiddley" components and parts
  • Configuring it to me is going to take a lot of time

There are going to be some upsides too of course:

  • I can up the background rates by adjusting the insulin rate …
➟ Read more

Colloquy demystifying JIRA references

The BBC’s Forge engineering team uses an IRC channel to hold meetings. It allows our team to not bother about exactly where everyone is - some folks work from home, people are (mostly) in the office, but can be in different parts of our buildings for all sorts of good reasons. So, we have JIRA instead of Agile “task cards” and we use IRC meetings instead of “standups”.

Link in any team, you’ll have some folks working on one aspect of the development, others on others. There’s an “idealisation” in many Agile software disciplines that suggests any developer …

➟ Read more

Top 25 Most Dangerous Programming Errors

The 2009 CWE/SANS Top 25 Most Dangerous Programming Errors is a list of the most significant programming errors that can lead to serious software vulnerabilities. They occur frequently, are often easy to find, and easy to exploit. They are dangerous because they will frequently allow attackers to completely take over the software, steal data, or prevent the software from working at all.

I’ve been going through these today for an internal security review - it’s a very good read with lots of helpful analogies. http://cwe.mitre.org/top25/


Running stunnel at startup

You might want your stunnels to be running all the time - and to start automatically when you log in. Here’s how:

  • get your stunnel working. You’ll need to fix your certs, choose the correct ports, and all that yourself.
  • write a script (/Users/you/stunnel.sh) that’s just like this:

    !/bin/sh

    /opt/local/bin/stunnel /opt/local/etc/stunnel/stunnel.conf

  • in a terminal window write this:

    sudo defaults write com.apple.loginwindow LoginHook /Users/you/stunnel.sh

Now you’re all set. Next time you login, the LoginHook will fire, running your “stunnel.sh” command …

➟ Read more