sequestering carbon, several books at a time V

This week's haul, from Amazon and from the Europa 2013 archaeology meeting:

I've nearly finished the Kontis.

Read More

Iain M. Banks, 1954-2013

Iain Menzies Banks

I remember during the 1987 Worldcon hearing the rumour that Iain M. Banks had climbed up the outside of the Brighton Metropole hotel the previous night, to get to his room.  He told the story himself at the 1990 Eastercon in Liverpool, during his Guest of Honour talk.  At the 1998 Eastercon, Iain M Banks and his alter ego Iain Banks took to the stage to discuss SF versus mainstream writing.  He told wonderful stories, both in person and in print.

In early April this year, he announced he had terminal cancer, and was unlikely to live beyond a year.  Just two months later, he has died.

Dammit.

Read More

rainfall statistics in d3

My explorations of the d3 data visualisation library for JavaScript continue apace.  I have now added "box-and-whisker" plots to the 2012 rainfall, helping demonstrate the anomalously wet April and July, and dry August, last year.  This required calculating the various means, medians and quartiles, which wasn't particularly difficult, and has given me a feel for working with arrays.

Box and whisker plots show minimum, lower quartile, median, upper quartile, and maximum.
The mean value is overlayed as a blob.

This latest version replicates the previous chart displaying this information, which I did in Excel. (The bars are slightly different, because here I have included the 2012 data in the statistics, whereas I excluded it in the previous version.)

I think this d3 version looks much cleaner.  It certainly lets me fiddle with things at the programmatic level I prefer!

Read More

interaction with d3

My explorations of the d3 data visualisation library for JavaScript continue.  Last weekend I dipped my toe in the water, and drew some simple rainfall charts.  The code was clunky, and the data was hardwired, but the charts were pretty enough to convince me to continue.

And so this weekend I've explored some more.  I've modified the code to read in the data from a csv file, and then refactored like mad to make it more elegant.  The result of half a day's work was a web page indistinguishable (on the surface) from before: but now the charts, and the headings, are generated from the csv file data, so I can add more data each month, and the page will update without any code having to change.  So, a success.

Then I decided to add a little bit of interaction: changing the colour of a bar or spot on mouse-over, and showing the actual data value.  The d3 library is powerful enough that it was very easy to do.

in May 2007 we had 76mm of rain

in July 2009 we had 63mm of rain

So that's some tasks completed off my to-do list.  I've still got a long way to go.  But, as when I was learning Python, I'm finding StackOverflow an invaluable resource.

The actual web page is here, but it may change over time as I add more functionality.

Read More

sequestering carbon, several books at a time IV

The haul for the last fortnight.

I've already read the Saintcrow, and will be reviewing it imminently.

Read More

my first d3

Way back when I was an undergraduate, I stumbled across a book called Curve Tracing, by Percival Frost.  The college library had a first edition, published in 1872.  Several years later, I came across a newer edition in a bookshop, and grabbed at it.

This beautiful little book has tons of curves defined by equations, and shown as graphs.

Plate IV from Curve Tracing

This was all originally done in the days before computers.  It's been a goal of mine to reimplement some of this work, in an interactive form, because many of these curves have parameters that affect their detailed shape.  So, I've been looking for a suitable tool.

Also, I'm interested in data visualisation (and have long been a disciple of Tufte).  I've tried to follow the "no chartjunk" ethos in my own work.  For example, to plot our rainfall statistics, I had to struggle with Excel charts to remove most of the garish "ink" provided by default. But it's still not perfect.  Additionally, I have some specific work I want to do with modified parallel coordinates, for which there is no existing library.  So, I've been looking for a suitable tool.

Last week a colleague of mine mentioned d3, a JavaScript library for Data-Driven Documents. I browsed the gallery for a while, and fell in love.  I spent yesterday playing around with evaluating it, on simple charts, to see what it could do.  This meant reading Scott Murray's d3 tutorial, and implementing a few simple charts to show the rainfall data. (Oh, and learning my first JavaScript.)

First off, I tried a simple bar chart, starting from Murray's tutorial example (when learning a new language, I usually find it easier to modify existing code than start from an empty file). After fiddling around to get the ordinal x-axis working, I got:

d3 chart of 2012 rainfall in mm

I think that looks suitably chartjunk-free and minimal.  It also has the nice feature of combining the actual numerical values into the bars, giving what Tufte calls both a macro-reading (the bars: gosh, July was wet!) and a micro-reading (the numbers: July had 113mm of rain) in one chart.

What's nice about d3 is the way the axes scale automatically.  Exactly the same code produced these charts (the only difference is the July data value: 13, 113, 233):

automatic axis scaling with changing data values

But, of course, we have several years of rainfall data.  A little more hacking coding gave me a grouped bar chart:

rainfall, June 2005 -- May 2013

Now it's possible to see how wet April 2012 was, compared to normal Aprils, even if it wasn't as wet as that July!  My implementation of this is a little kludgy, with too much hard-wiring, since my goal was to evaluate the capability, not (yet) learn the entire language; my next task is to code it more elegantly.

I've always found grouped bar charts rather cluttered, and so I wondered if there was a better way to show the data.  Rather than use some sort of surface plot, I decide to try a projection where the size of a spot is related to the amount of rainfall.  With a surprisingly small change to the code, the grouped bar chart metamorphosed into a "blob" chart:

(left): blob area proportional to monthly rainfall; (right) blob radius proportional to monthly rainfall

This enables comparisons in both dimensions (years, or months), depending on whether you view rows or columns.

So, based on a day's work, I'm very impressed with d3.  However, there are a lot more d3 facilities I need to get up to speed with before I can start my reFrost project in earnest:

• csv data import -- currently the data is hard-wired into the scripts (ugh)
• data manipulation -- to calculate medians and quartiles for box and whisker plots
• lines -- to draw graphs rather than charts
• maths -- to calculate the functions: and, presumably I'll need a lot more JavaScript
• interaction -- so parameter values can be chosen by the user
• transitions -- so the graphs will smoothly change as parameters are varied
• more -- stuff I don't know about yet, but will need

So, a way to go, but I think I might have identified the tools I need.

Read More

all the Doctors

The Post Office has recently issued a set of Doctor Who stamps, in commemoration of the programme's 50th anniversary.

Post Office Dr Who Presentation Pack

In addition to the stamps themselves, they also issued the pictures as a set of postcards.  We got a set and had them framed.

Our Dalek seems nervous at the presence of 11 Doctors.

Read More