chartjunk lives!

After an excellent talk on visualising data yesterday, a colleague and I were chatting about Tufte, and bemoaning the chartjunk infesting many publications nowadays.  Being academics, we of course discussed giving some lectures on data visualisation.

Later, she sent me the two examples below, in order that I could share her exasperation. (Exasperation is not the word I would have chosen, though.)

I merely cringed at the first, but literally lol’d at the second, in stunned disbelief.

WTF!?  (And what do you mean “normalized”? to 360°?)

Really?  What were you thinking?

Read More

Typography in ten minutes

Yesterday I came across Butterick’s Practical Typography. It promises to make you “a bet­ter ty­pog­ra­ph­er than 95% of pro­fes­sion­al writ­ers and 70% of pro­fes­sion­al de­sign­ers” in ten minutes, by reading 5 rules (and, presumably, then following them). I instead read the whole thing – it’s fascinating (if you are fascinated by typography).

At one point, when advocating using wider margins, and making better use of white space, he sets an exercise, of making 2 documents with the same text.

Document A is the usual default style, and has the following format: “page mar­gins of one inch per side, font is Times New Roman, point size is 12, line spacing is “Double” [...], first-line indent is half an inch, and no space be­tween para­graphs.”

Document B follows Butterick’s strictures, and has the following format: “page mar­gins of two inch­es per side, font is still Times New Roman, point size is 11, line spac­ing is ex­act­ly 15 points, first-line in­dent is still half an inch, and still no space between para­graphs.”

He claims that B looks more like a professionally typeset book, is more comfortable to read, and contains more words per page. And he's right. Here are A and B (using Lorem Ipsum, so neither is particularly comfortable to read!).

(left) document A, default margins, double spaced; (right) document B, wider margins, better spaced, more text

Then today I had to write a 250 word “vision” document. I heeded what I had learnt. I initally wrote the text in the Word default layout. Then I tweaked it, by changing the font, and increasing the margins and linespacing. I made the margins as wide as I could, and still get all the text on one page. The result:

(left) 250 words in Word default format; (right) the same words, after some minor formatting tweaks

I’m sold!

Read More

boxes, whiskers, and violins

My experiments with the charting language d3 continue.  I'm looking at different ways to display book review statistics.  For example, I'm interested in how long books sit on the shelf between being bought, and being read.  It can be a long time; but how long?

A scatter plot shows all the data:

Here I have colour-coded book types as purple for non-fiction, green for science fiction, and orange for other fiction.  The existence of the line is clear: I can't read books before I acquire them.  (The green dot below the line in 1998 is an anomaly: either the acquisition date or the review date is wrong.  I have investigated, but can't determine which, so have left it as it is.)

The scatter plot shows all the data, and provides a visual clue that maybe the time between acquisition and reading isn't too long.  (Of course, this only shows the books I've acquired and read, not the ones I've acquired and are still languishing on the unread shelves!)

In order to better visualise the time delay, I plotted the data as box-and-whisker plots (showing median, quartiles, outliers, and here also the mean), overlayed with violin plots, (showing a more detailed estimate of the underlying distribution):

This was all relatively easy to do using the d3 chart library, and Jason Davies' science.js library for the kernel density estimator needed for the violin plot.

So from this I can see that I tend to read fiction almost as soon as I buy it, SF quite soon after, and that non-fiction sticks around on the shelves for longer.  This probably indicates that I buy non-fiction partly as an investment (it's my pension fund!).

Maybe I should spend more time reading them, and less time learning new languages so I can analyse how fast I'm reading them?

Read More

book review statistics in d3

My experiments with the charting language d3 continue.  Now that I can happily import csv files, I decided to have a go at visualising some data about the various book reviews on my website.

First, I charted the cumulative number of reviews, of the fiction, SF, and non-fiction books:

I was surprised at the total number of reviews, as I 'd never seen this all in one place before. And I was also surprised at the relatively steady rate of reading demonstrated.

I wrote a little python script to extract the figures from the database into a csv file.  I discovered that a few weeks of writing JavaScript seems to have over-written a large chunk of python syntax in my brain...

I also had a go at viewing the data according to date of publication, of acquisition, and of review.  I'm still playing around with those views, working out what they can show.  More experimentation is needed.

Read More

rainfall statistics in d3

My explorations of the d3 data visualisation library for JavaScript continue apace.  I have now added "box-and-whisker" plots to the 2012 rainfall, helping demonstrate the anomalously wet April and July, and dry August, last year.  This required calculating the various means, medians and quartiles, which wasn't particularly difficult, and has given me a feel for working with arrays.

Box and whisker plots show minimum, lower quartile, median, upper quartile, and maximum.
The mean value is overlayed as a blob.

This latest version replicates the previous chart displaying this information, which I did in Excel. (The bars are slightly different, because here I have included the 2012 data in the statistics, whereas I excluded it in the previous version.)

I think this d3 version looks much cleaner.  It certainly lets me fiddle with things at the programmatic level I prefer!

Read More

interaction with d3

My explorations of the d3 data visualisation library for JavaScript continue.  Last weekend I dipped my toe in the water, and drew some simple rainfall charts.  The code was clunky, and the data was hardwired, but the charts were pretty enough to convince me to continue.

And so this weekend I've explored some more.  I've modified the code to read in the data from a csv file, and then refactored like mad to make it more elegant.  The result of half a day's work was a web page indistinguishable (on the surface) from before: but now the charts, and the headings, are generated from the csv file data, so I can add more data each month, and the page will update without any code having to change.  So, a success.

Then I decided to add a little bit of interaction: changing the colour of a bar or spot on mouse-over, and showing the actual data value.  The d3 library is powerful enough that it was very easy to do.

in May 2007 we had 76mm of rain

in July 2009 we had 63mm of rain

So that's some tasks completed off my to-do list.  I've still got a long way to go.  But, as when I was learning Python, I'm finding StackOverflow an invaluable resource.

The actual web page is here, but it may change over time as I add more functionality.

Read More

my first d3

Way back when I was an undergraduate, I stumbled across a book called Curve Tracing, by Percival Frost.  The college library had a first edition, published in 1872.  Several years later, I came across a newer edition in a bookshop, and grabbed at it.

This beautiful little book has tons of curves defined by equations, and shown as graphs.

Plate IV from Curve Tracing

This was all originally done in the days before computers.  It's been a goal of mine to reimplement some of this work, in an interactive form, because many of these curves have parameters that affect their detailed shape.  So, I've been looking for a suitable tool.

Also, I'm interested in data visualisation (and have long been a disciple of Tufte).  I've tried to follow the "no chartjunk" ethos in my own work.  For example, to plot our rainfall statistics, I had to struggle with Excel charts to remove most of the garish "ink" provided by default. But it's still not perfect.  Additionally, I have some specific work I want to do with modified parallel coordinates, for which there is no existing library.  So, I've been looking for a suitable tool.

Last week a colleague of mine mentioned d3, a JavaScript library for Data-Driven Documents. I browsed the gallery for a while, and fell in love.  I spent yesterday playing around with evaluating it, on simple charts, to see what it could do.  This meant reading Scott Murray's d3 tutorial, and implementing a few simple charts to show the rainfall data. (Oh, and learning my first JavaScript.)

First off, I tried a simple bar chart, starting from Murray's tutorial example (when learning a new language, I usually find it easier to modify existing code than start from an empty file). After fiddling around to get the ordinal x-axis working, I got:

d3 chart of 2012 rainfall in mm

I think that looks suitably chartjunk-free and minimal.  It also has the nice feature of combining the actual numerical values into the bars, giving what Tufte calls both a macro-reading (the bars: gosh, July was wet!) and a micro-reading (the numbers: July had 113mm of rain) in one chart.

What's nice about d3 is the way the axes scale automatically.  Exactly the same code produced these charts (the only difference is the July data value: 13, 113, 233):

automatic axis scaling with changing data values

But, of course, we have several years of rainfall data.  A little more hacking coding gave me a grouped bar chart:

rainfall, June 2005 -- May 2013

Now it's possible to see how wet April 2012 was, compared to normal Aprils, even if it wasn't as wet as that July!  My implementation of this is a little kludgy, with too much hard-wiring, since my goal was to evaluate the capability, not (yet) learn the entire language; my next task is to code it more elegantly.

I've always found grouped bar charts rather cluttered, and so I wondered if there was a better way to show the data.  Rather than use some sort of surface plot, I decide to try a projection where the size of a spot is related to the amount of rainfall.  With a surprisingly small change to the code, the grouped bar chart metamorphosed into a "blob" chart:

(left): blob area proportional to monthly rainfall; (right) blob radius proportional to monthly rainfall

This enables comparisons in both dimensions (years, or months), depending on whether you view rows or columns.

So, based on a day's work, I'm very impressed with d3.  However, there are a lot more d3 facilities I need to get up to speed with before I can start my reFrost project in earnest:

• csv data import -- currently the data is hard-wired into the scripts (ugh)
• data manipulation -- to calculate medians and quartiles for box and whisker plots
• lines -- to draw graphs rather than charts
• maths -- to calculate the functions: and, presumably I'll need a lot more JavaScript
• interaction -- so parameter values can be chosen by the user
• transitions -- so the graphs will smoothly change as parameters are varied
• more -- stuff I don't know about yet, but will need

So, a way to go, but I think I might have identified the tools I need.

Read More

food for thought

More than half of the world's population lives inside this circle despite it "being mostly water and including the most sparsely populated country on earth (Mongolia)"

Read More

Mathematical Models

Way back when I was at school, I came across a fascinating little book called Mathematical Models, by Cundy and Rollett.  It had instructions on how to build various mathematical objects, such as stellated polyhedra.  I liked the book so much I actually bought my own copy, new, for £2.95, which was a lot of money back then!

I made a few of the simpler models, but never got much further than the dodecahedron.  I certainly never got as far as making any of the fiddly stellated ones.

So today, when I came across a posting in Google+ about Anselm Levskaya's website polyHédronisme, I was taken right back to those days.  Playing with this interactive web-based systems is much easier than fiddling with card, glue, and scissors, though.  Type in a few commands, and a zoomable, rotatable polyhedron appears!

I've spent my afternoon playing around on this site, and reading up on Conway polyhedron notation that is used to define shapes, and now I can say I have at last "made" some of these polyhedra.

The small stellated dodecahedron is made by raising a pentagonal-based pyramid on every face of a regular dodecahedron. If the pentagons making up the dodecahedron have side length \(1\), then the height of each pyramid should be* \[ \frac{\sqrt{4\sqrt{5}-1}}{2} \approx 1.41\] The Conway notation command in polyHédronisme that achieves this is \(k(5,1.41)D\), which means: start with a dodecahedron \(D\), then raise a pyramid of height \(1.41\) on each \(5\)-sided face.

The great dodecahedron is made by making a pyramidal dimple in every face of a regular icosahedron. If the triangles making up the icosahedron have side length \(1\), then the depth of each pyramid should be* \[ \sqrt{\frac{1}{2}- \frac{\sqrt{5}}{6}} \approx 0.36\] The Conway notation command in polyHédronisme that achieves this is \(k(3,-0.36)I\), which means: start with an icosahedron \(I\), then indent a pyramid of height \(0.36\) on each \(3\)-sided face.

The great stellated dodecahedron is made by raising a pyramid on every face of a regular icosahedron. If the triangles making up the icosahedron have side length \(1\), then the height of each pyramid should be* \[ \sqrt{\frac{7+3\sqrt{5}}{6}} \approx1.51\] The Conway notation command in polyHédronisme that achieves this is \(k(3,1.51)I\), which means: start with an icosahedron \(I\), then raise a pyramid of height \(1.51\) on each \(3\)-sided face.

So much for standard polyhedra.  But polyHédronisme doesn't stop there.  I had great fun playing about with the notation language, producing weird and wonderful forms:

(i)  \(k(20,1)bk(3,2)I\)    (ii)  \(k(12,1)k(10,2)bk(5,1)D\)    (iii)  \(k(20,-0.3)k(6,0.3)bk(3,-0.3)I\)
(iv)  \(k(24,-0.5)k(6,0.2)k(20,-1)bk(3,-0.1)k(5,1)D\)

The results of play can only really be appreciated on the site itself, rotating the polyhedra, and getting a real feel for their shapes, with all their dips and bumps.  A marvellous site.

---

* The book Polyhedron Models has helpful stellation diagrams that allow these heights to be calculated, with a little trigonometry.

Read More

funfair mirror trees

One of the trees in our garden has died.  It died last summer in the drought, but we gave it a year to prove to us it really was dead.  It is.  So we need to replace it.

I was wandering around the web, looking for trees, when I saw a picture of the type we wanted.  The page also included a helpful impression of its mature size.

6m high, 8m spread: too slim

And a very impressionistic impression it is too.  This graphic of a slim-looking tree is labelled as being 6m high, with an 8m spread.  It's actually broader than it is high! 

So I had a look at a few other tree graphics on the site.

10m high, 10m spread: too slim

20m high, 10m spread: too wide

It's the exact same graphic every time, with not a single one of them using the same scale for the height and spread!  Only the "human figure for scale" and the labels change.  Why go to the effort of including a graphic to show the mature tree size, then not bother to do it right?

The pictures should look something more like this:

height/spread ratios just right

Now it's clear we shouldn't plant our new tree too close to the fence.

Read More

rough fonts

I was reading the Boing Boing post about fonts that look like graphs. More interesting was the link to "postscript hacks to render slightly differently every time". I followed up, and discovered the Beowolf font family, with parameterised roughness, caused by adding some randomness to the PostScript rendering process:

Beowolf family, from MyFonts 

What's great about this site is the number of different kinds of text you can choose to display the various fonts.  I find the mid-rough R22 style most pleasing to my eye. 

A 2008 post on the Typographica blog gives some interesting historical background to this family.  (There's a blog all about fonts!?!  I may never get anything useful done again...)

This reminds me of "roughness", which is one of Christopher Alexander's 15 "generative properties", from his Nature of Order work, used to produce architectural structures that exhibit "wholeness".  We did some work a while back trying to formalise a few of the other properties, through BlobWorld, and were interested to discover an optical illusion that led to an appearance of roughness emerging.  Rough brickwork, or rough wall alignment, okay.  But rough fonts?  Neat.

Read More

FatFonts

Edward Tufte advocates using minimum ink to maximum effect in displaying quantitative information, and designing graphic displays to have both macro (distant) and micro (close-up) readings.

One technique he describes that fits his philosophy is the "stem and leaf" plot. Let’s say you have two sets of 50 exam marks as percentages, and you want to examine the data.  One way to see the spread is to plot the marks, maybe as a histogram. That shows the distribution, but loses the information of the specific scores.  The stem and leaf plot provides the best of both worlds: a graphical overview and close up values.  The "stem" shows the tens values, and the "leaves" show the remaining digit values.

stem and leaf plots, of 50 values generated randomly from a normal distribution, with:

 (a) mean 50, std dev 20; (b) mean 70, std dev 10

This plot looks like a histogram from a distance, but closer inspection shows that the bars are made from digits with meaning.  So we can easily see that in the first case, the marks range from 9 to 96, and in the second case, the marks range from 54 to 94. 

Stem and leaf plots can also be used to good effect for bus and train timetables. In this case the "stem" shows the hours, and the "leaves" show the minutes when the transport is due.  For example, Nottingham tram timetables use this approach.

You can immediately see, with a distant macro-reading, on weekdays the most trams are at rush hour, on Saturdays they are more evenly spread , and they are rather more infrequent out of hours and on Sundays.  Then you can look closer for a micro-reading, and see the actual tram times.  

These plots have been around for a long time.  The latest New Scientist (2 May 2012) reports an analogous approach for greyscale plots: "FatFonts", conceived and developed by Miguel Nacenta, Uta Hinrichs, and Sheelagh Carpendale.

the "Rotunda" FatFont

With a FatFont, the amount of ink used to print a digit is proportional to the value of the digit: so a 9 uses nine times the ink as a 1, for example.  Instead of plotting a greyscale pixel on a graphic, plot the actual value using the FatFont.  For more numerical resolution, simply use a second digit one-tenth the area of the first (and a third digit, one tenth smaller again, if needed, and so on, until resolution fails). From a distance, the macro-reading is a greyscale image; closer up, the micro-reading shows the digits, showing the numerical values. 

Here's an image of the terrain around Etna on Sicily, from the FatFonts galley page: 

A FatFont plot is to a greyscale image what a stem and leaf plot is to a histogram.

Typewriter art, ACSII art, and photo mosaics achieve a given picture (macro-reading) by using very many small characters or picture "tiles" chosen to have the right density or colour.  But with those techniques, the micro-readings are not usually of interest.  FatFonts, on the other hand, give a useful graphical tool.

It’s a really neat idea.

Read More