Friday, 2 May 2014

Amazon Top 100 Kindle Books – Analysis of Average Ages of Writers in the Top 100


Since this blog is fairly data-rich, I will repeat the summary remarks here at the beginning, for those of you who are in a hurry.  But it the subject of what age is the "best age" for writers interests you, you should read the whole blog.
==================================================================
Summing up, when it comes to the average age of writers in the Amazon Top 100, we can say:

· Their average age was 52.3 years, though there was a wide spread of ages in the data, from a low of 33 to a high of 86. So, writing success is possible at any age, though it appears to be an activity where success builds over time.

· Female writers were significantly younger than male writers, on average, about 51 years of ages versus 56 years of age.

· There was no strong tendency as to writers being older or younger by sales rank decile or quartile - average ages were about the same throughout the top 100 when broken out in these groupings.

· Indies were significantly younger than Trads, at about 45 years old versus 55 years old.

· Younger writers tended to sell at lower prince points than older writers.

· The Romance category had the youngest writers (46), on average. The Thriller category had the oldest writers (62), on average. The other genres were somewhere in the middle.

================================================================
Now, to the blog in its entirety:

In a series of recent blogs, I have performed some analyses on Amazon’s Top 100 Kindle eBooks of 2013.  These blogs have looked at that data in various ways – specifically, the demographic and publishing characteristics of writers in the top 100, as well as their average Amazon ratings and imputed numbers of book sales and dollar revenues.  Among the factors examined were:

1.       Gender (sex) of writer

2.       Relative position within the top 100 (by decile and quartile)

3.       Publisher

4.       Indie vs Trad status

5.       Price Range

6.       Genre
Many people, especially writers and publishers, are bound to be curious about just what is the age at which writers’ have their greatest success.  So to this very interesting list, we will now add age of writer (as of 2013, calculated as 2013 minus Birth Year).

First, a quick note on data sources and reliability.  About 62% of the writers in the Top 100 had Wiki entries that included explicit birth year information, about 4% had newspaper articles or other sources that indicated age reliably, and the remaining 34% had to be estimated from author bio information that didn’t mention age – basically that meant estimating from photographs or vague cultural references (e.g. read Sweet Valley High books as a teen).  Though the photos for this latter group seemed fairly up-to-date, one can never be sure, so it will be noted which groups had to have many of their ages estimated when we look at the data.  That will give the reader a chance to judge for himself or herself, whether that fact seems likely to have influenced the conclusions significantly.  Now to the data.

Overall Age Distribution
First off, the table below shows the overall age distribution of writers in the Amazon Top 100 for the year 2013.  I should note that if a particular writer had more than one book in the top 100, he or she will be counted more than once in the data below.  That seems fair to me, since it effectively gives more successful writers more weight in the data, and it makes the math easy (the counts also happen to be percentages of the total, since there are 100 titles in the dataset).
 


Writer Age Range


34 or Less

35-44

45-54

55 - 64

65 Plus

Total

Writer Count

3

21

36

18

22

100


As we can see, the majority of writers were in the 45-54 age range, followed by the 65 plus range.  Few writers were under 35.  This yielded an average age of 52.3, with a standard deviation of 11.3 years.  The latter figure gives a numeric value for the “spread” of the data.  Roughly speaking, you can think of this as meaning that two thirds of the writers were within 11 years of the average age of  52 (or two-thirds of the writers were between 41 and 63).
It is important to note that there was a substantial difference between the writers whose age was given on Wiki (56) versus those for whom I had to impute age (46).  So the writers whose age had to be imputed from photos were significantly younger than the others – it turns out that these were mostly female romance writers.  Given the possible implications of age stereotypes on romance writers, they may well have been a bit reluctant to be pinned down on the subject.  Nonetheless, I think the estimate is fairly accurate.

The graph above shows this effect clearly.  If we look only at the solid blue bars (the writers for whom we had wiki data on ages), there is a definite step pattern to the data – as the age range goes up, the number of books in the Top 100 goes up.  But when we add in the red speckled bars (the writers for whom we had to impute ages from photos on the web), the pattern changes dramatically, revealing a (more or less) normally distributed shape, with the peak near the middle age range of 45-54, then falling off on either side.  This is the pattern that we usually see in datasets of this type – that’s part of the reason that it’s called the “normal distribution” J.  So, that lends some confidence to the notion that our age estimates from the photos are probably reasonably accurate.

We will make a few more comments on the possible implications of this graph later in the blog, after detailing more of the key results of the analysis.

Age Distribution by gender
Females tended to be considerably younger than males, with the most common age for female writers being 45-55, while the most common age for males was 65 plus.  The average age for females was 50.8, whereas for males it was 55.9, a not inconsiderable difference of about 5 years. 
Writer Count
Writer Age Range
Gender
34 or Less
35-44
45-54
55 - 64
65 Plus
Total
Female
3
16
28
10
13
70
Male
5
8
8
9
30
Grand Total
3
21
36
18
22
100
In the case of both males and females, the average age of writers whose age had to be imputed was younger (about 45) than those for whom age was easily available on wiki (about 56).  However only about 3% of males required age imputation, while 53% of females did, so there was quite a strong gender effect in that respect.
Age Distribution by Sales Decile and Quartile
First, a refresher on what these terms mean.  A decile is one tenth of the dataset, while a quartile is one quarter of the dataset.  So, Decile 1 is the group whose sales ranked first to tenth, Decile 2 is the group in 11th to 20th, and so on.  Similarly, Quartile 1 is the group whose sales ranked first to twenty-fifth, and so on.  Note that because there are 100 books in the dataset, the math is easy.
All that being said, there was very little difference in writer ages by quartile of sales rank (Q1=52.5, Q2=52.4, Q3=52.3, Q4=52.0).  The same was true for deciles – almost all had average writer ages between 50 and 55.
It should be noted that writers whose age had to be imputed were somewhat more likely to be found in the second and third sales quartiles than the first and fourth.   As for deciles, the most interesting finding was that no writers in the first sales decile had to have their age imputed.
Age Distribution by Publisher
The table below shows the average age of writers, by publisher.  There appears to be no particularly strong trend, other than the observation that Indies are younger than the others, on average.  More on that in the next section.
Publisher2
Avg Age
Doubleday
58.0
Hachette
54.7
Harlequin
61.5
Harper Collins
46.8
Indie
45.2
MacMillan
64.0
Penguin
57.2
Random House
56.5
Simon & Schuster
49.0
William Morrow
49.0
 
Age Distribution by Indie vs Trad Status
A quick definition is in order.  Indie books are those that were self-published or published by very small publishers, while Trad books were published by one of the (usually larger) publishing corporations, such as Penguin or Random House (see above for a list).
This comparison is always interesting.  As we see in the table below, Indies tend to be substantially younger than Trads.  Indeed, the majority of Indies are under 55 (20 were under 55 versus only 4 that were older than that), while there are almost as many Trads 55 and over (36), as under 55 (40).  This gives an average age for Indies of 45.2 versus an average age for Trads of 54.6, a difference of just shy of ten years, quite a pronounced contrast.

Writer Count

Writer Age Range

Indie vs Trad

34 or Less

35-44

45-54

55 - 64

65 Plus

Total

Indie


10

10

2

2

24

Trad

3

11

26

16

20

76

Grand Total

3

21

36

18

22

100
 It is interesting to note that there was little difference between Indies and Trads in terms of the average age of those writers whose age had to be imputed from photos – 46.8 for Indies versus 44.3 for Trads.  So, if these writers seem to be somewhat reluctant to be entirely transparent about age, it is not because Indies are less  open than Trads.  Instead, it seems to be related to genre (Romance), with Indies and Trads probably both behaving in a way normative for that genre.  It is possible that Romance writers are particularly circumspect about revealing their ages,  for fear that readers might consider certain age ranges inappropriate - either too young to really understand the more complicated features of relationships, or too old to still appreciate being in the throes of passion.
Age Distribution by Price Range
Again, a definition is needed to put this data into context.   Low priced books are defined as being under $4.00, moderately priced books are between $4.00 and $7.99 and high priced books are $8.00 and up.
When we look at the data, we see that there is a clear distinction between books written by younger writers as opposed to those written by older writers.  The former tended to dominate the lower price range, while the latter dominated the higher price ranges.  In fact, the average age for writers of lower priced books was 48.3, while for higher priced books it was 60.8.  The average age for writers of moderately priced books was intermediate, at 51.7 years.

Writer Count

Writer Age Range

Price2

34 or Less

35-44

45-54

55 - 64

65 Plus

Total

1-Low

1

10

10

4

5

30

2-Mod

2

10

21

8

11

52

3-High


1

5

6

6

18

Grand Total

3

21

36

18

22

100
Once more we should note the trend with writers whose age had to be imputed from photos versus those whose age was given on Wiki.  The former had all of their books in the low and moderate categories;  the latter had books in all price ranges, but they were primarily in the moderate and high price ranges.
Age Distribution by Genre
The genre categories are fairly standard breakdowns.  The most obvious tendency in the data is that Romance writers tended to skew younger (mostly in the 35-44 and 45-54 ranges), though older writers also wrote their share of books in that genre.  Conversely, writers in the Thriller/Suspense/Crime genre tended to skew older (with half in the 65 Plus age range), though a few younger writers also wrote in that genre.  The remaining genres didn’t seem to show any clear age related trends.
 In terms of average age, in ascending order, the figures were: Romance (46.0), Science Fiction and Fantasy  (52.1), Literary Fiction (53.0), Other (54.4) and Thriller/Suspense/Crime (62.1).
Writer Count
Writer Age Range
Genre2
34 or Less
35-44
45-54
55 - 64
65 Plus
Total
LitFic
8.0
1.0
2.0
11.0
Other
2.0
5.0
7.0
Romance
3.0
16.0
20.0
3.0
5.0
47.0
SFF
2.0
2.0
2.0
1.0
7.0
Thriller/Suspense/Crime
1.0
6.0
7.0
14.0
28.0
Grand Total
3.0
21.0
36.0
18.0
22.0
100.0
Again, it is worthwhile to note that the group of writers whose age had to be imputed from photos were mostly in the Romance genre.
Summary
Summing up, when it comes to the average age of writers in the Amazon Top 100, we can say:
·         Their average age was 52.3 years, though there was a wide spread of ages in the data, from a low of 33 to a high of 86.  So, writing success is possible at any age, though it appears to be an activity where success builds over time.
·         Female writers were significantly younger than male writers, on average, about 51 years of ages versus 56 years of age.
·         There was no strong tendency as to writers being older or younger by sales rank decile or quartile - average ages were about the same throughout the top 100 when broken out in these groupings.
·         Indies were significantly younger than Trads, at about 45 years old versus 55 years old.
·         Younger writers tended to sell at lower prince points than older writers.
·         The Romance category had the youngest writers (46), on average.  The Thriller category had the oldest writers (62), on average.  The other genres were somewhere in the middle.
Going back to the graph with the solid blue and speckled red bars, it is interesting to speculate about the observation that most of the people that made up the red speckled bars were young female Romance writers (the actual breakdown is shown below).  About half of those were Indies.  Were these Indie writers being blocked from publishing by “gatekeepers”?  Were the Trad writers in this group being mostly ignored in sales rank lists, previous to the Amazon ebook revolution, due to the assumed less serious nature of their work?  If gatekeepers were excluding/ignoring a major portion of the writing population, we might expect the artificial looking step function that we observed with the blue bars to be replaced by the more natural looking normally distributed  function, once the “missing” writers were added back in.  After all, the latter function is generally what we seen in most natural situations, so it would make sense if it re-asserted itself once gatekeeper effects  were diminished.
Other = 2 females, average age 56
Romance = 34 females, average age 45
SFF = 1 male, age 45
Thriller = 1 female, age 47
Later blogs will look at reader satisfaction by age, sales (as estimated from number of reviews) by age, and revenue (as estimated by sales and book price) by age.  We will also look at how much the writers themselves earned in royalties by age.  At the end of the process, we hope to be able to see what the sweet spots for financial success are - i.e. age, gender, genre.





Thursday, 24 April 2014

Astrophysics Corner, Part 8 - The Shape of Space, with a Particular Emphasis on the Dodecahedron


In Book 1 of the Witches Stones series, a form of space travel takes advantage of a previously unknown peculiarity of the shape and dimensionality of space, called Omega-space:

“No human being understood exactly what happened when a spaceship slipped through "omega-space", instantaneously passing from one portion of the galaxy to another.”


The subject of the shape of space has long fascinated people.  The more primitive ancients tended to see the universe as a flat surface (the Earth), surrounded by water (the ocean), and with a dome-like structure above (the heavens) which were the domain of the bodies of the sky - the sun, moon, planets and stars.  Some theories said all of this was held up by a giant (Atlas), who was standing on the back of an equally enormous turtle.  What the turtle was standing on wasn’t considered a fair question - the famously facetious phrase “it’s turtles all the way down” was invented as a humorous evasion of the issue and has now become a standard joking response to skirt around any question involving an inconvenient  infinite regression.  I don’t think what was outside the dome was considered a fair question either.  Perhaps “it’s domes all the way up” might have been a good response.
Plato, in the Timaeus, looked at the universe both in its physical form and in an idealized form.  The former is subject to change and decay, the latter is eternal.  The former is also apprehended by the senses while the latter could be apprehended by pure reason.  In overall form, the universe was thought of as a sphere, the most perfect form, which the divine would naturally choose.
 It was also conceived to be related to one of the Platonic solids, the dodecahedron.  There are five platonic solids - the other four were thought to be what the matter of the physical universe was made of.  These are tetrahedron (fire), the octahedron (air), icosahedron (water) and cube (earth).   The shapes were matched with the four elements as then conceived, on the basis of such physical characteristics as spikiness, resistance to rolling and so forth.   The fifth, the dodecahedron was thought to represent the shape of the universe itself, perhaps partly because it is the platonic solid that looks most like a sphere.  You might think of a football (in North American terms, a soccer ball) to help you visualize the dodecahedron.   A dodecahedron is composed of 12 equal sized pentagons, however, while a soccer ball is technically a spherical truncated icosahedron, which is a combination of dodecahedron and icosahedron.
 Eventually other views of the universe came to be dominant.  Perhaps the one that we are all most familiar with is what might be described as the Newtonian view - that there are three dimensions that all go on forever, along with an absolute time that clicks along uniformly and relentlessly.  That’s what the average person probably thinks of when he or she hears the phrase “the shape of space”.
It might surprise you to know that mathematicians and astrophysicists (cosmologists) don’t see things that simply.  Mathematicians have devised a whole branch of mathematics (topology) that concerns itself with what spaces are logically possible and what characteristics those various spaces would have - dimensionality, distance metrics, axiomatic foundations and so forth.   Astrophysicists are actively seeking ways to measure the actual observable universe on the large scale to attempt to determine the shape of the space that we live in.  While doing so, they incorporate the complexities of Einstein’s theories of general and special relativity, which involve a much more elastic notion of space-time than Newton’s absolute space and time.
What is meant by measuring the large scale properties of the universe in order to infer its shape?  A useful analogy is that of intelligent creatures that live on a sphere, in only two dimensions.  How would they conceive of the space that they inhabit?  Well, since they can’t jump out into the third dimension, they have to do experiments to infer the large scale properties of their space.  If you find this example to be too contrived, you might think of people living on a planet with a very flat surface and a very opaque, cloudy atmosphere, tidally locked to its sun, such that all they saw when they looked up was a flat white sky and all that they saw when they looked around was a horizon stretching away.
One experiment these spherical surface bound creatures might try, is to simply walk in a straight line for a long time.  As we know, if you walked straight forward on an infinite plane, you would never come back to the same point, regardless of how long you walked.  But if you were walking in a straight line on the surface of a sphere, you would eventually come back to your starting point.  For example, if one of these creatures started at his equivalent of our north pole, he would eventually pass the equator, then the south pole, then the equator again, then back to the north pole.   Assuming that he could recognize some sort of unique landmark (or created one himself), he would know that he was back where he started.  From that, he could tell his space was unbounded (he didn’t come to a barrier he couldn’t cross) but still closed (he didn’t go on forever without coming back to his starting point).
Another experiment that these creatures could do involves measuring triangles.  On a flat Euclidian plane, we know that a triangle’s angles sum to 180 degrees (or pi radians).  But on a sphere, a triangle sums to more than 180 degrees, while on a saddle shaped surface it  sums to less than 180 degrees.  To see this for a sphere, just think of starting at the north pole, travelling straight down 0 longitude, turning right at the equator (i.e. a 90 degree turn), following the equator one fourth of the way, turning right again at the 90th longitude (another 90 degree turn), then ending up at the north pole. You would have done two right turns (180 degrees) and still had a 90 degree angle between the path you started on from the north pole, and the path you returned on.  That’s 270 degrees in all, more than the 180 degrees in a triangle on a plane.  So, once again, these creatures would know that the space that they inhabit is very different from a flat, infinite plane.
Similarly, human astrophysicists would like to measure certain large scale features of our observable universe to see what sort of space we live in.  One experiment would be to head out in a rocket for a long, long time and see if eventually you seemed to be passing the same bunch of galaxies that you started from.  Obviously that’s not exactly an achievable experiment for a number of reasons - for one thing, we lack such a rocket and for another the galaxies are constantly changing so you might not recognize them when you passed them again.  Plus, by the time you got back to let people know your result, they would probably have evolved into something else and you wouldn’t know how to communicate with them anyway.
But there are ways to attempt to measure the universe.  One is to examine large scale maps of the observable universe (usually based on satellite observations in various wavelengths, but especially microwave) to see whether the universe looks to be homogeneous and isotropic on all scales (i.e. it appears to be the same wherever you look in the sky, and on every scale).
This is generally done using a technique called spectral analysis, which is a mathematical algorithm for finding patterns or regularities in data of all sorts, based on math first developed by Fourier in the 19th century and elaborated by many others since then.   In simple cases, these patterns can be detected by humans without sophisticated mathematics or computer analysis, but in more complex cases, spectral analysis is used to discover where spikes of the “power” in a signal are located, which indicate some kind of regularity in the data, which could be in the time or the spatial domains.
For instance, in music we know there are various harmonics along with the fundamental note.  These can be uncovered using spectral analysis.  Extra-solar planets are often discovered via this technique - regularities in the dips in a stars light curve can reveal the presence of a planet or multiple planets.   Geophysical exploration makes wide use of these methods, to find interesting and possibly profitable anomalies in magnetic or gravitational data.  Electrical engineering makes wide use of it to examine the frequency responses of circuits - that’s where the expression “power spectra” comes from.  Many other examples of the use of spectral analysis could be listed.
In 2003, Luminet, Weeks et al produced a paper that examined the WMAP data (Wilkinson Microwave Anisotropy Probe), to examine the spherical harmonics in that dataset.  Basically, it is a temperature map of the sky, produced by a satellite telescope.  Much of the data maps quite well to infinite flat space, especially the higher harmonics.  But there are problems at some levels of the spherical harmonics that can’t be well explained by such a model.    Essentially some of the lower harmonics imply a finite universe, where the size of space itself cuts off some of the expected wavelengths in the spectrum.  In their paper they use the analogy of a bell, where some overtones are impossible because the wavelengths would be bigger than the bell itself.
They argue that something called a Poincare dodecahedral space fits the power spectra of the data very nicely.  Essentially, in this space any object “leaving the universe” (including light) goes out one face of the dodecahedron and returns from the opposite face, with a twist.  In this case, space would be unbounded but closed, rather like the sphere is for two dimensional creatures.  In this case it would be akin to a hyper-sphere and the Poincare dodecahedral space would bear a similar relationship to it that the dodecahedron bears to the sphere in our “normal” space.
So, amusingly enough, Plato might have been (more or less) right all along.  At least I think that’s one way of looking at it.  Read the paper “Dodecahedral space topology as an explanation for weak wide-angle temperature correlations in the cosmic microwave background” and judge for yourself.  It was in Nature and is also on the physics arxiv site (there was another paper on the subject in 2008 as well, using more data).
And here are a few visualizations just for fund (not strictly mathematically the same as what is described above, which is pretty hard to visualize J).