A half thought out critique

Inspired by a recent post by Joern Fischer I have decided to share one of my (many) half baked ideas. It’s based on a paper I read recently that I have a few issues with and want to work up into a letter to the editor, so please see all this as a work in progress and if you want to co-author it with me feel free, because that way I’m less likely to get a bad rep.So to the paper.Liu_coverIt’s this one by Liu et al in Global Ecology and Biogeography on how climate and age determine biomass in global mature forests.It sounded right up my street.I like forests.I like carbon. So I gave it a look.In the paper Liu et al aim to:

  • Investigate the relationship between aboveground biomass and climatic conditions and stand age in mature forests across the globe.
  •  Identify an age threshold at which forests should be considered ‘mature.’

The first question is interesting because people have done similar things with secondary forests in the past but I’m not sure I’ve ever seen this done looking at stand age in mature forests as a factor affecting biomass.The second I’m not a big fan of, but I will come to why later in the post.So Liu et al carry out a few analyses looking at the effects of mean annual temperature, mean annual rainfall, and stand age on biomass. However, the graphs of the analysis looks like this:Liu_fig_1 Liu_fig_2The figures had me worried and on closer inspection my suspicions were confirmed. They considered each explanatory variable independently in separate models. This is bad statistics but also doesn’t take account of the fact that previous studies have suggested that age, precipitation and temperature may interact to determine carbon accumulation rates. In addition the paper fails to account for spatial autocorrelation or differences between datasets that may be purely because of different methods used in their collection, rendering the results they present as questionable.To their credit Liu et al provide the data they used as supplementary materials so I thought I’d have a play with it to try and fix some of their errors.First I created a distance matrix and used that to look at spatial autocorrelation in biomass – surprise, surprise there were signs of spatial autocorrelation.I built a model that accounted for this and used a random effect to distinguish between each of the different datasets used in the study to account for between study error. I then did some model averaging so that all possible combinations of precipitation, temperature and age were included. I’ve put all of the code on github so feel free to look there and comment if you have any suggestions about the technical aspects of what I was up to.To cut a long story short the results suggest that all the variables considered by Liu et al are important, with one model that included all of them, as well as an interaction between temperature and age coming out as by far and away the best model.Age_tempTemp_age

Comparison of coefficients of our model compared to that of Liu et al. In each case the dotted black line represent’s Liu et al’s models and the coloured lines our models. Predictions were only made for interactions where there was sufficient data for both variables to allow this.


This model was much better than those of Liu et al (Table 1) – suggesting their approach was overly simplistic, as well as being statistically flawed. So, the models I developed explained much more variability than the equivalent ones in the Liu et al paper and changes the spin they put on their results.

Table 1 – Comparison of my top model and the models of Liu. AICc indicates relative parsimony of the model.

Model AICc AICc delta R squared
My model 623.98 0 0.29
Liu – Precipitation only 678.51 54.53 0.11
Liu – Temperature only 698.88 74.90 0.08
Liu – Age only 733.94 109.96 0.02


This model has an R squared 0.28, which is very good given the scale of the analysis but also suggests that there is quite a lot going on that we aren’t capturing in this model.  Part of this is probably because of the noise inherently added by using data collected in different ways.  In what I think is the best study of it’s type to date suggests that biomass in mature forests is only weakly related to commonly used climate metrics like mean temperature and mean precipitation. Instead, James Stegen and colleagues suggest that total biomass is well predicted by the biomass of the largest individual tree and that this is constrained by water deficit.

Now I to the second aim of Liu et al: to define a threshold age for mature forest.

This I have a big problem with. Even mature forests subject to relatively similar climatic conditions can vary massively in biomass and the reasons for this are not completely clear. Given this it is unwise to try to define a global threshold. It would be a much better idea to use chronosequence studies or long-term monitoring to try to discern dynamics at a landscape scale and build upon that to determine when forest should be classed as mature (and I’m only partly saying that because that’s what we did with secondary forest data…). I also have fears about defining ‘mature forest’ only using the biomass of these forests, and would be interested in seeing how biodiversity varies along with age in these old growth forests. Given that secondary forest carbon can get close to recovery quite quickly while biodiversity lags behind similar relationships may be seen for old growth forest. Any policy definition of what mature forest is could potentially have big implications for global biodiversity, so it important we get it right.

So those are my ideas. Critique them or add to them as you wish. And as I said, all code is available on github along with the data from the paper. I’m serious about writing a response, but like I said it needs more work so if you want to join me drop me a message below or in an email.

7 thoughts on “A half thought out critique

  1. Generally I agree with all your comments and at-least for me it is not understandable how this paper could get published in Global Ecol. Biogeogr. without accounting for spatial autocorrelation.
    By the way: Have you considered taking in the “Site” index as nested random effect within “Ref” in your topmodell? This improves the corrected AIC to 571.9873 and your r-squared to 0.31 compared to Liu’s models! Makes sense as different plot location in the same publication account for site-specific variation.

    1. Good point Martin. I’m not used to doing analyses that account for Spatial Autocorrelation so could you give me an example of how to nest that in Ref in some code.

      On another point do you fancy co-authoring a letter? I’m sure you can offer all sorts of help from the statistics side.

      1. Sure why not. But i have to give the topic a bit more reading time. For now I just looked and executed your code. Generally correcting SAC in mixed effects models is not an easy task (which is maybe also why they left it out). Based on the histograms i looked at we might even consider a poison error structure, thus using lmer (lme4) or another package. However those don’t allow spatial correlation matrices… Have to play around a bit.
        You can easily fit a nested effect by using a blackslash in lme as “..,random=~1|Ref/Site,..”
        I will send you a mail the upcoming days. For now i am still at WCMC throwing data around.

      2. Ok that all sounds fine, apart from the poisson bit since that is only valid for count data, isn’t it?

        What are you doing at WCMC? I used to work there many moons ago.

      3. You are right, just mentioning that the response you used is non-normal (which is why you log-transformed it). Another distribution with a log-link might be appropriate as well. Anyway I looked at the residuals of the models you produced again and they look pretty good/normal, thus i guess its fine.
        I will play a bit around and donate some code and come back to you.
        At WCMC right now i am working with Neil Burgess and Tim Newbold on the Predicts project, which is part of my thesis.

  2. Sounds like you are on the right track. It is odd that Liu et al. would consider those explanatory factors would be analyzed independently since they are clearly not independent variables. Some sort of spatial autocorrelation analysis should definitely have been included.
    I also agree with you that the threshold for mature forests concept is problematic. Its similarly difficult to define at what age a forest is considered “old-growth” in that there are a lot of landscape and climatic factors that need to be considered, which I don’t see being discussed thoroughly in the article. Mature in a rainforest probably shouldn’t be the same term for mature in a boreal forest, etc.

    1. Hi Relena. Thanks for the comment.

      I obviously agree with your points. The statistical problems invalidate their conclusions, but I think that Liu et al also lack a lot of data from the tropics and subtropics which would limit their conclusions anyway.

      The threshold is a tricky one. I think that the separation of secondary and old-growth is a bit false because really it’s a gradient. That aside, I really don’t think we can define ‘old growth’ in the terms Liu et al think we can. I think good landscape scale studies of chronosequences and long-term monitoring of forest plots is useful for tidying up this situation. I would really like to see more work looking at the landscape determinants of carbon and biodiversity in forests.

      As you say I can’t imagine that forests in different climatic zones would be likely to have similar biomass, just as you don’t expect them to have similar biodiversity.

      Anyway, after all that, do you fancy co-signing the letter with me when I get it tidied up? I’d just need to read a draft and help make some corrections as you see fit.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s