Split data.frame by country, and create linear regression model on each subset [duplicate]

1
votes

I have a data.frame of data from the World Bank which looks something like this;

  country date BirthRate     US.
4   Aruba 2011    10.584 25354.8
5   Aruba 2010    10.804 24289.1
6   Aruba 2009    11.060 24639.9
7   Aruba 2008    11.346 27549.3
8   Aruba 2007    11.653 25921.3
9   Aruba 2006    11.977 24015.4

All in all there 70 something sub sets of countries in this data frame that I would like to run a linear regression on.

If I use the following I get a nice lm for a single country;

andora = subset(high.sub, country == "Andorra")

andora.lm = lm(BirthRate~US., data = andora)

anova(andora.lm)
summary(andora.lm)

But when I try to use the same type of code in a for loop I an error which I'll print below the code;

high.sub = subset(highInc, date > 1999 & date < 2012)
high.sub <- na.omit(high.sub)
highnames <- unique(high.sub$country)

for (i in highnames) {
  linmod <- lm(BirthRate~US., data = high.sub, subset = (country == "[i]"))  
}

Error message:

Error in lm.fit(x, y, offset = offset, singular.ok = singular.ok, ...) : 
  0 (non-NA) cases

If I can get this loop to run I would ideally like to append the coefficients and even better the r-squared values for each model to an empty data.frame. Any help would be greatly appreciated.

Thanks

Josh

2
rdataframeregressionlinear-regression
Take off those quotes and brackets from around the i. R is not a macro language. Plus you need to stop overwriting values in loops. Either use lapply to return values in a list or learn to index list elements. – IRTFM
@BondedDust, thank you. That solved my problem. Could you offer a way to perform the task I am trying to do using lapply? – Josh
mods <- lapply( highnames, function(nm) lm(BirthRate~US., data = high.sub, subset = (country == nm)) ); mods[["Andorra"]] – IRTFM
I wonder if lmList from nlme would work for you. – Roman Luštrik

2 Answers

3
votes

This is a slight modification of @BondedDust's comment.

models <- sapply(unique(as.character(df$country)),
                 function(cntry)lm(BirthRate~US.,df,subset=(country==cntry)),
                 simplify=FALSE,USE.NAMES=TRUE)

# to summarize all the models
lapply(models,summary)
# to run anova on all the models
lapply(models,anova)

This produces a named list of models, so you could extract the model for Aruba as:

models[["Aruba"]]
2
votes

Have a look at the lmList function of the nlme package:

library(nlme)
lmList(BirthRate ~ US. | country, df)

Here, | country is used to create a regression for each individual country.