R extract regression coefficients from multiply regression via lapply command

Question

I have a large dataset with several variables, one of which is a state variable, coded 1-50 for each state. I'd like to run a regression of 28 variables on the remaining 27 variables of the dataset (there are 55 variables total), and specific for each state.

In other words, run a regression of variable1 on covariate1, covariate2, ..., covariate27 for observations where state==1. I'd then like to repeat this for variable1 for states 2-50, and the repeat the whole process for variable2, variable3,..., variable28.

I think I've written the correct R code to do this, but the next thing I'd like to do is extract the coefficients, ideally into a coefficient matrix. Could someone please help me with this? Here's the code I've written so far:

for (num in 1:50) {

    #PUF is the data set I'm using

    #Subset the data by states
    PUFnum <- subset(PUF, state==num)

    #Attach data set with state specific data
    attach(PUFnum)

    #Run our prediction regression
    #the variables class1 through e19700 are the 27 covariates I want to use
    regression <- lapply(PUFnum,  function(z) lm(z ~ class1+class2+class3+class4+class5+class6+class7+
                                                     xtot+e00200+e00300+e00600+e00900+e01000+p04470+e04800+
                                                     e09600+e07180+e07220+e07260+e06500+e10300+
                                                     e59720+e11900+e18425+e18450+e18500+e19700))

    Beta <- lapply(regression, function(d) d<- coef(regression$d))


    detach(PUFnum)
}

Ramnath Ramnath · Accepted Answer · 2012-01-26T20:35:17

This is another example of the classic Split-Apply-Combine problem, which can be addressed using the plyr package by @hadley. In your problem, you want to

Split data frame by state
Apply regressions for each subset
Combine coefficients into data frame.

I will illustrate it with the Cars93 dataset available in MASS library. We are interested in figuring out the relationship between horsepower and enginesize based on origin of country.

# LOAD LIBRARIES
require(MASS); require(plyr)

# SPLIT-APPLY-COMBINE
regressions <- dlply(Cars93, .(Origin), lm, formula = Horsepower ~ EngineSize)
coefs <- ldply(regressions, coef)

   Origin (Intercept) EngineSize
1     USA    33.13666   37.29919
2 non-USA    15.68747   55.39211

EDIT. For your example, substitute PUF for Cars93, state for Origin and fm for the formula

R extract regression coefficients from multiply regression via lapply command

2 Answers