panel regression with non-individual-specific fixed effects

Question

I have weekly observations of revenues from the sale of different products, separately for different countries, like so:

df <- data.frame(year=rep(c(2002,2003), each=16),
             week=rep(1:4,4),
             product=rep(c('A','B'), each=8, times=2),
             country=rep(c('usa','germany'), each=4, times=4),
             revenue=abs(rnorm(32)))

That means observations of revenues are only unique for a combination of year-week-country-product

I would now like to estimate a model that includes fixed effects for the interaction of country and year and for each product but cannot figure out how to do this:

estimating via summary(lm(revenue~factor(paste(country,year)) + factor(product) + ..., data=df)) fails for lack of memory because my data set is rather larger than the example above, which means I have to estimate something on the order of 1000 fixed effects
as far as I understand it panels are better estimated using the plm package but my case doesn't seem to fit neatly within the standard framework of a panel in which observations differ only across one time and one cross-sectional dimension each and fixed effects are estimated for each. I can generate a time index from year and week but that (a) still leaves me with two cross-sectional dimensions and (b) will give me fixed effects for each year-week interaction, which is rather more fine than I want it to be.

Are there any ways of estimating this with plm or are there other packages which do this sort of thing? I know I could demean the data within the groups described above, estimate via lm and then do a df-correction, but I'd rather avoid this.

Matthew Matthew · Accepted Answer · 2014-10-21T17:52:12

First, create a variable, "fe", that identifies unique combinations of country, year, product.

library(data.table)
# convert data.frame to data.table
setDT(df)
# create a new group variable
df[, fe := .GRP, by = list(country, year, product)]
head(df)
   year week product country    revenue fe
1: 2002    1       A     usa 0.84131750  1
2: 2002    2       A     usa 0.07530538  1
3: 2002    3       A     usa 0.56183346  1
4: 2002    4       A     usa 0.80720792  1
5: 2002    1       A germany 1.25329883  2
6: 2002    2       A germany 0.44860296  2

Now use plm or felm. I like felm since it also works with multiple fixed effects and interactive fixed effects

library(lfe)
felm(revenue ~ week | fe, df)

panel regression with non-individual-specific fixed effects

1 Answers