glmer with user-defined link function giving error: (maxstephalfit) PIRLS step-halvings failed to reduce deviance

Question

While attempting to leverage a user-defined link function with a random-effect glmer, I've run into an error that I don't know how to troubleshoot:

Error: (maxstephalfit) PIRLS step-halvings failed to reduce deviance in pwrssUpdate

Does anyone have any advice on how one might approach resolving this error? It doesn't provide much direction.

I've attempted to follow the instructions for defining a new link function (specifically a scaled logit) as outlined at rpubs.com/bbolker/logregexp, but I would not be surprised if some aspect of my definition is incorrect. See anything I'm missing?

scaled_logit <- function(s = 1) {
  linkfun <- function(mu) log( max(0, mu / (s-mu)) )
  linkinv <- function(eta) s / (1 + exp(-eta))
  mu.eta <- function(eta) s * exp(-eta) / (1 + exp(-eta))^2
  valideta <- function(eta) TRUE
  link <- paste0('scaled_logit(',s,')')
  structure(list(linkfun = linkfun, linkinv = linkinv, mu.eta = mu.eta, valideta = valideta, name = link), class = 'link-glm')
}

There must be something wrong with this implementation, since estimation works fine with the standard binomial family (assumed logit link), but errors out when I reference this link with s=1 (which should be identical). Sample data can be generated as follows:

library(data.table)

courts <- 50
test_courts <- data.table(court = 1:courts,
                     court_factor = pmax(0, rnorm(courts, mean=1, sd=0.25)))
setkey(test_courts, court)
pros <- 100
test_pros <- data.table(ID = 1:pros,
                       deg1_rate = pmax(0, rnorm(pros, mean=0.02, sd=0.0075)))
setkey(test_pros, ID)

test_data <- data.table(expand.grid(ID = 1:pros, court = 1:courts))
setkeyv(test_data, c('ID','court'))
test_data <- merge(test_data, test_courts, by='court', all.x=TRUE)
test_data <- merge(test_data, test_pros  , by='ID'   , all.x=TRUE)

test_data[ , indict := sample(0:20, nrow(test_data), replace=TRUE)]
test_data[ , deg1 := rbinom(pros*courts, size=indict, prob=court_factor*deg1_rate)]

I've then been attempting to estimate the simple model

logit_link <- glmer(cbind(deg1, indict-deg1) ~ (1|ID) + (1|court), family=binomial, data=test_data[indict > 0])

and corresponding alternative

scaled_link <- glmer(cbind(deg1, indict-deg1) ~ (1|ID) + (1|court), family=binomial(link=scaled_logit()), data=test_data[indict > 0])

Any insights would be appreciated! I'm using lme4 1.1.6 on R 3.0.3.

Ben Bolker Ben Bolker · Accepted Answer · 2014-10-02T02:31:08

I thought your problem would turn out to be with failing to "clamp" the inverse-link function (i.e. keeping the results strictly between 0 and 1), but it turns out that (I think) it's much simpler than that -- just a confusion of max() and pmax(). (max() has a pretty dangerous design!) This works for me:

scaled_logit <- function(s = 1) {
  linkfun <- function(mu) log( pmax(0, mu / (s-mu)) )
  linkinv <- function(eta) s / (1 + exp(-eta))
  mu.eta <- function(eta) s * exp(-eta) / (1 + exp(-eta))^2
  valideta <- function(eta) TRUE
  link <- paste0('scaled_logit(',s,')')
  structure(list(linkfun = linkfun, linkinv = linkinv, mu.eta = mu.eta, valideta = valideta, name = link), class = 'link-glm')
}

That said, it would probably be a good idea for future robustness to make that pmax(epsilon,...) rather than pmax(0,...) and to constrain the inverse link function between epsilon and 1-epsilon (where epsilon is something like 1e-6).

PS we (lme4 maintainers) should probably try to insert some more robust error-checking in the PIRLS step -- a lot of issues with NaN/non-finite values pop up looking like PIRLS failures, when that's not really what they are (nans seem to propagate through the C++ code without triggering any immediate failures ...)

glmer with user-defined link function giving error: (maxstephalfit) PIRLS step-halvings failed to reduce deviance

1 Answers