Essential fit and recovery metrics for an L0glm model

Usage

L0glm.metrics(
  object,
  newdata = NULL,
  y = NULL,
  weights = NULL,
  offset = NULL,
  mu_floor = 1e-08,
  beta_true = NULL,
  r2_mu_floor = 0.1,
  ...
)

# S3 method for class 'L0glm_fit'
L0glm.metrics(
  object,
  newdata = NULL,
  y = NULL,
  weights = NULL,
  offset = NULL,
  mu_floor = 1e-08,
  beta_true = NULL,
  r2_mu_floor = 0.1,
  ...
)

# Default S3 method
L0glm.metrics(object, ...)

# S3 method for class 'sparseglm'
L0glm.metrics(
  object,
  newdata = NULL,
  y = NULL,
  weights = NULL,
  offset = NULL,
  mu_floor = object$mu_floor,
  beta_true = object$beta_true,
  r2_mu_floor = 0.1,
  ...
)

Arguments

object: A model returned by L0glm.fit() or L0nnpois(), or a comparison from sparseglm().
newdata: New predictor matrix, without an automatically added intercept, or NULL for training data.
y: Responses for newdata; required when it is supplied. NULL otherwise uses the training responses. Special-family response formats are described in L0glm.fit().
weights: Optional prior observation weights for scoring. Training scores default to the original prior weights, rather than IRLS working weights. New data default to unit weights. Supports the fitting family's vector or matrix weight formats.
offset: Optional prediction offsets. Training scores use the stored offsets; new data default to zero. Supports the fitting family's offset formats.
mu_floor: Nonnegative scoring-only floor for Poisson, NB, Gamma and inverse-Gaussian means. The default is 1e-8; zero requests exact scoring. This does not alter the fitted model. logLik() and the criterion methods default to zero.
beta_true: Optional true coefficient vector or matrix, on the original scale and with the same dimensions as coef(object), including any intercept. Multinomial truth must use the displayed CLR coordinates.
r2_mu_floor: Nonnegative floor for count-model means when calculating R-squared working weights. The default is 0.1; zero uses only the machine-precision guard. Applies to Poisson, quasipoisson and negative binomial models. Residuals retain the original predictions. This is independent of mu_floor and does not affect fitting, deviance, likelihood or information criteria.
...: Further S3 method arguments; currently unused.

Value

A one-row data frame of fit metrics and information criteria. Ordinary GLMs include row and coefficient summaries, with NA values where truth is unavailable. Special-family scores retain their model-specific objective summaries; supplying beta_true also adds the same recovery metrics. The information criteria are aic, bic, gic, ebic and mbic. For a sparseglm collection, one row per requested method includes timings, status, grouping scope, approximation and tuning values. Failed methods retain a row with unavailable fit metrics.

Details

Row support is defined by at least one exactly nonzero coefficient in a row. Intercepts and other always.include terms are excluded from support classification. Overall row metrics include TP, FP, TN, FN, precision, recall, F1 and Matthews correlation coefficient (MCC/Phi).

For multichannel data, within_row_* metrics assess coefficient zero/nonzero classification within correctly recovered rows. If \(R=\{j:\|\widehat\beta_{j\cdot}\|_1>0, \|\beta^*_{j\cdot}\|_1>0\}\) excludes mandatory rows, these counts pool all entries \((j,k)\) with \(j\in R\) across channels. They are micro-averaged, rather than an average of per-row scores. Missed and spurious rows remain visible in the overall row metrics and are not included in the conditional scores. within_row_TP, within_row_FP, within_row_TN and within_row_FN use exact nonzero indicators, and their derived precision, recall, within_row_F1_score and within_row_MCC describe recovery of the within-row sparsity pattern. No additional numerical coefficient threshold is applied.

Within-row metrics are NA for a single response or unavailable truth. If no rows are correctly recovered, the multichannel counts are zero and the ratios are NA. Any ratio with zero denominator is NA; MCC is NA if either binary margin is constant.

Coefficient accuracy includes mean absolute error (MAD_coefs), RMSE_coefs and R2_coefs across all entries, including mandatory terms. R2_coefs is the through-origin regression R-squared, \((\widehat\beta^T\beta^*)^2/ (\|\widehat\beta\|_2^2\|\beta^*\|_2^2)\), with matrices vectorized. It measures alignment and can equal one despite multiplicative bias; MAE and RMSE quantify absolute error.

Prediction accuracy includes R2_weighted and adj_R2_weighted for ordinary GLMs and negative binomial models. Working weights are recomputed at the fitted means as \(w_{ij}=v_{ij}\{d\mu_{ij}/d\eta_{ij}\}^2/V(\mu_{ij})\), using original prior weights \(v\), the fitted link, and fitted NB theta. For Poisson, quasipoisson and negative-binomial models, these weights use \(\widetilde\mu=\max(\widehat\mu,\mathtt{r2\_mu\_floor})\), with a default floor of 0.1. This prevents zero or tiny fitted count means from receiving enormous inverse-variance weights. The original fitted means are retained in the residuals. Other positive families use only a machine-precision guard for their weight calculation. The separate deviance scoring floor does not change these R-squared scores. These are response-scale residual scores, not working-response or deviance pseudo-R-squared measures.

Let \(SSE=\sum_{ij}w_{ij}(y_{ij}-\widehat\mu_{ij})^2\) and \(SST=\sum_{ij}w_{ij}(y_{ij}-c_j)^2\). With a unit intercept column, \(c_j\) is the channel's working-weighted response mean; without one it is zero, following the centered/uncentered convention of stats::summary.lm(). Then \(R_w^2=1-SSE/SST\) and \(R_{w,adj}^2=1-(SSE/SST)(N-b)/(N-k)\). Here \(N\) counts positive-working-weight response entries, \(b\) counts channel intercepts, and \(k\) counts nonzero coefficients plus mandatory terms even when zero. Multichannel sums pool within-channel residuals and baselines, rather than averaging channel R-squared values. Precision weights do not change \(N\) into a sum of weights. Dispersion/theta are not mean-model degrees of freedom. The adjustment uses active-set size, not an estimate of selection or ridge effective degrees of freedom. Negative scores are retained.

Prediction R-squared is NA for zero total sum of squares, undefined working weights, or Cox/multinomial models, which do not have the scalar independent GLM working weights used by this definition. Adjusted R-squared is also NA when residual degrees of freedom are nonpositive. New-data scores use predictions on the new rows and the original fitted active-set size.

Likelihood and criterion conventions, including dispersion estimates, sample sizes and grouped parameter counts, are detailed in model_criteria. Ordinary GLMs also report deviance_true when truth is supplied: deviance comparing the noiseless true mean with the fitted mean, using the scoring weights and offsets.

Examples

set.seed(12)
X <- matrix(runif(300), 60, 5)
truth <- rbind(c(3, 0, 2), c(0, 4, 1), matrix(0, 3, 3))
y <- matrix(rpois(180, 1 + X %*% truth), 60, 3)
fit <- L0nnpois(X, y, offset = rep(1, 60), parallelize = FALSE)
scores <- L0glm.metrics(fit, beta_true = truth)
scores[c("F1_score", "MCC", "within_row_F1_score", "within_row_MCC")]