Check SSE Diagnostics for Mixture Models

This function performs diagnostic checks to investigate SSE (Sum of Squared Errors) behavior in mixture model fits, particularly when SSE increases monotonically with latent dimension r. It examines three key aspects: EM convergence quality, noise variance (Psi) distribution, and VAF-SSE correlation.

Usage

check_SSE_diagnostics(
  fit_results,
  list_of_data,
  mixtureFA_fit = NULL,
  mixturePCA_fit = NULL,
  output_dir = NULL,
  psi_small_thresh = 0.01,
  psi_large_thresh = 100
)

Arguments

fit_results

A data frame returned by fit_all_methods_on_data, containing columns: Method, BIC, ARI, SSE, VAF. The fitted model objects may be stored as attributes (mixtureFA_fit, mixturePCA_fit) if available.

list_of_data

The original data used for fitting (list of N matrices, each T_i x M). Required for VAF-SSE correlation theoretical line calculation.

mixtureFA_fit

Optional fitted MFA model object. If NULL, will attempt to extract from fit_results attributes. If not available, EM convergence and Psi diagnostics for MFA will be skipped.

mixturePCA_fit

Optional fitted MPCA model object. If NULL, will attempt to extract from fit_results attributes. If not available, EM convergence and Psi diagnostics for MPCA will be skipped.

output_dir

Optional directory path where diagnostic plots will be saved. If NULL (default), plots are displayed on screen only.

psi_small_thresh

Threshold for detecting overfitting (Psi values below this). Default is 0.01.

psi_large_thresh

Threshold for detecting poor fit (Psi values above this). Default is 100.

Value

A list with diagnostic results:

convergence_ok

Logical; TRUE if convergence appears stable

psi_ok

Logical; TRUE if Psi values are in reasonable range

correlation_ok

Logical; TRUE if VAF-SSE correlation is strongly negative

warnings

Character vector of warning messages

psi_summary_mfa

Summary statistics for MFA Psi values

psi_summary_mpca

Summary statistics for MPCA Psi values

vaf_sse_correlation

Correlation coefficient between VAF and SSE

Details

The function performs three main diagnostic checks:

1. EM Convergence Quality Check

Since the C++ EM implementation doesn't return a convergence trace, this check examines the final model quality by comparing log-likelihoods and checking for degenerate solutions (e.g., empty clusters, extreme parameter values).

2. Psi (Noise Variance) Distribution Check

Examines the distribution of noise variance parameters in both MFA and MPCA models. Extreme values (very small < 0.01 or very large > 100) may indicate:

• Overfitting (Psi approaches 0)

• Poor model fit (Psi approaches infinity)

• Numerical instability

Creates histograms showing the distribution of Psi values across all clusters and muscles.

3. VAF-SSE Correlation Verification

Since VAF = 1 - SSE/SST, there should be a strong negative correlation (approximately -1) between VAF and SSE across methods. Deviations from this expected relationship may indicate:

• Scaling inconsistencies in SSE calculation

• Different SST values being used across methods

• Numerical precision issues

Creates a scatter plot of VAF vs SSE with the expected theoretical relationship.

Examples

if (FALSE) { # \dontrun{
# Example 1: Basic usage with fit_all_methods_on_data()
# The fitted models are stored as attributes and automatically extracted
sim_data <- simulate_mixture_data(N = 30, K = 3, r = 3, M = 8, T_each = 200, seed = 123)
fit_results <- fit_all_methods_on_data(
  list_of_data = sim_data$list_of_data,
  z_true = sim_data$z_true,
  N = 30, K = 3, r = 3, M = 8,
  seed = 123
)

# Run diagnostics (fitted models extracted from attributes automatically)
diagnostics <- check_SSE_diagnostics(
  fit_results = fit_results,
  list_of_data = sim_data$list_of_data,
  output_dir = "diagnostics"
)

# Check results
print(diagnostics$warnings)
print(diagnostics$psi_summary_mfa)
print(diagnostics$vaf_sse_correlation)

# Example 2: Pass fitted models explicitly
# Useful when you have fitted models from other sources
mfa_fit <- mfa_em_fit(sim_data$list_of_data, K = 3, r = 3)
mpca_fit <- mixture_pca_em_fit(sim_data$list_of_data, K = 3, r = 3)

# Create a minimal results data frame
results_df <- data.frame(
  Method = c("MixtureFA", "MixturePCA"),
  SSE = c(100, 120),
  VAF = c(0.85, 0.80)
)

# Run diagnostics with explicit fitted models
diagnostics2 <- check_SSE_diagnostics(
  fit_results = results_df,
  list_of_data = sim_data$list_of_data,
  mixtureFA_fit = mfa_fit,
  mixturePCA_fit = mpca_fit,
  output_dir = "diagnostics2",
  psi_small_thresh = 0.005,  # Custom threshold
  psi_large_thresh = 200     # Custom threshold
)

# Example 3: VAF-SSE correlation only (no fitted models)
# Useful for quick checks without EM convergence/Psi diagnostics
diagnostics3 <- check_SSE_diagnostics(
  fit_results = fit_results,
  list_of_data = sim_data$list_of_data
)
# Note: EM convergence and Psi diagnostics will be skipped with a warning
} # }