Skip to contents

Run Parameter Sweep for Baseline Method Comparisons (v2)

Source: R/run_param_sweep_baselines.R
run_param_sweep_baselines.Rd

Orchestrates a parameter sweep experiment to evaluate baseline method performance (SingleFA, SinglePCA, TwoStep_FA, TwoStep_PCA, MixtureFA, MixturePCA) using the v2 dynamic synergy simulator with three orthogonal control axes. This function focuses on method-level benchmarking rather than model comparison.

Usage

run_param_sweep_baselines(
  test = FALSE,
  grid_type = NULL,
  cores = 4,
  output_dir = NULL,
  param_grid = NULL,
  verbose = TRUE,
  align_basis = TRUE,
  refine_assignments = TRUE,
  parallel_strategy = c("grid", "sequential", "pbs", "future"),
  pbs_config = list(),
  pbs_submit = FALSE,
  pbs_wait = FALSE,
  future_backend = "multisession",
  future_resources = list()
)

Arguments

test

Logical. If TRUE, uses a smaller test grid for quick validation. Default is FALSE (full parameter grid).

grid_type

Character. Parameter grid type: "test", "paper_core", or "paper_stress". If NULL (default), uses the test parameter to determine the grid. When specified, overrides the test parameter.

cores

Integer. Number of CPU cores to use for parallel processing. Default is 4. Set to 1 for sequential processing.

output_dir

Character. Directory to save results. Default is "inst/paper/results_baselines_v2" if it exists (repo use), otherwise "results_baselines_v2" in the current working directory.

param_grid

Optional. Custom parameter grid data frame. If NULL, uses the built-in full or test grid based on the test parameter. Required columns: N, K, r, M, T_each, cluster_sep_spatial, cluster_sep_temporal, cluster_sep_stability, seed, run_id.

verbose

Logical. If TRUE, prints progress information. Default is TRUE.

align_basis

Logical. If TRUE (default), applies Hungarian algorithm alignment in Step 1.5 of TwoStep methods to ensure corresponding components across subjects are in the same order before clustering. Set to FALSE to disable alignment and compare performance with/without this step in the parameter sweep.

refine_assignments

Logical. If TRUE (default), performs a single reassignment pass after fitting cluster-specific models in Step 4 of TwoStep methods. Each subject is reassigned to the cluster that minimizes reconstruction error. Set to FALSE to disable reassignment for ablation studies.

parallel_strategy

Character. Parallelization strategy:

  • "grid": Local multicore parallelization using parLapplyLB

  • "sequential": No parallelization (single core)

  • "pbs": PBS/OpenPBS array job submission for HPC clusters

  • "future": Use future framework (supports local and HPC backends)

Default is "grid".

pbs_config

List. PBS configuration options (only used when parallel_strategy = "pbs"). See submit_pbs_sweep for details.

pbs_submit

Logical. If TRUE and parallel_strategy = "pbs", submit the job to PBS. If FALSE, only generate scripts (dry run). Default is FALSE.

pbs_wait

Logical. If TRUE, wait for PBS job completion and aggregate results. Default is FALSE.

future_backend

Character. Backend for future parallelization (only used when parallel_strategy = "future"):

  • "multisession": Local parallel R sessions (default)

  • "multicore": Local forked processes (Unix only)

  • "batchtools_torque": PBS/TORQUE scheduler

  • "batchtools_slurm": SLURM scheduler

See setup_future_backend for all options.

future_resources

List. HPC resource specifications for future.batchtools backends (e.g., list(walltime = "04:00:00", memory = "4gb")).

Value

Invisibly returns a list containing:

  • raw_results: Data frame with raw results from all runs

  • aggregated_results: Data frame with results aggregated across seeds

  • summary_file: Path to the summary CSV file

  • win_rates_file: Path to the win rates CSV file

  • output_dir: Directory where results were saved

Details

This function complements run_param_sweep by focusing on method-level benchmarking rather than model comparison. It evaluates:

  • 6 methods: SingleFA, SinglePCA, TwoStep_FA, TwoStep_PCA, MixtureFA, MixturePCA

  • 1 model type: Dynamic_v2 (with 3 orthogonal axes)

  • Metrics: ARI (clustering accuracy), VAF (reconstruction quality), BIC, SSE

  • Variable parameters: N, K, r, cluster_sep_spatial, cluster_sep_temporal, cluster_sep_stability

For parallel processing (cores > 1), the function uses parLapplyLB for load balancing. If the pbapply package is installed, a progress bar will be displayed.

PBS Mode

When parallel_strategy = "pbs", this function generates PBS array job scripts and optionally submits them. Each array task runs one parameter combination. Results are saved as individual RDS files and can be aggregated after job completion using aggregate_partial_results.

Future Mode

When parallel_strategy = "future", this function uses the future framework for flexible parallelization. Local backends (multisession, multicore) work out of the box. HPC backends (batchtools_torque, batchtools_slurm) require the future.batchtools package and appropriate cluster configuration.

See also

compare_baselines for single-run baseline comparison, run_param_sweep for model comparison parameter sweep, plot_sweep_summary for visualization

Examples

if (FALSE) { # \dontrun{
# Run test mode with 2 cores
results <- run_param_sweep_baselines(test = TRUE, cores = 2)

# Run full sweep with 8 cores
results <- run_param_sweep_baselines(test = FALSE, cores = 8)

# Run with custom output directory
results <- run_param_sweep_baselines(
  test = TRUE,
  cores = 2,
  output_dir = "my_baselines_results"
)

# Use future with local multisession backend
results <- run_param_sweep_baselines(
  test = TRUE,
  parallel_strategy = "future",
  future_backend = "multisession",
  cores = 4
)

# Use PBS for HPC cluster
results <- run_param_sweep_baselines(
  test = FALSE,
  parallel_strategy = "pbs",
  pbs_config = list(queue = "batch", walltime = "04:00:00", mem = "4gb"),
  pbs_submit = TRUE,
  pbs_wait = TRUE
)

# Run with custom parameter grid
library(dplyr)
custom_grid <- expand.grid(
  N = c(20, 50),
  K = c(2, 3),
  r = 2,
  M = 8,
  T_each = 200,
  cluster_sep_spatial = c(0.3, 0.6),
  cluster_sep_temporal = 0.6,
  cluster_sep_stability = 1.0,
  seed = 1:3,
  stringsAsFactors = FALSE
)
custom_grid$run_id <- seq_len(nrow(custom_grid))

results <- run_param_sweep_baselines(
  param_grid = custom_grid,
  cores = 8,
  output_dir = "inst/paper/results_baselines_custom"
)

# Access results directly
head(results$raw_results)
head(results$aggregated_results)
} # }