Tutorial 5: Parallel and cloud execution¶

Learn to run multiple simulations simultaneously on local hardware or cloud resources.

What you'll learn¶

• Local parallel execution with goliat parallel
• Config splitting strategies
• Managing multiple GUI instances
• Results merging
• oSPARC batch execution overview

Related documentation: Cloud setup

Prerequisites¶

• Tutorial 1-4 completed
• Multi-core CPU (4+ cores recommended)
• For oSPARC: API credentials in .env file

Bash setup¶

Run this once per notebook session:

This helper function lets you run bash commands from Python cells using run_bash('command'). The setup also disables output truncation so you can see all command output.

If you're using bash directly (recommended), ignore the Python code blocks and just run the commands directly. Make sure to always run source .bashrc first.

Why parallel execution matters¶

A single simulation can take 15-40 minutes depending on complexity. A study with 4 phantoms and 2 frequencies requires 8 simulations, which would take 2-5 hours sequentially.

With parallel execution: - 4-core machine: Run 4 simulations simultaneously (cut time by 4x) - Cloud resources: Run dozens simultaneously (finish large studies in hours)

GOLIAT supports two parallel strategies:

This tutorial focuses on local parallel execution. oSPARC batch is covered briefly with links to detailed documentation.

Important limitation: GPU and iSolve execution¶

Important: When running parallel simulations on a single machine with one GPU, iSolve will only execute one simulation at a time. This is a fundamental limitation of Sim4Life's GPU-based solver.

What this means: - Setup and extract phases can run in parallel (no GPU required) - Run phase (iSolve) cannot run in parallel on a single GPU machine - Multiple parallel processes will queue for GPU access, effectively running sequentially

Impact: Running goliat parallel with --num-splits 4 on a single-GPU machine will: - Speed up setup phases (4x faster) - Speed up extract phases (4x faster) - NOT speed up run phases (still sequential, same total time as running individually)

Solutions for true parallel iSolve execution: 1. oSPARC batch: Submit simulations to cloud platform where each job gets its own GPU 2. Multiple Windows PCs: Set up GOLIAT on multiple machines as described in Cloud Setup

For large studies requiring many simulations, oSPARC batch is the recommended approach.

Understanding config splitting¶

The goliat parallel command splits a large config into smaller configs, each handling a subset of the work.

Splitting strategies¶

GOLIAT uses a smart algorithm that considers: - Number of phantoms - Number of frequencies (far-field) or antennas (near-field) - Target number of splits

The algorithm finds the best factorization to distribute work evenly.

Example 1: 4 phantoms, 2 frequencies, 4 splits - Split phantoms into 4 groups (1 phantom each) - Keep all frequencies in each config - Result: 4 configs, each with 1 phantom and 2 frequencies

Example 2: 2 phantoms, 8 frequencies, 4 splits - Keep all phantoms in each config - Split frequencies into 4 groups (2 frequencies each) - Result: 4 configs, each with 2 phantoms and 2 frequencies

Example 3: 4 phantoms, 4 frequencies, 4 splits - Split phantoms into 2 groups (2 phantoms each) - Split frequencies into 2 groups (2 frequencies each) - Result: 4 configs (2 phantom groups × 2 frequency groups)

The algorithm balances work across splits to minimize idle time.

The configuration file¶

This tutorial uses a far-field config with 4 phantoms and 2 frequencies, totaling 16 simulations (4 phantoms × 2 frequencies × 2 directions × 1 polarization).

Key parameters¶

{
  "study_type": "far_field",
  "phantoms": ["duke", "thelonious", "eartha", "ella"],
  "frequencies_mhz": [700, 900],
  "far_field_setup": {
    "type": "environmental",
    "environmental": {
      "incident_directions": ["x_pos", "z_neg"],
      "polarizations": ["theta"]
    }
  }
}

This config generates: - 4 phantoms - 2 frequencies - 2 incident directions - 1 polarization - Total: 4 × 2 × 2 × 1 = 16 simulations

Running sequentially would take 4-8 hours. Running in parallel on 4 cores takes 1-2 hours.

Running parallel studies¶

The goliat parallel command handles splitting, launching, and coordinating multiple studies.

Step 1: Split the config¶

Run:

Output:

Creating parallel configs in: configs/tutorial_5_parallel_parallel
  - Copied: base_config.json
Smart split strategy: 4 phantom group(s) × 1 frequencies group(s) = 4 total configs
Phantom groups: [1, 1, 1, 1]
Frequencies groups: [2]
  - Created: tutorial_5_parallel_0.json (phantoms: ['duke'], frequencies: 2)
  - Created: tutorial_5_parallel_1.json (phantoms: ['thelonious'], frequencies: 2)
  - Created: tutorial_5_parallel_2.json (phantoms: ['eartha'], frequencies: 2)
  - Created: tutorial_5_parallel_3.json (phantoms: ['ella'], frequencies: 2)
Config splitting complete.
Found 4 configs to run in parallel.
Removed stale lock file: goliat.lock
--- Study 1/4 started ---
--- Study 2/4 started ---
--- Study 3/4 started ---
--- Study 4/4 started ---

Terminal showing config split details.

Step 2: Inspect split configs¶

The split configs are in configs/tutorial_5_parallel_parallel/:

Output:

base_config.json
tutorial_5_parallel_0.json
tutorial_5_parallel_1.json
tutorial_5_parallel_2.json
tutorial_5_parallel_3.json

Each config is a complete, runnable configuration with a subset of the original work.

Check one split:

You'll see it contains only duke in the phantoms list, while keeping both frequencies.

Directory showing split config files.

Step 3: Multiple GUIs¶

After splitting, GOLIAT launches one goliat study process per config. Each process opens its own GUI window.

You'll see 4 GUI windows, each tracking one phantom's simulations.

Four GOLIAT GUIs running in parallel, one per phantom.

Each GUI shows: - Its subset of simulations (e.g., duke at 700 MHz and 900 MHz) - Progress through setup, run, extract phases - Independent ETA calculations - System utilization (shared across all processes)

Step 4: Monitor system resources¶

With 4 parallel processes, CPU usage should be high across multiple cores.

Open Task Manager (Windows) or Activity Monitor (Mac) to verify: - CPU usage: 80-100% (varies based on FDTD vs setup work) - RAM usage: Increases per simulation (2-8 GB each) - Disk I/O: Moderate (reading/writing project files)

Task Manager showing high CPU usage across multiple cores.

If any process idles while others work, the split may be unbalanced. Adjust split strategy or reduce number of splits.

Step 5: Results merging¶

When all processes complete, results are in the standard directory structure:

results/far_field/duke/700MHz/...
results/far_field/duke/900MHz/...
results/far_field/thelonious/700MHz/...
results/far_field/thelonious/900MHz/...
results/far_field/eartha/700MHz/...
results/far_field/eartha/900MHz/...
results/far_field/ella/700MHz/...
results/far_field/ella/900MHz/...

No manual merging is needed. Each process writes to its own phantom/frequency subdirectories, and the results structure is identical to running sequentially.

Single results directory containing outputs from all parallel processes.

Command options¶

The goliat parallel command supports several options.

Number of splits¶

goliat parallel my_config --num-splits N

Choose N based on: - Number of CPU cores (N ≤ cores for best performance) - Available RAM (each simulation needs 2-8 GB) - Number of phantoms and frequencies (N must factor evenly)

Valid splits: - 4 phantoms, 2 frequencies: 1, 2, 4, 8 splits work - 2 phantoms, 3 frequencies: 1, 2, 3, 6 splits work - 1 phantom, 8 frequencies: 1, 2, 4, 8 splits work

If you request an invalid split, GOLIAT reports the closest achievable factorization.

Skip split¶

If you already split a config and want to re-run:

goliat parallel my_config --skip-split

This reuses the existing split directory without regenerating configs. Useful for: - Re-running after errors - Tweaking execution control settings - Testing with --no-cache

No cache¶

Force all simulations to re-run:

goliat parallel my_config --no-cache

This disables the caching system, ignoring any existing config.json metadata. Useful for: - Testing config changes - Debugging setup issues - Regenerating results after code updates

When to use parallel execution¶

Good cases: - 10-50 simulations - Multi-core machine (4+ cores) - Enough RAM (8-32 GB) - Local compute sufficient for timeline

Avoid when: - Only 2-5 simulations (overhead not worth it) - Low RAM (causes thrashing) - 50+ simulations (use oSPARC batch instead)

oSPARC cloud execution overview¶

For large studies (50+ simulations), oSPARC provides cloud compute resources. This is a brief overview; see the oSPARC cloud execution for complete instructions.

Workflow¶

1. Generate input files locally:
2. Set only_write_input_file: true

3. Run goliat study to create .h5 solver files

4. Submit batch to oSPARC:

5. Set batch_run: true
6. Run goliat study again

7. GOLIAT uploads files and monitors job status

8. Download and extract:

9. When jobs complete, GOLIAT downloads results
10. Set do_extract: true to process SAR data

Credentials setup¶

oSPARC requires API credentials in a .env file:

OSPARC_API_KEY=your_api_key_here
OSPARC_API_SECRET=your_api_secret_here

Get credentials from https://api.sim4life.science

Example .env file with oSPARC credentials (redacted).

Batch configuration¶

Enable batch mode in your config:

{
  "execution_control": {
    "do_setup": true,
    "only_write_input_file": true,
    "do_run": false,
    "do_extract": false,
    "batch_run": true
  }
}

Config section enabling batch mode.

Monitoring¶

During batch execution, the GUI shows job status:

--- Submitting Jobs to oSPARC in Parallel ---
  - Submitted job 1/50: duke_700_x_pos_theta
  - Submitted job 2/50: duke_700_x_neg_theta
  ...
  - Job 1/50: PENDING
  - Job 2/50: SUCCESS (downloaded)
  - Job 3/50: RUNNING

Jobs progress through states: - PENDING: Queued, waiting for resources - RUNNING: Executing on oSPARC compute node - SUCCESS: Completed, results downloaded - FAILED: Error occurred (check logs)

GUI showing oSPARC job status monitoring.

oSPARC dashboard¶

You can also monitor jobs at https://api.sim4life.science

oSPARC web interface showing running jobs.

Costs and limits¶

oSPARC charges based on: - Compute time (per hour per core) - Storage (per GB)

The GOLIAT EU Project has access to dedicated resources. For individual use, check pricing and quotas.

oSPARC limits: - ~61 parallel jobs per user - Storage quotas (varies by plan) - API rate limits (handled by GOLIAT)

For more details, see: - oSPARC cloud execution

Choosing the right strategy¶

Hybrid approach: - Use local parallel for setup (fast) - Generate input files - Submit batch to oSPARC for run phase - Extract locally

This balances speed, cost, and resource usage.

What's next¶

You've completed the GOLIAT tutorial series. Next steps:

Explore advanced features: - Free-space antenna validation - Custom analysis scripts - Result visualization tools - Integration with other EMF tools

Read the documentation: - Configuration reference: All config parameters - Advanced features: Phantom rotation, profiling, custom setups - Cloud setup: VM deployment - oSPARC: Cloud batch execution

Summary¶

You learned: - Local parallel execution uses goliat parallel to split configs and run multiple studies simultaneously - Config splitting uses smart factorization based on phantoms and frequencies - Each split runs in its own process with its own GUI window - Results merge automatically into the standard directory structure - Choose number of splits based on CPU cores, RAM, and work distribution - oSPARC batch execution scales to hundreds of simulations on cloud resources - oSPARC requires API credentials and uses a three-step workflow (generate, submit, extract) - Hybrid approaches combine local setup with cloud execution for efficiency

You've completed all five tutorials and can now run sophisticated EMF dosimetry studies with GOLIAT.