superstats.diagnostics#

Metrics and plotting helpers for validating superstats workflows.

superstats.diagnostics.calibration_error_per_step(estimates, targets, resolution=20, aggregation=<function median>, min_quantile=0.005, max_quantile=0.995)[source]#

Marginal calibration error per step and parameter.

Computes an aggregate score for the marginal calibration error over an ensemble of approximate posteriors, per step (time step). The calibration error is given as the aggregate (e.g. median) of the absolute deviation between an alpha-CI and the relative number of inliers from estimates, over multiple alphas in (0, 1).

Parameters:

estimatesnp.ndarray of shape (num_sim, num_samples, num_steps, num_params): Posterior samples.
targetsnp.ndarray of shape (num_sim, num_steps, num_params): Ground-truth parameter values.
resolutionint, optional, default: 20: Number of credibility intervals (CIs) to consider.
aggregationcallable(), optional, default: np.median: Function used to aggregate the per-alpha calibration errors. Typically np.mean or np.median.
min_quantilefloat in (0, 1), optional, default: 0.005: Minimum posterior quantile to consider.
max_quantilefloat in (0, 1), optional, default: 0.995: Maximum posterior quantile to consider.

Returns:

calibration_errornp.ndarray of shape (num_steps, num_params) -: aggregated calibration error, per step and parameter

Parameters:

estimates (ndarray)
targets (ndarray)
resolution (int)
aggregation (Callable)
min_quantile (float)
max_quantile (float)

Return type:

ndarray

superstats.diagnostics.correlation_per_step(estimates, targets, aggregation=<function median>)[source]#

Pearson correlation between point estimates and true values, per step and parameter.

Posterior samples are first collapsed to a point estimate per simulation, step, and parameter (using aggregation), then the Pearson correlation between those point estimates and targets is computed across simulations.

Parameters:

estimatesnp.ndarray of shape (num_sim, num_samples, num_steps, num_params): Posterior samples.
targetsnp.ndarray of shape (num_sim, num_steps, num_params): Ground-truth parameter values.
aggregationcallable(), optional, default: np.median: Function used to collapse posterior samples into a point estimate per simulation, step, and parameter. Typically np.mean or np.median.

Returns:

correlationnp.ndarray of shape (num_steps, num_params) - Pearson: correlation per step and parameter

Parameters:

estimates (ndarray)
targets (ndarray)
aggregation (Callable)

Return type:

ndarray

superstats.diagnostics.nrmse_per_step(estimates, targets, aggregation=<function median>)[source]#

Normalized RMSE between posterior samples and targets, per step and parameter.

RMSE is computed across posterior draws (not aggregated first) for each simulation, then normalized by a prior-only bootstrap RMSE aggregated across simulations. This follows the “prior” normalization scheme: 0 indicates a maximally informative posterior (point mass at ground truth), 1 indicates a non-informative posterior (equivalent to the prior). The per-simulation ratios are then aggregated over simulations (using aggregation, mirroring the bayesflow convention of aggregating the final metric with the same function used for the normalizer), yielding one value per step and parameter.

Parameters:

estimatesnp.ndarray of shape (num_sim, num_samples, num_steps, num_params): Posterior samples per simulation.
targetsnp.ndarray of shape (num_sim, num_steps, num_params): Target parameter trajectories (themselves prior draws, in a simulation-based calibration setting).
aggregationcallable(), optional, default: np.median: Function used to aggregate both the prior-only bootstrap RMSE (for the normalizer) and the final per-simulation nRMSE values across simulations. Typically np.mean or np.median.

Returns:

nrmsenp.ndarray of shape (num_steps, num_params) - RMSE across: posterior draws, normalized by the aggregated prior-only bootstrap RMSE, aggregated across simulations

Parameters:

estimates (ndarray)
targets (ndarray)
aggregation (Callable)

Return type:

ndarray

superstats.diagnostics.plot_calibration(estimates, targets, variable_keys=None, variable_names=None, color='#356673', title_fontsize=22, label_fontsize=18, metric_fontsize=18, tick_fontsize=16, **kwargs)[source]#

Plot time-invariant calibration (ECDF).

Thin wrapper around bf.diagnostics.plots.calibration_ecdf that accepts dict-or-array input via prepare_plot_data, consistent with plot_time_varying_verification.

Parameters:

estimatesMapping[str, np.ndarray] or np.ndarray: Posterior estimates. If a dict, per-key arrays sharing the same leading shape, keyed by variable. If an array, shape (num_sims, num_samples, num_params) directly.
targetsMapping[str, np.ndarray] or np.ndarray: Ground-truth values, matching the input type of estimates. If a dict, per-key arrays. If an array, shape (num_sims, num_params) directly.
variable_keyssequence of str or None, optional, default: None: Which keys to select and plot, and in what order, when estimates/targets are dicts. By default, all keys, in dict insertion order. Ignored for array input.
variable_namessequence of str or None, optional, default: None: Display names for the plotted columns. Defaults to variable_keys (dict input) or param_0, param_1, … (array input).
colorstr, optional, default: “#822621”: Base color for the calibration ECDF lines.
title_fontsizeint, optional, default: 22: The font size of the panel titles.
label_fontsizeint, optional, default: 18: The font size of the axis label texts.
metric_fontsizeint, optional, default: 18: The font size of the displayed calibration metric text.
tick_fontsizeint, optional, default: 16: The font size of the axis tick labels.
**kwargs: Forwarded to bf.diagnostics.plots.calibration_ecdf (e.g. figsize, num_row, num_col).

Returns:

figplt.Figure - the calibration diagnostic figure

Parameters:

estimates (Mapping[str, ndarray] | ndarray)
targets (Mapping[str, ndarray] | ndarray)
variable_keys (Sequence[str] | None)
variable_names (Sequence[str] | None)
color (str)
title_fontsize (int)
label_fontsize (int)
metric_fontsize (int)
tick_fontsize (int)

superstats.diagnostics.plot_joint_prior(local_params, hyper_params, shared_params, param_bounds=None, mixture_names=None, hyper_param_groups=None, marginal=True, color='#356673', title_fontsize=22, tick_fontsize=16, alpha=0.5, figsize=None)[source]#

Plot joint prior diagnostics combining hyperparameter distributions, shared parameter histograms, and time-varying trajectories.

Parameters:

local_paramsdict of np.ndarray, each of shape (num_trajectories, num_steps): Mapping from parameter name to an array of trajectories. Every StochasticTransition parameter is unconditionally added here by JointPrior.sample, so together with shared_params this is the authoritative set of row names.
hyper_paramsdict of np.ndarray: Mapping from hyperparameter name to an array of samples. Keys are f”{param_name}_{hyper_key}”.
shared_paramsdict of np.ndarray: Mapping from parameter name to an array of shared parameter samples.
param_boundsdict or None, optional, default: None: Mapping from parameter name to (lower, upper) y-axis limits.
mixture_namesdict or None, optional, default: None: Mapping from parameter name to a list of component names for mixture weight parameters.
hyper_param_groupsdict or None, optional, default: None: Mapping from each parameter name to the exact list of hyper_params keys it owns. Required to correctly separate rows when one parameter name is a prefix of another at an underscore boundary (e.g. “v_1” and “v_1_2”), since str.startswith cannot disambiguate that case from key strings alone. When provided (e.g. by JointPrior.plot_joint_prior), this is used instead of prefix matching. If omitted, falls back to prefix matching, which can misassign hyperparameters in the presence of such name collisions.
marginalbool, optional, default: True: Whether to draw a marginal KDE panel beside each trajectory panel.
colorstr, optional, default: BASE_COLOR: Base plotting color for KDEs and trajectories.
title_fontsizeint, optional, default: 22: The font size of the panel titles (parameter names).
tick_fontsizeint, optional, default: 16: The font size of the axis tick labels.
alphafloat in [0, 1], optional, default: 0.5: The opacity of individual trajectories.
figsizetuple of two floats or None, optional, default: None: Explicit figure size in inches. If None, the default layout size is used.

Returns:

figplt.Figure - the figure instance for optional saving

Raises:

ValueError: If no plottable parameters are found across local_params, hyper_params, and shared_params.

Parameters:

local_params (Mapping[str, ndarray])
hyper_params (Mapping[str, ndarray])
shared_params (Mapping[str, ndarray])
param_bounds (Mapping[str, tuple[float, float]] | None)
mixture_names (Mapping[str, Sequence[str]] | None)
hyper_param_groups (Mapping[str, Sequence[str]] | None)
marginal (bool)
color (str)
title_fontsize (int)
tick_fontsize (int)
alpha (float)
figsize (tuple[float, float] | None)

Return type:

Figure

superstats.diagnostics.plot_posterior_resimulation(pred_data, real_data, data_dim=0, kind='trajectory', aggregation=None, aggregate_strategy='full_uncertainty', uncertainty_fun='95hdi', smoothing=None, smoothing_window=5, marginal=True, spaghetti=False, num_cols=3, color='#356673', real_color='black', alpha=0.4, label_fontsize=14, tick_fontsize=12, figsize=None, max_discrete_values=30)[source]#

Plot posterior predictive resimulations against the observed data.

Parameters:

pred_datamapping of np.ndarray: Posterior resimulated data, mapping observation names to arrays of shape (num_datasets, num_resims, num_steps).
real_datamapping of np.ndarray: Observed data, mapping observation names to arrays of shape (num_datasets, num_steps).
data_dimint or str, optional, default: 0: Which observation variable to plot. Strings select by key and integers index the predictive mapping’s key order.
kind{“trajectory”, “dist”}, optional, default: “trajectory”: “trajectory”: band/center over steps. “dist”: distribution across steps.
aggregationcallable() or None, optional, default: None: None: one panel per dataset. callable: a single panel aggregated across datasets. Called as aggregation(x, axis=…) (e.g. np.mean, np.median). Also used (instead of a hardcoded median) to collapse resims into a per-dataset representative when aggregate_strategy=”no_epistemic”.
aggregate_strategy{“full_uncertainty”, “no_epistemic”}, optional, default: “full_uncertainty”: Only used when aggregation is not None. “full_uncertainty”: flatten datasets and posterior resims together, then summarize. Captures both epistemic and aleatoric uncertainty. “no_epistemic”: collapse resims to one representative trajectory per dataset first (via aggregation), then aggregate across datasets. Removes epistemic uncertainty.
uncertainty_fun{“std”, “95ci”, “mad”, “95hdi”} or callable() or None, optional, default: “95hdi”: “trajectory” mode only. Function to draw a band around the resimulated center line.
smoothing{“sma”, “ema”} or None, optional, default: None: “trajectory” mode only. Causal (past-only) smoothing applied to the real trajectories and, for resimulated data, to the trajectories that result after aggregate_strategy has pooled resims - i.e. pooling happens on raw data, smoothing is applied afterward, and the center/uncertainty band are computed on the smoothed result.
smoothing_windowint, optional, default: 5: Window size for sma, or span parameter for ema.
marginalbool, optional, default: True: “trajectory” mode only. Attach a marginal KDE panel of the resimulated draws to the right of each trajectory axis.
spaghettibool, optional, default: False: “trajectory” mode only. Per-dataset panels: overlay individual resim draws behind the band. Aggregated panel: overlay each dataset’s own representative trajectory (via aggregation) behind the aggregate band.
num_colsint, optional, default: 3: Number of columns when aggregation is None (per-dataset grid).
colorstr, optional, default: BASE_COLOR: Color for bands / centers / histograms.
real_colorstr, optional, default: “black”: Color for the observed data.
alphafloat in [0, 1], optional, default: 0.4: Alpha for spaghetti lines.
label_fontsizeint, optional, default: 14: The font size of the axis label texts.
tick_fontsizeint, optional, default: 12: The font size of the axis tick labels.
figsizetuple of two floats or None, optional, default: None: Explicit figure size in inches. If None, the default layout size is used.
max_discrete_valuesint, optional, default: 30: “dist” mode, per-dataset panels only. Maximum number of discrete categories to treat the data as discrete.

Returns:

figplt.Figure - the figure instance for optional saving

Raises:

ValueError: If kind is not “trajectory” or “dist”, if pred_data or real_data don’t have the expected shape, if their (num_datasets, num_steps) don’t match, or if aggregate_strategy is not “full_uncertainty” or “no_epistemic”.

Parameters:

pred_data (Mapping[str, ndarray])
real_data (Mapping[str, ndarray])
data_dim (int | str)
kind (Literal['trajectory', 'dist'])
aggregation (Callable | None)
aggregate_strategy (Literal['full_uncertainty', 'no_epistemic'])
uncertainty_fun (Literal['std', '95ci', 'mad', '95hdi'] | ~collections.abc.Callable | None)
smoothing (Literal['sma', 'ema'] | None)
smoothing_window (int)
marginal (bool)
spaghetti (bool)
num_cols (int)
color (str)
real_color (str)
alpha (float)
label_fontsize (int)
tick_fontsize (int)
figsize (tuple[float, float] | None)
max_discrete_values (int)

Return type:

Figure

superstats.diagnostics.plot_push_forward(data, data_dim=0, kind='dist', aggregation=None, uncertainty_fun='95ci', marginal=True, spaghetti=False, alpha=0.5, num_cols=3, color='#356673', title_fontsize=22, label_fontsize=18, tick_fontsize=16, figsize=None, max_discrete_values=30)[source]#

Plot prior push-forward for a single data dimension.

Parameters:

datamapping of np.ndarray: Simulation data from the generative model, mapping observation names to arrays of shape (batch_size, steps).
data_dimint or str, optional, default: 0: Which observation variable to plot. Strings select by key and integers index the mapping’s key order.
kind{“dist”, “trajectory”}, optional, default: “dist”: Plot type: distribution of summary statistics or time-series trajectories.
aggregationcallable() or None, optional, default: None: Aggregation function over the dataset dimension, called as aggregation(x, axis=…) (e.g. np.mean, np.median). If None, individual datasets are shown in separate panels. If specified, all datasets are aggregated into a single panel.
uncertainty_fun{“std”, “95ci”, “mad”, “95hdi”} or callable() or None, optional, default: “95ci”: Uncertainty function. Only used when aggregation is not None and kind is “trajectory”. Ignored (with a warning) otherwise.
marginalbool, optional, default: True: Whether to draw marginal distributions beside trajectory plots.
spaghettibool, optional, default: False: Whether to draw individual trajectories behind the aggregate line.
num_colsint, optional, default: 3: Number of columns when rendering individual panels.
alphafloat in [0, 1], optional, default: 0.5: Alpha value for individual dataset traces.
colorstr, optional, default: “#822621”: Base color for plotted lines and fills.
title_fontsizeint, optional, default: 22: The font size of the panel titles.
label_fontsizeint, optional, default: 18: The font size of the axis label texts.
tick_fontsizeint, optional, default: 16: The font size of the axis tick labels.
figsizetuple of two floats or None, optional, default: None: Explicit figure size in inches. If None, the default layout size is used.
max_discrete_valuesint, optional, default: 30: Maximum number of discrete categories to treat the data as discrete.

Returns:

figplt.Figure - the figure instance for optional saving

Raises:

ValueError: If kind is not “dist” or “trajectory”, or if data has an unsupported shape.

Parameters:

data (Mapping[str, ndarray])
data_dim (int | str)
kind (Literal['trajectory', 'dist'])
aggregation (Callable | None)
uncertainty_fun (Literal['std', '95ci', 'mad', '95hdi'] | ~collections.abc.Callable | None)
marginal (bool)
spaghetti (bool)
alpha (float)
num_cols (int)
color (str)
title_fontsize (int)
label_fontsize (int)
tick_fontsize (int)
figsize (tuple[float, float] | None)
max_discrete_values (int)

superstats.diagnostics.plot_recovery(estimates, targets, variable_keys=None, variable_names=None, color='#356673', title_fontsize=22, label_fontsize=18, metric_fontsize=18, tick_fontsize=16, **kwargs)[source]#

Plot time-invariant parameter recovery.

Thin wrapper around bf.diagnostics.plots.recovery that accepts dict-or-array input via prepare_plot_data, consistent with plot_time_varying_verification.

Parameters:

estimatesMapping[str, np.ndarray] or np.ndarray: Posterior estimates. If a dict, per-key arrays sharing the same leading shape, keyed by variable. If an array, shape (num_sims, num_samples, num_params) directly.
targetsMapping[str, np.ndarray] or np.ndarray: Ground-truth values, matching the input type of estimates. If a dict, per-key arrays. If an array, shape (num_sims, num_params) directly.
variable_keyssequence of str or None, optional, default: None: Which keys to select and plot, and in what order, when estimates/targets are dicts. By default, all keys, in dict insertion order. Ignored for array input.
variable_namessequence of str or None, optional, default: None: Display names for the plotted columns. Defaults to variable_keys (dict input) or param_0, param_1, … (array input).
colorstr, optional, default: “#822621”: Base color for plotted lines and fills.
title_fontsizeint, optional, default: 22: The font size of the panel titles.
label_fontsizeint, optional, default: 18: The font size of the axis label texts.
metric_fontsizeint, optional, default: 18: The font size of the displayed recovery metric text.
tick_fontsizeint, optional, default: 16: The font size of the axis tick labels.
**kwargs: Forwarded to bf.diagnostics.plots.recovery (e.g. figsize, num_row, num_col).

Returns:

figplt.Figure - the recovery diagnostic figure

Parameters:

estimates (Mapping[str, ndarray] | ndarray)
targets (Mapping[str, ndarray] | ndarray)
variable_keys (Sequence[str] | None)
variable_names (Sequence[str] | None)
color (str)
title_fontsize (int)
label_fontsize (int)
metric_fontsize (int)
tick_fontsize (int)

superstats.diagnostics.plot_time_invariant_posterior(estimates, targets=None, variable_keys=None, variable_names=None, aggregation=None, mixture_names=None, num_cols=2, color='#356673', title_fontsize=22, label_fontsize=18, tick_fontsize=16, figsize=None)[source]#

Plot time-invariant parameter posteriors.

Parameters:

estimatesMapping[str, np.ndarray] or np.ndarray: Posterior samples. If a dict, values of shape (num_datasets, num_post_samples, num_steps, num_components), keyed by variable. If an array, shape (num_datasets, num_post_samples, num_steps, num_params) directly - treated as single-component parameters; mixture grouping is not inferable from array input.
targetsMapping[str, np.ndarray], np.ndarray, or None, optional, default: None: Ground-truth values, matching the input type of estimates. If a dict, values of shape (num_datasets, num_components). If an array, shape (num_datasets, num_params) directly. If given, drawn as black dashed vertical lines. When aggregation is None, one solid line per panel marks that panel’s specific dataset’s true value. When aggregation is given, the per-dataset true values are collapsed with aggregation and a single solid line is drawn per panel.
variable_keyssequence of str or None, optional, default: None: Which variables to select and plot, and in what order, when estimates is a dict. By default, all keys, in dict insertion order. Ignored for array input.
variable_namessequence of str or None, optional, default: None: Display names (used for panel labels/titles), in the same order as variable_keys (or the array’s last axis). Defaults to variable_keys for dict input, or param_0, param_1, … for array input.
aggregationcallable() or None, optional, default: None: Controls both the posterior layout and the target summary. If None: one panel per (dataset, parameter) pair; rows=params, cols=datasets, param name as row label, dataset index as column title; targets (if given) are shown per dataset. If a callable (e.g. np.mean, np.median): posterior samples are pooled across datasets into one panel per parameter, arranged in a num_cols-column grid; targets (if given) are collapsed across datasets with aggregation into a single reference value per panel.
mixture_namesdict or None, optional, default: None: Mapping from base parameter name (e.g. “a”, without any “_mixture_weights” suffix) to a list of component names. Defaults to “component 0”, “component 1”, … when not supplied. Only applies to dict input with multi-component values.
num_colsint, optional, default: 2: Number of subplot columns when aggregation is not None.
colorstr, optional, default: BASE_COLOR: Base color for non-mixture parameters.
title_fontsizeint, optional, default: 22: The font size of the panel titles.
label_fontsizeint, optional, default: 18: The font size of the row labels (param names, non-pooled layout only).
tick_fontsizeint, optional, default: 16: The font size of the axis tick labels.
figsizetuple of two floats or None, optional, default: None: Explicit figure size in inches. If None, the default layout size is used.

Returns:

figplt.Figure - the figure instance for optional saving

Raises:

ValueError: If no variables are found to plot (empty variable_keys, whether resolved by default or passed explicitly), or if variable_names doesn’t match the number of variables for array input.

Parameters:

estimates (Mapping[str, ndarray] | ndarray)
targets (Mapping[str, ndarray] | ndarray | None)
variable_keys (Sequence[str] | None)
variable_names (Sequence[str] | None)
aggregation (Callable | None)
mixture_names (dict | None)
num_cols (int)
color (str)
title_fontsize (int)
label_fontsize (int)
tick_fontsize (int)
figsize (tuple[float, float] | None)

Return type:

Figure

superstats.diagnostics.plot_time_invariant_prior(hyper_params, shared_params, mixture_names=None, color='#356673', num_cols=2, title_fontsize=22, label_fontsize=18, tick_fontsize=16, figsize=None)[source]#

Plot time-invariant parameter distributions.

Parameters:

hyper_paramsdict of np.ndarray: Mapping from parameter name to an array of hyperparameter samples.
shared_paramsdict of np.ndarray: Mapping from parameter name to an array of shared parameter samples.
mixture_namesdict or None, optional, default: None: Mapping from parameter name to a list of component names for mixture weight parameters.
colorstr, optional, default: BASE_COLOR: Base color for non-mixture histograms.
num_colsint, optional, default: 2: Number of subplot columns.
title_fontsizeint, optional, default: 22: The font size of the panel titles.
label_fontsizeint, optional, default: 18: The font size of the axis labels.
tick_fontsizeint, optional, default: 16: The font size of the axis tick labels.
figsizetuple of two floats or None, optional, default: None: Explicit figure size in inches. If None, the default layout size is used.

Returns:

figplt.Figure - the figure instance for optional saving

Raises:

ValueError: If both hyper_params and shared_params are empty.

Parameters:

hyper_params (Mapping[str, ndarray])
shared_params (Mapping[str, ndarray])
mixture_names (Mapping[str, Sequence[str]] | None)
color (str)
num_cols (int)
title_fontsize (int)
label_fontsize (int)
tick_fontsize (int)
figsize (tuple[float, float] | None)

Return type:

Figure

superstats.diagnostics.plot_time_varying_posterior(estimates, targets=None, variable_keys=None, variable_names=None, aggregation=None, aggregate_strategy='full_uncertainty', uncertainty_fun='95ci', smoothing=None, smoothing_window=5, marginal=True, num_cols=2, alpha=0.5, color='#356673', title_fontsize=22, label_fontsize=18, tick_fontsize=16, figsize=None)[source]#

Plot time-varying parameter posteriors.

Parameters:

estimatesMapping[str, np.ndarray] or np.ndarray: Posterior samples. If a dict, values of shape (num_datasets, num_post_samples, num_steps, 1), keyed by variable. If an array, shape (num_datasets, num_post_samples, num_steps, num_params) directly.
targetsMapping[str, np.ndarray], np.ndarray, or None, optional, default: None: Ground-truth trajectories, matching the input type of estimates. If a dict, values of shape (num_datasets, num_steps, 1). If an array, shape (num_datasets, num_steps, num_params) directly. If given, drawn as a black dashed line on top of each panel: the raw per-dataset trajectory when aggregation is None, or aggregated across datasets (using aggregation) when aggregation is not None. Smoothed with the same smoothing settings as the posterior trajectories, for a fair visual comparison.
variable_keyssequence of str or None, optional, default: None: Which variables to select and plot, and in what order, when estimates/targets are dicts. By default, all keys, in dict insertion order. Ignored for array input.
variable_namessequence of str or None, optional, default: None: Display names (used for panel labels/titles), in the same order as variable_keys (or the array’s last axis). Defaults to variable_keys for dict input, or param_0, param_1, … for array input.
aggregationcallable() or None, optional, default: None: None: one panel per (param, dataset). callable: one panel per param, aggregated across datasets. Called as aggregation(trajectories, axis=0) and must return a (T,) center. The same function aggregates targets across datasets when both targets and aggregation are given.
aggregate_strategy{“full_uncertainty”, “no_epistemic”}, optional, default: “full_uncertainty”: Only used when aggregation is not None. “full_uncertainty”: flatten datasets and posterior samples, then summarize. “no_epistemic”: median across posterior samples per dataset first, then aggregate.
uncertainty_fun{“std”, “95ci”, “mad”, “95hdi”} or callable() or None, optional, default: “95ci”: Band drawn around the center line. A callable receives (N, T) trajectories and must return (lo, hi), each of shape (T,).
smoothing{“sma”, “ema”} or None, optional, default: None: Applied to each trajectory (and to targets, if given) before computing the center, uncertainty, and marginal.
smoothing_windowint, optional, default: 5: Window size for sma, or span parameter for ema.
marginalbool, optional, default: True: Attach a marginal KDE panel to the right of each trajectory axis. The KDE is computed on the same array used for the uncertainty band.
num_colsint, optional, default: 2: Number of subplot columns when aggregation is not None.
colorstr, optional, default: BASE_COLOR: Line and band color.
alphafloat in [0, 1], optional, default: 0.5: Alpha for the uncertainty band.
title_fontsizeint, optional, default: 22: The font size of the panel titles.
label_fontsizeint, optional, default: 18: The font size of the axis label texts.
tick_fontsizeint, optional, default: 16: The font size of the axis tick labels.
figsizetuple of two floats or None, optional, default: None: Explicit figure size in inches. If None, the default layout size is used.

Returns:

figplt.Figure - the figure instance for optional saving

Raises:

ValueError: If estimates/targets are inconsistent (mismatched dict keys, or a variable_names length mismatch for array input), or if aggregate_strategy, or uncertainty_fun when given as a string, is not one of the recognized values.

Parameters:

estimates (Mapping[str, ndarray] | ndarray)
targets (Mapping[str, ndarray] | ndarray | None)
variable_keys (Sequence[str] | None)
variable_names (Sequence[str] | None)
aggregation (Callable | None)
aggregate_strategy (Literal['full_uncertainty', 'no_epistemic'])
uncertainty_fun (Literal['std', '95ci', 'mad', '95hdi'] | ~collections.abc.Callable | None)
smoothing (Literal['sma', 'ema'] | None)
smoothing_window (int)
marginal (bool)
num_cols (int)
alpha (float)
color (str)
title_fontsize (int)
label_fontsize (int)
tick_fontsize (int)
figsize (tuple[float, float] | None)

Return type:

Figure

superstats.diagnostics.plot_time_varying_prior(local_params, param_bounds=None, num_cols=2, marginal=True, alpha=0.5, color='#356673', title_fontsize=22, label_fontsize=18, tick_fontsize=16, figsize=None)[source]#

Plot time-varying parameter trajectories with marginal KDE.

Parameters:

local_paramsdict of np.ndarray, each of shape (num_trajectories, num_steps): Mapping from parameter name to an array of trajectories.
param_boundsdict or None, optional, default: None: Mapping from parameter name to (lower, upper) y-axis limits.
num_colsint, optional, default: 2: Number of subplot columns.
marginalbool, optional, default: True: Whether to draw a marginal KDE panel beside each trajectory panel.
alphafloat in [0, 1], optional, default: 0.5: The opacity of individual trajectories.
colorstr, optional, default: BASE_COLOR: Line color for individual trajectories and marginal KDE.
title_fontsizeint, optional, default: 22: The font size of the panel titles.
label_fontsizeint, optional, default: 18: The font size of the axis labels.
tick_fontsizeint, optional, default: 16: The font size of the axis tick labels.
figsizetuple of two floats or None, optional, default: None: Explicit figure size in inches. If None, the default layout size is used.

Returns:

figplt.Figure - the figure instance for optional saving

Raises:

ValueError: If local_params is empty.

Parameters:

local_params (Mapping[str, ndarray])
param_bounds (Mapping[str, tuple[float, float]] | None)
num_cols (int)
marginal (bool)
alpha (float)
color (str)
title_fontsize (int)
label_fontsize (int)
tick_fontsize (int)
figsize (tuple[float, float] | None)

Return type:

Figure

superstats.diagnostics.plot_time_varying_verification(estimates, targets, variable_keys=None, variable_names=None, aggregation=<function median>, colors=['#356673', '#AE534C', '#6B4A6E', '#566B54'], title_fontsize=22, label_fontsize=18, tick_fontsize=16, figsize=None)[source]#

Plot recovery diagnostics over steps for time-varying parameters.

Parameters:

estimatesMapping[str, np.ndarray] or np.ndarray: Posterior samples. If a dict, values of shape (num_sim, num_samples, num_steps), keyed by variable. If an array, shape (num_sim, num_samples, num_steps, num_params) directly.
targetsMapping[str, np.ndarray] or np.ndarray: Ground-truth parameter trajectories, matching the input type of estimates. If a dict, values of shape (num_sim, num_steps). If an array, shape (num_sim, num_steps, num_params) directly.
variable_keyssequence of str or None, optional, default: None: Which variables to select and plot, and in what order, when estimates/targets are dicts. By default, all keys, in dict insertion order. Ignored for array input.
variable_namessequence of str or None, optional, default: None: Display names for the plotted columns, in the same order as variable_keys (or the array’s last axis). Defaults to variable_keys for dict input, or param_0, param_1, … for array input.
aggregationcallable(), optional, default: np.median: Aggregation function passed through to each metric (nrmse, contraction, calibration) when collapsing across simulations. Typically np.mean or np.median.
colorsstr or sequence of str, optional, default: METRIC_COLORS: Row colors, one per metric in the fixed order: correlation, nrmse, contraction, calibration. A single str is applied to all four rows.
title_fontsizeint, optional, default: 22: The font size of the column titles (parameter names).
label_fontsizeint, optional, default: 18: The font size of the axis label texts and row labels.
tick_fontsizeint, optional, default: 16: The font size of the axis tick labels.
figsizetuple of two floats or None, optional, default: None: Explicit figure size in inches. If None, the default layout size is used.

Returns:

figplt.Figure - the figure instance for optional saving

Raises:

ValueError: If estimates/targets are inconsistent (see _prepare_plot_data), or if colors is a sequence whose length doesn’t match the number of metrics (4).

Parameters:

estimates (Mapping[str, ndarray] | ndarray)
targets (Mapping[str, ndarray] | ndarray)
variable_keys (Sequence[str] | None)
variable_names (Sequence[str] | None)
aggregation (Callable)
colors (str | Sequence[str])
title_fontsize (int)
label_fontsize (int)
tick_fontsize (int)
figsize (tuple[float, float] | None)

superstats.diagnostics.posterior_contraction_per_step(estimates, targets, aggregation=<function median>)[source]#

Posterior contraction per step and parameter.

Computes 1 minus the ratio of posterior to prior variance (using the unbiased/sample variance, ddof=1) for each simulation, step, and parameter, clipped to [0, 1], then aggregates across simulations (using aggregation) to yield one value per step and parameter. Matches the bayesflow posterior_contraction metric, extended over an additional time-step axis.

Parameters:

estimatesnp.ndarray of shape (num_sim, num_samples, num_steps, num_params): Posterior samples.
targetsnp.ndarray of shape (num_sim, num_steps, num_params): Ground-truth parameter values, used to estimate the prior variance per step and parameter.
aggregationcallable(), optional, default: np.median: Function used to aggregate the per-simulation contraction values across simulations. Typically np.mean or np.median.

Returns:

contractionnp.ndarray of shape (num_steps, num_params) - 1 minus: the ratio of posterior to prior variance, per step and parameter, clipped to [0, 1], aggregated across simulations

Parameters:

estimates (ndarray)
targets (ndarray)
aggregation (Callable)

Return type:

ndarray

Modules

`metrics`	Numerical diagnostics for posterior estimates over time.
`plots`	Plotting functions for priors, posteriors, and recovery diagnostics.