Informative distances and summary statistics
Approximate Bayesian computation (ABC) relies on the efficient comparison of relevant features in simulated and observed data, via distance metrics and potentially summary statistics. Separately, methods have been developed to adaptively scale-normalize the distance metric, and to semi-automatically derive informative, low-dimensional summary statistics.
In the notebook on “Adaptive distances” we demonstrated how distances adjusting weights to normalize scales are beneficial for heterogeneous, including outlier-corrupted, data. However, when parts of the data are uninformative, it is desirable to further concentrate the analysis on informative data points. Various methods have been developed to capture information of data on parameters in a low-dimensional summary statistics representation, see e.g. Blum et al. 2013 for a review. A particular approach constructs summary statistics as outputs of regression models of parameters on data, see the similar work by Fearnhead and Prangle 2012. In this notebook, we illustrate the use of regression methods to construct informative summary statistics and sensitivity distance weights in pyABC.
The notebook can be downloaded here.
[1]:
# install if not done yet
!pip install pyabc[plotly] --quiet
[2]:
import logging
import tempfile
from functools import partial
import matplotlib.pyplot as plt
import numpy as np
import scipy as sp
from IPython.display import SVG, display
import pyabc
from pyabc.distance import *
from pyabc.predictor import *
from pyabc.sumstat import *
from pyabc.util import EventIxs, ParTrafo, dict2arrlabels
pyabc.settings.set_figure_params("pyabc") # for beautified plots
# for debugging
for logger in ["ABC.Distance", "ABC.Predictor", "ABC.Sumstat"]:
logging.getLogger(logger).setLevel(logging.DEBUG)
Simple illustration example
To illustrate informativeness of data points, we consider a simple test problem. It consists of a single model output \(y_1\) informative of parameter \(p_1\), and uninformative model outputs \(y_2\).
[3]:
# problem definition
sigmas = {"p1": 0.1}
def model(p):
return {
"y1": p["p1"] + 1 + sigmas["p1"] * np.random.normal(),
"y2": 2 + 0.1 * np.random.normal(size=3),
}
gt_par = {"p1": 3}
data = {"y1": gt_par["p1"] + 1, "y2": 2 * np.ones(shape=3)}
prior_bounds = {"p1": (0, 10)}
prior = pyabc.Distribution(
**{
key: pyabc.RV("uniform", lb, ub - lb)
for key, (lb, ub) in prior_bounds.items()
},
)
We employ three approaches to perform inference on this problem.
Firstly, we use a distance adaptively scale-normalizing all statistics by their respective in-sample median absolute deviations (MAD), as introduced in the “Adaptive distances” notebook (“L1+Ada.+MAD”).
Secondly, we employ an approach similar to Fearnhead and Prangle 2012, using a linear regression model, trained after 40% of the total sample budget, as summary statistic, with a simple L1 distance (“L1+StatLR”).
Thirdly, we complement the MAD scale-normalizing weights by sensitivity weights, derived via normalized sensitivities of a linear regression model trained similarly to the second approach. This method thus accounts for informativeness by re-weighting of model outputs, without explicitly employing a low-dimensional summary statistics representation.
The approaches use as distance a PNormDistance, AdaptivePNormDistance for scale normalization, or InfoWeightedPNormDistance additionally defining regression-based sensitivity weights. Regression models are derived from the basic Predictor class in the corresponding module. Summary statistics that can be combined with all above distances are derived from the Sumstat class, in particular regression-based ones via PredictorSumstat.
[4]:
# analysis definition
pop_size = 100
total_sims = 3000
fit_sims = 0.4 * total_sims
YPredictor = LinearPredictor
# YPredictor = MLPPredictor
distances = {
"L1+Ada.+MAD": AdaptivePNormDistance(
p=1,
# adaptive scale normalization
scale_function=mad,
),
"L1+StatLR": PNormDistance(
p=1,
# regression-based summary statistics
sumstat=PredictorSumstat(
# regression model used
predictor=YPredictor(),
# when to fit the regression model
fit_ixs=EventIxs(sims=fit_sims),
),
),
"L1+Ada.+MAD+SensiLR": InfoWeightedPNormDistance(
p=1,
# adaptive scale normalization
scale_function=mad,
# regression model used to define sensitivity weights
predictor=YPredictor(),
# when to fit the regression model
fit_info_ixs=EventIxs(sims=fit_sims),
),
}
colors = {distance_id: f"C{i}" for i, distance_id in enumerate(distances)}
ABC.Distance DEBUG: Fit scale ixs: <EventIxs, ts={inf}>
ABC.Sumstat DEBUG: Fit model ixs: <EventIxs, sims=[1200.0]>
ABC.Distance DEBUG: Fit scale ixs: <EventIxs, ts={inf}>
ABC.Distance DEBUG: Fit info ixs: <EventIxs, sims=[1200.0]>
We perform the analysis using all above distance functions and summary statistics. Additionally, below we specify various logging files to capture relevant information for further analysis.
[5]:
# runs
db_file = tempfile.mkstemp(suffix=".db")[1]
scale_log_file = tempfile.mkstemp()[1]
info_log_file = tempfile.mkstemp()[1]
info_sample_log_file = tempfile.mkstemp()[1]
hs = []
for distance_id, distance in distances.items():
print(distance_id)
if isinstance(distance, AdaptivePNormDistance):
distance.scale_log_file = f"{scale_log_file}_{distance_id}.json"
if isinstance(distance, InfoWeightedPNormDistance):
distance.info_log_file = f"{info_log_file}_{distance_id}.json"
distance.info_sample_log_file = f"{info_sample_log_file}_{distance_id}"
abc = pyabc.ABCSMC(model, prior, distance, population_size=pop_size)
h = abc.new(db="sqlite:///" + db_file, observed_sum_stat=data)
abc.run(max_total_nr_simulations=total_sims)
hs.append(h)
ABC.Sampler INFO: Parallelize sampling on 4 processes.
L1+Ada.+MAD
ABC.History INFO: Start <ABCSMC id=1, start_time=2022-03-01 22:24:13>
ABC INFO: Calibration sample t = -1.
ABC.Distance DEBUG: Scale weights[0] = {'y1': 4.1696e-01, 'y2:0': 1.4194e+01, 'y2:1': 1.4542e+01, 'y2:2': 1.1623e+01}
ABC.Population INFO: Recording also rejected particles: True
ABC INFO: t: 0, eps: 4.62946034e+00.
ABC INFO: Accepted: 100 / 189 = 5.2910e-01, ESS: 1.0000e+02.
ABC.Distance DEBUG: Scale weights[1] = {'y1': 4.1132e-01, 'y2:0': 1.5028e+01, 'y2:1': 1.2665e+01, 'y2:2': 1.5191e+01}
ABC INFO: t: 1, eps: 3.50413868e+00.
ABC INFO: Accepted: 100 / 360 = 2.7778e-01, ESS: 8.7731e+01.
ABC.Distance DEBUG: Scale weights[2] = {'y1': 5.5391e-01, 'y2:0': 1.4445e+01, 'y2:1': 1.4380e+01, 'y2:2': 1.3888e+01}
ABC INFO: t: 2, eps: 3.07090591e+00.
ABC INFO: Accepted: 100 / 459 = 2.1786e-01, ESS: 9.1323e+01.
ABC.Distance DEBUG: Scale weights[3] = {'y1': 5.7254e-01, 'y2:0': 1.4952e+01, 'y2:1': 1.4680e+01, 'y2:2': 1.3888e+01}
ABC INFO: t: 3, eps: 2.59302288e+00.
ABC INFO: Accepted: 100 / 522 = 1.9157e-01, ESS: 9.1287e+01.
ABC.Distance DEBUG: Scale weights[4] = {'y1': 8.7880e-01, 'y2:0': 1.5887e+01, 'y2:1': 1.5671e+01, 'y2:2': 1.6205e+01}
ABC INFO: t: 4, eps: 2.60156160e+00.
ABC INFO: Accepted: 100 / 1035 = 9.6618e-02, ESS: 8.0667e+01.
ABC.Distance DEBUG: Scale weights[5] = {'y1': 1.1388e+00, 'y2:0': 1.5340e+01, 'y2:1': 1.5152e+01, 'y2:2': 1.5268e+01}
ABC INFO: t: 5, eps: 2.28604105e+00.
ABC INFO: Accepted: 100 / 1206 = 8.2919e-02, ESS: 9.3241e+01.
ABC.Distance DEBUG: Scale weights[6] = {'y1': 1.3174e+00, 'y2:0': 1.5479e+01, 'y2:1': 1.5188e+01, 'y2:2': 1.5372e+01}
ABC INFO: Stop: Total simulations budget.
ABC.History INFO: Done <ABCSMC id=1, duration=0:00:03.646413, end_time=2022-03-01 22:24:16>
ABC.Sampler INFO: Parallelize sampling on 4 processes.
ABC.History INFO: Start <ABCSMC id=2, start_time=2022-03-01 22:24:16>
ABC INFO: Calibration sample t = -1.
L1+StatLR
ABC.Population INFO: Recording also rejected particles: True
ABC INFO: t: 0, eps: 2.94366175e+00.
ABC INFO: Accepted: 100 / 203 = 4.9261e-01, ESS: 1.0000e+02.
ABC INFO: t: 1, eps: 1.66536767e+00.
ABC INFO: Accepted: 100 / 175 = 5.7143e-01, ESS: 9.9290e+01.
ABC INFO: t: 2, eps: 1.01567662e+00.
ABC INFO: Accepted: 100 / 193 = 5.1813e-01, ESS: 9.9368e+01.
ABC INFO: t: 3, eps: 6.20967359e-01.
ABC INFO: Accepted: 100 / 226 = 4.4248e-01, ESS: 9.9646e+01.
ABC INFO: t: 4, eps: 3.85565817e-01.
ABC INFO: Accepted: 100 / 259 = 3.8610e-01, ESS: 9.5384e+01.
ABC INFO: t: 5, eps: 2.94100337e-01.
ABC INFO: Accepted: 100 / 417 = 2.3981e-01, ESS: 9.7841e+01.
ABC.Predictor INFO: Fitted <LinearPredictor predictor=LinearRegression()> in 0.00s
ABC.Predictor INFO: Pearson correlations: 0.898
ABC.Predictor DEBUG: Linear regression coefficients (n_target, n_feature):
[[ 0.89692715 0.00324766 -0.001746 -0.02353367]]
ABC INFO: t: 6, eps: 2.27047591e-01.
ABC INFO: Accepted: 100 / 404 = 2.4752e-01, ESS: 9.4596e+01.
ABC INFO: t: 7, eps: 1.26808917e-01.
ABC INFO: Accepted: 100 / 571 = 1.7513e-01, ESS: 7.7628e+01.
ABC INFO: t: 8, eps: 7.37053186e-02.
ABC INFO: Accepted: 100 / 933 = 1.0718e-01, ESS: 7.5607e+01.
ABC INFO: Stop: Total simulations budget.
ABC.History INFO: Done <ABCSMC id=2, duration=0:00:04.178861, end_time=2022-03-01 22:24:21>
ABC.Sampler INFO: Parallelize sampling on 4 processes.
ABC.History INFO: Start <ABCSMC id=3, start_time=2022-03-01 22:24:21>
ABC INFO: Calibration sample t = -1.
L1+Ada.+MAD+SensiLR
ABC.Distance DEBUG: Scale weights[0] = {'y1': 3.8736e-01, 'y2:0': 1.3394e+01, 'y2:1': 1.5512e+01, 'y2:2': 1.9479e+01}
ABC.Population INFO: Recording also rejected particles: True
ABC.Population INFO: Recording also rejected particles: True
ABC INFO: t: 0, eps: 4.60879843e+00.
ABC INFO: Accepted: 100 / 222 = 4.5045e-01, ESS: 1.0000e+02.
ABC.Distance DEBUG: Scale weights[1] = {'y1': 3.9304e-01, 'y2:0': 1.4688e+01, 'y2:1': 1.4903e+01, 'y2:2': 1.4876e+01}
ABC INFO: t: 1, eps: 3.22709995e+00.
ABC INFO: Accepted: 100 / 443 = 2.2573e-01, ESS: 9.5363e+01.
ABC.Distance DEBUG: Scale weights[2] = {'y1': 5.2186e-01, 'y2:0': 1.4188e+01, 'y2:1': 1.4905e+01, 'y2:2': 1.3964e+01}
ABC INFO: t: 2, eps: 2.70962535e+00.
ABC INFO: Accepted: 100 / 550 = 1.8182e-01, ESS: 9.1834e+01.
ABC.Distance DEBUG: Scale weights[3] = {'y1': 7.2399e-01, 'y2:0': 1.3541e+01, 'y2:1': 1.4226e+01, 'y2:2': 1.4469e+01}
ABC.Predictor INFO: Fitted <LinearPredictor predictor=LinearRegression()> in 0.00s
ABC.Predictor INFO: Pearson correlations: 0.999
ABC.Predictor DEBUG: Linear regression coefficients (n_target, n_feature):
[[ 0.99835555 0.00106467 -0.00180873 0.00178089]]
ABC.Distance DEBUG: Optimal FD delta: [0.1 0.1 0.1 0.1]
ABC.Distance DEBUG: Info weights[3] = {'y1': 9.9563e-01, 'y2:0': 1.0150e-03, 'y2:1': 1.7085e-03, 'y2:2': 1.6436e-03}
ABC INFO: t: 3, eps: 6.57635045e-01.
ABC INFO: Accepted: 100 / 231 = 4.3290e-01, ESS: 9.9434e+01.
ABC.Distance DEBUG: Scale weights[4] = {'y1': 9.5460e-01, 'y2:0': 1.4551e+01, 'y2:1': 1.6221e+01, 'y2:2': 1.4333e+01}
ABC INFO: t: 4, eps: 4.73707012e-01.
ABC INFO: Accepted: 100 / 176 = 5.6818e-01, ESS: 9.8122e+01.
ABC.Distance DEBUG: Scale weights[5] = {'y1': 2.6570e+00, 'y2:0': 1.4511e+01, 'y2:1': 1.1904e+01, 'y2:2': 1.3900e+01}
ABC INFO: t: 5, eps: 6.22117846e-01.
ABC INFO: Accepted: 100 / 253 = 3.9526e-01, ESS: 9.9050e+01.
ABC.Distance DEBUG: Scale weights[6] = {'y1': 3.6227e+00, 'y2:0': 1.6931e+01, 'y2:1': 1.5359e+01, 'y2:2': 1.6303e+01}
ABC INFO: t: 6, eps: 4.70729296e-01.
ABC INFO: Accepted: 100 / 256 = 3.9062e-01, ESS: 9.7525e+01.
ABC.Distance DEBUG: Scale weights[7] = {'y1': 5.5853e+00, 'y2:0': 1.5594e+01, 'y2:1': 1.7064e+01, 'y2:2': 1.4924e+01}
ABC INFO: t: 7, eps: 3.40709003e-01.
ABC INFO: Accepted: 100 / 349 = 2.8653e-01, ESS: 8.1211e+01.
ABC.Distance DEBUG: Scale weights[8] = {'y1': 9.2987e+00, 'y2:0': 1.4565e+01, 'y2:1': 1.4281e+01, 'y2:2': 1.4820e+01}
ABC INFO: t: 8, eps: 2.65949576e-01.
ABC INFO: Accepted: 100 / 708 = 1.4124e-01, ESS: 9.2487e+01.
ABC.Distance DEBUG: Scale weights[9] = {'y1': 9.6240e+00, 'y2:0': 1.4199e+01, 'y2:1': 1.4983e+01, 'y2:2': 1.4280e+01}
ABC INFO: Stop: Total simulations budget.
ABC.History INFO: Done <ABCSMC id=3, duration=0:00:04.815952, end_time=2022-03-01 22:24:26>
The comparison of the obtained posterior approximations with the true posterior reveals that L1+Ada.+MAD gave a worse fit compared to the other approaches. This is because it only applies scale-normalization, but does not account for informativeness of data, and thus spends a lot of time on fitting \(y_2\).
[6]:
# plot ABC posterior approximations
fig, axes = plt.subplots(ncols=len(prior_bounds), figsize=(6, 4))
if len(prior_bounds) == 1:
axes = [axes]
# plot ground truth
def unnorm_1d_normal_pdf(p, y_obs, sigma, p_to_y=None):
"""Non-normalized 1-d normal density.
Parameters
----------
p: Parameter to evaluate at.
y_obs: Observed data.
sigma: Noise standard deviation.
p_to_y: Function to deterministically transform p to simulated data.
Returns
-------
pd: Probability density/densities at p.
"""
if p_to_y is None:
p_to_y = lambda p: p
y = p_to_y(p)
pd = np.exp(-((y - y_obs) ** 2) / (2 * sigma**2))
return pd
for i_par, par in enumerate(gt_par.keys()):
# define parameter-simulation transformation
p_to_y = lambda p: p + 1
# observed data corresponding to parameter
y_obs = p_to_y(gt_par[par])
# bounds
xmin, xmax = prior_bounds[par]
# standard deviation
sigma = sigmas[par]
# pdf as function of only p
pdf = partial(
unnorm_1d_normal_pdf,
y_obs=y_obs,
sigma=sigma,
p_to_y=p_to_y,
)
# integrate density
norm = sp.integrate.quad(pdf, xmin, xmax)[0]
# plot density
xs = np.linspace(xmin, xmax, 300)
axes[i_par].plot(
xs,
pdf(xs) / norm,
linestyle="dashed",
color="grey",
label="ground truth",
)
# plot ABC approximations
for i_par, par in enumerate(prior_bounds.keys()):
for distance_id, h in zip(distances.keys(), hs):
pyabc.visualization.plot_kde_1d_highlevel(
h,
x=par,
xname=par,
xmin=prior_bounds[par][0],
xmax=prior_bounds[par][1],
ax=axes[i_par],
label=distance_id,
numx=500,
)
# prettify
for ax in axes[1:]:
ax.set_ylabel(None)
fig.tight_layout(rect=(0, 0.1, 1, 1))
axes[-1].legend()
[6]:
<matplotlib.legend.Legend at 0x7f51d3e90040>
Via the log files, we can further examine the employed weights, firstly scale-normalizing weights based on MAD, and secondly sensitivity weights quantifying informativeness. Indeed, while both L1+Ada.+MAD and L1+Ada.+MAD+SensiLR assign large weights to \(y_2\), the additional sensitivity weights employed by L1+Ada.+MAD+SensiLR counteract this by assigning a large weight to \(y_1\).
[7]:
# plot weights
fig, axes = plt.subplots(nrows=2, ncols=len(gt_par), figsize=(4, 8))
# scale weights
scale_distance_ids = [
distance_id
for distance_id in distances.keys()
if "Ada." in distance_id and "Stat" not in distance_id
]
scale_log_files = []
for i_dist, distance_id in enumerate(scale_distance_ids):
scale_log_files.append(f"{scale_log_file}_{distance_id}.json")
pyabc.visualization.plot_distance_weights(
scale_log_files,
labels=scale_distance_ids,
colors=[colors[distance_id] for distance_id in scale_distance_ids],
xlabel="Model output",
title="Scale weights",
ax=axes[0],
keys=dict2arrlabels(data, keys=data.keys()),
)
# info weights
info_distance_ids = [
distance_id for distance_id in distances.keys() if "Sensi" in distance_id
]
info_log_files = []
for i_dist, distance_id in enumerate(info_distance_ids):
info_log_files.append(f"{info_log_file}_{distance_id}.json")
pyabc.visualization.plot_distance_weights(
info_log_files,
labels=info_distance_ids,
colors=[colors[distance_id] for distance_id in info_distance_ids],
xlabel="Model output",
title="Sensitivity weights",
ax=axes[1],
keys=dict2arrlabels(data, keys=data.keys()),
)
fig.tight_layout()
To further understand the employed sensitivity matrix, we can visualize the connections between model outputs and parameters (in this case a single one) via a “Sankey” flow diagram. In this case, this gives no further information beyond the above weight diagram.
[8]:
# plot flow diagram
fig = pyabc.visualization.plot_sensitivity_sankey(
info_sample_log_file=f"{info_sample_log_file}_L1+Ada.+MAD+SensiLR",
t=f"{info_log_file}_L1+Ada.+MAD+SensiLR.json",
h=hs[-1],
predictor=LinearPredictor(),
)
# here just showing a non-interactive plot to reduce storage
img_file = tempfile.mkstemp(suffix=(".svg"))[1]
fig.write_image(img_file)
display(SVG(img_file))
ABC.Predictor INFO: Fitted <LinearPredictor predictor=LinearRegression()> in 0.00s
ABC.Predictor INFO: Pearson correlations: 0.999
ABC.Predictor DEBUG: Linear regression coefficients (n_target, n_feature):
[[ 0.99835555 0.00106467 -0.00180873 0.00178089]]
ABC.Distance DEBUG: Optimal FD delta: [0.1 0.1 0.1 0.1]
Challenging problem features
Now, we turn to a slightly more challenging problem:
\(y_1\sim\mathcal{N}(\theta_1,0.1^2)\) is informative of \(\theta_1\), with a relatively wide corresponding prior \(\theta_1\sim U[-7, 7]\),
\(y_2\sim\mathcal{N}(\theta_2,100^2)\) is informative of \(\theta_2\), with corresponding prior \(\theta_2\sim U[-700, 700]\),
\(y_3\sim\mathcal{N}(\theta_3, 4 \cdot 100^2)^{\otimes 4}\) is a four-dimensional vector informative of \(\theta_3\), with corresponding prior \(\theta_3\sim U[-700, 700]\),
\(y_4\sim\mathcal{N}(\theta_4^2, 0.1^2)\) is informative of \(\theta_4\), with corresponding symmetric prior \(\theta_4\sim U[-1, 1]\), however is quadratic in the parameter, resulting in a bimodal posterior distribution for \(y_{\text{obs},4}\neq 0\),
\(y_5\sim\mathcal{N}(0, 10)^{\otimes 10}\) is an uninformative 10-dimensional vector.
The problem encompasses multiple challenging features established methods have problems with:
A substantial part of the data, \(y_5\), is uninformative, such that approaches not accounting for informativeness of data may converge slower.
Both data and parameters are on different scales, such that approaches comparing data, or, via regression-based summary statistics, parameters, without normalization may be biased towards large-scale variables. Further, e.g. the prior of \(\theta_1\) is relatively wide, such that pre-calibrated weighting is sub-optimal, as discussed in \citet{Prangle2017}.
\(y_4\) is quadratic in \(\theta_4\) and symmetric over the prior, such that the inverse first-order regression approaches as in FearnheadPra2012} cannot capture a meaningful relationship.
While the distributions of \(y_3\) and \(y_4\) are such that the posterior distributions of \(\theta_3\), \(\theta_4\) are identical, in solely scale-normalized approaches such as Fearnhead and Prangle 2012, the impact of \(y_4\) on the distance value is roughly four times as high as that of \(y_3\), leading to potentially uneven convergence.
[9]:
# problem definition
sigmas = {"p1": 1e-1, "p2": 1e2, "p3": 1e2, "p4": 1e-1}
def model(p):
return {
"y1": p["p1"] + sigmas["p1"] * np.random.normal(),
"y2": p["p2"] + sigmas["p2"] * np.random.normal(),
"y3": p["p3"]
+ np.sqrt(4 * sigmas["p3"] ** 2) * np.random.normal(size=4),
"y4": p["p4"] ** 2 + sigmas["p4"] * np.random.normal(),
"y5": 1e1 * np.random.normal(size=10),
}
prior_bounds = {
"p1": (-7e0, 7e0),
"p2": (-7e2, 7e2),
"p3": (-7e2, 7e2),
"p4": (-1e0, 1e0),
}
prior = pyabc.Distribution(
**{
key: pyabc.RV("uniform", lb, ub - lb)
for key, (lb, ub) in prior_bounds.items()
},
)
gt_par = {"p1": 0, "p2": 0, "p3": 0, "p4": 0.5}
data = {
"y1": 0,
"y2": 0,
"y3": 0 * np.ones(4),
"y4": 0.5**2,
"y5": 0 * np.ones(10),
}
To tackle these problems, we suggest to firstly consistently employ scale normalization, both on the raw model outputs and on the level of summary statistics. Secondly, we suggest to instead of only inferring a mapping \(s: y \mapsto \theta\), we target augmented parameter vectors, \(s: y \mapsto \lambda(\theta)\), with e.g. \(\lambda(\theta) = (\theta^1,\ldots,\theta^4)\). This practically allows to break symmetry, e.g. if only \(\theta^2\) can be expressed as a function of the data. Conceptually, this further allows to obtain a more accurate description of the posterior distribution, as the summary statistics may be regarded as approximations to \(s(y) = \mathbb{E}[\lambda(\theta)|y]\), using which as summary statistics preserves the corresponding posterior moments, i.e.
\[\lim_{\varepsilon\rightarrow 0}\mathbb{E}_{\pi_{\text{ABC},\varepsilon}}[\lambda(\Theta)|s(y_\text{obs})] = \mathbb{E}[\lambda(\Theta)|Y=y_\text{obs}].\]
Methods employing scale normalization, accounting for informativeness, and augmented regression targets, are L1+Ada.+MAD+StatLR+P4, which uses regression-based summary statistics, and L1+Ada.+MAD+SensiLR+P4, which uses sensitivity weights. For comparison, we consider L1+Ada.+MAD only normalizing scales, and L1+StatLR, using non-scale normalised summary statistics, as well as L1+Ada.+MAD+StatLR and L1+Ada.+MAD+SensiLR using only a subset of methods.
[10]:
# analysis definition
pop_size = 1000
total_sims = 100000
par_trafos = [
lambda x: x,
lambda x: x**2,
lambda x: x**3,
lambda x: x**4,
]
trafo_ids = ["{par_id}^" + str(i + 1) for i in range(4)]
fit_sims = 0.4 * total_sims
YPredictor = LinearPredictor
# YPredictor = MLPPredictor
distances = {
"L1+Ada.+MAD": AdaptivePNormDistance(
p=1,
scale_function=mad,
),
"L1+StatLR": PNormDistance(
p=1,
sumstat=PredictorSumstat(
predictor=YPredictor(
normalize_features=False, normalize_labels=False
),
fit_ixs=EventIxs(sims=fit_sims),
),
),
"L1+Ada.+MAD+StatLR": AdaptivePNormDistance(
p=1,
scale_function=mad,
sumstat=PredictorSumstat(
predictor=YPredictor(),
fit_ixs=EventIxs(sims=fit_sims),
),
),
"L1+Ada.+MAD+StatLR+P4": AdaptivePNormDistance(
p=1,
scale_function=mad,
sumstat=PredictorSumstat(
predictor=YPredictor(),
fit_ixs=EventIxs(sims=fit_sims),
par_trafo=ParTrafo(trafos=par_trafos, trafo_ids=trafo_ids),
),
),
"L1+Ada.+MAD+SensiLR": InfoWeightedPNormDistance(
p=1,
scale_function=mad,
predictor=YPredictor(),
fit_info_ixs=EventIxs(sims=fit_sims),
feature_normalization="mad",
),
"L1+Ada.+MAD+SensiLR+P4": InfoWeightedPNormDistance(
p=1,
scale_function=mad,
predictor=YPredictor(),
fit_info_ixs=EventIxs(sims=fit_sims),
feature_normalization="mad",
par_trafo=ParTrafo(trafos=par_trafos, trafo_ids=trafo_ids),
),
}
colors = {distance_id: f"C{i}" for i, distance_id in enumerate(distances)}
ABC.Distance DEBUG: Fit scale ixs: <EventIxs, ts={inf}>
ABC.Sumstat DEBUG: Fit model ixs: <EventIxs, sims=[40000.0]>
ABC.Sumstat DEBUG: Fit model ixs: <EventIxs, sims=[40000.0]>
ABC.Distance DEBUG: Fit scale ixs: <EventIxs, ts={inf}>
ABC.Sumstat DEBUG: Fit model ixs: <EventIxs, sims=[40000.0]>
ABC.Distance DEBUG: Fit scale ixs: <EventIxs, ts={inf}>
ABC.Distance DEBUG: Fit scale ixs: <EventIxs, ts={inf}>
ABC.Distance DEBUG: Fit info ixs: <EventIxs, sims=[40000.0]>
ABC.Distance DEBUG: Fit scale ixs: <EventIxs, ts={inf}>
ABC.Distance DEBUG: Fit info ixs: <EventIxs, sims=[40000.0]>
For the analysis, we suggest the use of sufficiently large population sizes, as the process model is more complex.
[11]:
%%time
# runs
db_file = tempfile.mkstemp(suffix=".db")[1]
scale_log_file = tempfile.mkstemp()[1]
info_log_file = tempfile.mkstemp()[1]
info_sample_log_file = tempfile.mkstemp()[1]
hs = []
for distance_id, distance in distances.items():
print(distance_id)
if isinstance(distance, AdaptivePNormDistance):
distance.scale_log_file = f"{scale_log_file}_{distance_id}.json"
if isinstance(distance, InfoWeightedPNormDistance):
distance.info_log_file = f"{info_log_file}_{distance_id}.json"
distance.info_sample_log_file = f"{info_sample_log_file}_{distance_id}"
abc = pyabc.ABCSMC(model, prior, distance, population_size=pop_size)
h = abc.new(db="sqlite:///" + db_file, observed_sum_stat=data)
abc.run(max_total_nr_simulations=total_sims)
hs.append(h)
ABC.Sampler INFO: Parallelize sampling on 4 processes.
ABC.History INFO: Start <ABCSMC id=1, start_time=2022-03-01 22:24:27>
ABC INFO: Calibration sample t = -1.
L1+Ada.+MAD
ABC.Distance DEBUG: Scale weights[0] = {'y1': 2.9903e-01, 'y2': 2.8508e-03, 'y3:0': 2.8734e-03, 'y3:1': 2.7775e-03, 'y3:2': 2.8285e-03, 'y3:3': 2.8304e-03, 'y4': 4.9034e+00, 'y5:0': 1.4995e-01, 'y5:1': 1.5096e-01, 'y5:2': 1.4396e-01, 'y5:3': 1.5448e-01, 'y5:4': 1.5084e-01, 'y5:5': 1.5618e-01, 'y5:6': 1.5170e-01, 'y5:7': 1.5049e-01, 'y5:8': 1.5569e-01, 'y5:9': 1.4825e-01}
ABC.Population INFO: Recording also rejected particles: True
ABC INFO: t: 0, eps: 1.94111297e+01.
ABC INFO: Accepted: 1000 / 2166 = 4.6168e-01, ESS: 1.0000e+03.
ABC.Distance DEBUG: Scale weights[1] = {'y1': 2.7884e-01, 'y2': 2.8670e-03, 'y3:0': 2.7673e-03, 'y3:1': 2.7863e-03, 'y3:2': 2.6965e-03, 'y3:3': 2.8352e-03, 'y4': 4.6091e+00, 'y5:0': 1.4676e-01, 'y5:1': 1.5150e-01, 'y5:2': 1.5204e-01, 'y5:3': 1.4841e-01, 'y5:4': 1.5419e-01, 'y5:5': 1.4403e-01, 'y5:6': 1.4776e-01, 'y5:7': 1.5315e-01, 'y5:8': 1.4297e-01, 'y5:9': 1.4573e-01}
ABC INFO: t: 1, eps: 1.64619227e+01.
ABC INFO: Accepted: 1000 / 3039 = 3.2906e-01, ESS: 7.8425e+02.
ABC.Distance DEBUG: Scale weights[2] = {'y1': 3.4183e-01, 'y2': 3.3534e-03, 'y3:0': 3.5982e-03, 'y3:1': 3.6601e-03, 'y3:2': 3.5481e-03, 'y3:3': 3.5396e-03, 'y4': 5.6202e+00, 'y5:0': 1.4201e-01, 'y5:1': 1.5040e-01, 'y5:2': 1.4600e-01, 'y5:3': 1.4591e-01, 'y5:4': 1.5085e-01, 'y5:5': 1.4724e-01, 'y5:6': 1.4962e-01, 'y5:7': 1.5093e-01, 'y5:8': 1.4958e-01, 'y5:9': 1.4961e-01}
ABC INFO: t: 2, eps: 1.61270195e+01.
ABC INFO: Accepted: 1000 / 4442 = 2.2512e-01, ESS: 6.6259e+02.
ABC.Distance DEBUG: Scale weights[3] = {'y1': 3.6011e-01, 'y2': 3.4257e-03, 'y3:0': 4.1717e-03, 'y3:1': 4.2160e-03, 'y3:2': 4.1158e-03, 'y3:3': 4.2058e-03, 'y4': 6.0246e+00, 'y5:0': 1.4671e-01, 'y5:1': 1.4989e-01, 'y5:2': 1.4962e-01, 'y5:3': 1.4915e-01, 'y5:4': 1.4510e-01, 'y5:5': 1.4583e-01, 'y5:6': 1.5134e-01, 'y5:7': 1.4732e-01, 'y5:8': 1.5020e-01, 'y5:9': 1.4796e-01}
ABC INFO: t: 3, eps: 1.54480347e+01.
ABC INFO: Accepted: 1000 / 6222 = 1.6072e-01, ESS: 6.2204e+02.
ABC.Distance DEBUG: Scale weights[4] = {'y1': 3.9063e-01, 'y2': 3.7227e-03, 'y3:0': 4.8907e-03, 'y3:1': 4.9644e-03, 'y3:2': 4.9102e-03, 'y3:3': 4.9302e-03, 'y4': 6.2074e+00, 'y5:0': 1.5319e-01, 'y5:1': 1.5103e-01, 'y5:2': 1.4754e-01, 'y5:3': 1.5241e-01, 'y5:4': 1.5008e-01, 'y5:5': 1.4875e-01, 'y5:6': 1.4883e-01, 'y5:7': 1.5209e-01, 'y5:8': 1.4942e-01, 'y5:9': 1.5341e-01}
ABC INFO: t: 4, eps: 1.51452899e+01.
ABC INFO: Accepted: 1000 / 10680 = 9.3633e-02, ESS: 6.0607e+02.
ABC.Distance DEBUG: Scale weights[5] = {'y1': 4.1023e-01, 'y2': 3.8337e-03, 'y3:0': 5.2679e-03, 'y3:1': 5.2907e-03, 'y3:2': 5.2068e-03, 'y3:3': 5.3311e-03, 'y4': 6.1898e+00, 'y5:0': 1.4419e-01, 'y5:1': 1.4795e-01, 'y5:2': 1.4716e-01, 'y5:3': 1.4681e-01, 'y5:4': 1.5100e-01, 'y5:5': 1.4792e-01, 'y5:6': 1.4748e-01, 'y5:7': 1.4848e-01, 'y5:8': 1.4408e-01, 'y5:9': 1.4874e-01}
ABC INFO: t: 5, eps: 1.42231150e+01.
ABC INFO: Accepted: 1000 / 15273 = 6.5475e-02, ESS: 5.8159e+02.
ABC.Distance DEBUG: Scale weights[6] = {'y1': 4.3083e-01, 'y2': 4.0665e-03, 'y3:0': 5.7448e-03, 'y3:1': 5.6721e-03, 'y3:2': 5.6564e-03, 'y3:3': 5.6503e-03, 'y4': 6.1348e+00, 'y5:0': 1.4929e-01, 'y5:1': 1.4905e-01, 'y5:2': 1.4909e-01, 'y5:3': 1.4710e-01, 'y5:4': 1.4997e-01, 'y5:5': 1.4902e-01, 'y5:6': 1.4838e-01, 'y5:7': 1.4795e-01, 'y5:8': 1.4877e-01, 'y5:9': 1.4908e-01}
ABC INFO: t: 6, eps: 1.36601651e+01.
ABC INFO: Accepted: 1000 / 25872 = 3.8652e-02, ESS: 2.2490e+02.
ABC.Distance DEBUG: Scale weights[7] = {'y1': 4.6938e-01, 'y2': 4.2968e-03, 'y3:0': 5.9625e-03, 'y3:1': 5.9273e-03, 'y3:2': 5.9491e-03, 'y3:3': 5.9452e-03, 'y4': 6.3012e+00, 'y5:0': 1.4817e-01, 'y5:1': 1.4906e-01, 'y5:2': 1.4856e-01, 'y5:3': 1.4769e-01, 'y5:4': 1.4862e-01, 'y5:5': 1.4796e-01, 'y5:6': 1.4879e-01, 'y5:7': 1.4898e-01, 'y5:8': 1.4958e-01, 'y5:9': 1.4840e-01}
ABC INFO: t: 7, eps: 1.30550884e+01.
ABC INFO: Accepted: 1000 / 46778 = 2.1378e-02, ESS: 2.6316e+02.
ABC.Distance DEBUG: Scale weights[8] = {'y1': 4.9279e-01, 'y2': 4.9823e-03, 'y3:0': 5.5964e-03, 'y3:1': 5.5968e-03, 'y3:2': 5.6035e-03, 'y3:3': 5.5883e-03, 'y4': 6.0601e+00, 'y5:0': 1.4959e-01, 'y5:1': 1.4996e-01, 'y5:2': 1.4843e-01, 'y5:3': 1.4722e-01, 'y5:4': 1.4830e-01, 'y5:5': 1.4732e-01, 'y5:6': 1.4858e-01, 'y5:7': 1.4828e-01, 'y5:8': 1.4909e-01, 'y5:9': 1.4880e-01}
ABC INFO: Stop: Total simulations budget.
ABC.History INFO: Done <ABCSMC id=1, duration=0:01:38.270476, end_time=2022-03-01 22:26:06>
ABC.Sampler INFO: Parallelize sampling on 4 processes.
ABC.History INFO: Start <ABCSMC id=2, start_time=2022-03-01 22:26:06>
ABC INFO: Calibration sample t = -1.
L1+StatLR
ABC.Population INFO: Recording also rejected particles: True
ABC INFO: t: 0, eps: 1.90073547e+03.
ABC INFO: Accepted: 1000 / 1939 = 5.1573e-01, ESS: 1.0000e+03.
ABC INFO: t: 1, eps: 1.31242141e+03.
ABC INFO: Accepted: 1000 / 2229 = 4.4863e-01, ESS: 8.6586e+02.
ABC INFO: t: 2, eps: 1.04364953e+03.
ABC INFO: Accepted: 1000 / 2866 = 3.4892e-01, ESS: 7.3923e+02.
ABC INFO: t: 3, eps: 8.61095576e+02.
ABC INFO: Accepted: 1000 / 4138 = 2.4166e-01, ESS: 7.2489e+02.
ABC INFO: t: 4, eps: 7.36415669e+02.
ABC INFO: Accepted: 1000 / 5928 = 1.6869e-01, ESS: 5.7328e+02.
ABC INFO: t: 5, eps: 6.22306047e+02.
ABC INFO: Accepted: 1000 / 9777 = 1.0228e-01, ESS: 4.8916e+02.
ABC INFO: t: 6, eps: 5.51735039e+02.
ABC INFO: Accepted: 1000 / 15557 = 6.4280e-02, ESS: 2.4570e+02.
ABC.Predictor INFO: Fitted <LinearPredictor predictor=LinearRegression() normalize_features=False normalize_labels=False> in 0.01s
ABC.Predictor INFO: Pearson correlations: 1.000 0.900 0.767 0.111
ABC.Predictor DEBUG: Linear regression coefficients (n_target, n_feature):
[[ 9.99432199e-01 9.39633441e-06 9.55159460e-06 -7.59615180e-06
2.92146205e-06 5.71472629e-06 -1.09706500e-03 2.70151331e-05
-1.71054160e-04 -4.64265954e-05 -7.72835999e-05 3.90842094e-05
1.06955781e-04 -3.84259920e-06 7.54826153e-05 1.65503443e-05
4.34150663e-06]
[ 1.25998627e+00 8.01458146e-01 3.83746278e-03 5.33655187e-03
7.40670980e-03 2.80437140e-04 -1.39199692e+00 -2.16441449e-03
2.81230580e-02 3.72724481e-02 1.43726173e-02 -1.09845403e-01
-7.05272626e-02 1.09475130e-02 3.09149659e-02 3.17397280e-02
-2.91388764e-02]
[-5.03069382e-01 1.03216101e-02 1.46573794e-01 1.47547322e-01
1.47889552e-01 1.44883701e-01 -1.54612660e+00 -1.63031991e-01
5.57474815e-02 -7.45239703e-02 1.23409469e-01 1.09327576e-03
-5.35254693e-02 6.22323435e-02 -9.21821712e-03 1.04238807e-02
1.26242483e-01]
[ 4.80207377e-03 7.42641330e-05 -8.54115598e-05 -9.35763651e-05
-8.28161305e-05 -6.21100196e-05 -4.07461891e-02 -4.95200504e-05
-1.66255825e-04 2.99811623e-04 6.75650937e-04 -3.26470949e-04
4.94428214e-04 -1.79189483e-05 6.36720992e-04 -4.86577427e-04
-9.84227938e-04]]
ABC INFO: t: 7, eps: 9.03589291e+01.
ABC INFO: Accepted: 1000 / 6257 = 1.5982e-01, ESS: 1.5990e+02.
ABC INFO: t: 8, eps: 6.61361600e+01.
ABC INFO: Accepted: 1000 / 10706 = 9.3406e-02, ESS: 5.3293e+02.
ABC INFO: t: 9, eps: 4.76701345e+01.
ABC INFO: Accepted: 1000 / 17514 = 5.7097e-02, ESS: 1.2118e+02.
ABC INFO: t: 10, eps: 3.27874437e+01.
ABC INFO: Accepted: 1000 / 41389 = 2.4161e-02, ESS: 5.9247e+02.
ABC INFO: Stop: Total simulations budget.
ABC.History INFO: Done <ABCSMC id=2, duration=0:02:02.998217, end_time=2022-03-01 22:28:09>
ABC.Sampler INFO: Parallelize sampling on 4 processes.
ABC.History INFO: Start <ABCSMC id=3, start_time=2022-03-01 22:28:09>
ABC INFO: Calibration sample t = -1.
L1+Ada.+MAD+StatLR
ABC.Distance DEBUG: Scale weights[0] = {'y1': 2.7740e-01, 'y2': 2.9350e-03, 'y3:0': 2.9481e-03, 'y3:1': 2.8059e-03, 'y3:2': 2.6870e-03, 'y3:3': 2.5650e-03, 'y4': 4.4857e+00, 'y5:0': 1.5030e-01, 'y5:1': 1.5961e-01, 'y5:2': 1.4499e-01, 'y5:3': 1.4444e-01, 'y5:4': 1.5222e-01, 'y5:5': 1.6019e-01, 'y5:6': 1.3794e-01, 'y5:7': 1.4617e-01, 'y5:8': 1.4959e-01, 'y5:9': 1.4944e-01}
ABC.Population INFO: Recording also rejected particles: True
ABC.Population INFO: Recording also rejected particles: True
ABC INFO: t: 0, eps: 1.90890472e+01.
ABC INFO: Accepted: 1000 / 2011 = 4.9727e-01, ESS: 1.0000e+03.
ABC.Distance DEBUG: Scale weights[1] = {'y1': 2.8655e-01, 'y2': 2.9998e-03, 'y3:0': 2.8583e-03, 'y3:1': 2.8170e-03, 'y3:2': 2.8404e-03, 'y3:3': 2.8071e-03, 'y4': 4.3381e+00, 'y5:0': 1.4700e-01, 'y5:1': 1.5099e-01, 'y5:2': 1.4352e-01, 'y5:3': 1.5116e-01, 'y5:4': 1.5607e-01, 'y5:5': 1.4533e-01, 'y5:6': 1.5055e-01, 'y5:7': 1.5209e-01, 'y5:8': 1.4187e-01, 'y5:9': 1.4566e-01}
ABC INFO: t: 1, eps: 1.65554756e+01.
ABC INFO: Accepted: 1000 / 2976 = 3.3602e-01, ESS: 7.3306e+02.
ABC.Distance DEBUG: Scale weights[2] = {'y1': 3.4367e-01, 'y2': 3.4848e-03, 'y3:0': 3.6549e-03, 'y3:1': 3.7046e-03, 'y3:2': 3.7519e-03, 'y3:3': 3.7943e-03, 'y4': 5.7393e+00, 'y5:0': 1.4871e-01, 'y5:1': 1.5122e-01, 'y5:2': 1.4834e-01, 'y5:3': 1.4695e-01, 'y5:4': 1.4520e-01, 'y5:5': 1.4712e-01, 'y5:6': 1.4338e-01, 'y5:7': 1.5197e-01, 'y5:8': 1.5195e-01, 'y5:9': 1.4897e-01}
ABC INFO: t: 2, eps: 1.62761321e+01.
ABC INFO: Accepted: 1000 / 4484 = 2.2302e-01, ESS: 7.1380e+02.
ABC.Distance DEBUG: Scale weights[3] = {'y1': 3.6275e-01, 'y2': 3.5032e-03, 'y3:0': 4.2611e-03, 'y3:1': 4.4184e-03, 'y3:2': 4.3637e-03, 'y3:3': 4.2507e-03, 'y4': 5.9359e+00, 'y5:0': 1.4773e-01, 'y5:1': 1.5038e-01, 'y5:2': 1.5235e-01, 'y5:3': 1.5143e-01, 'y5:4': 1.5439e-01, 'y5:5': 1.4706e-01, 'y5:6': 1.4729e-01, 'y5:7': 1.4911e-01, 'y5:8': 1.4508e-01, 'y5:9': 1.5073e-01}
ABC INFO: t: 3, eps: 1.56618757e+01.
ABC INFO: Accepted: 1000 / 6350 = 1.5748e-01, ESS: 7.3764e+02.
ABC.Distance DEBUG: Scale weights[4] = {'y1': 3.6942e-01, 'y2': 3.6436e-03, 'y3:0': 4.8543e-03, 'y3:1': 4.9473e-03, 'y3:2': 4.9029e-03, 'y3:3': 4.8655e-03, 'y4': 6.1607e+00, 'y5:0': 1.5114e-01, 'y5:1': 1.4759e-01, 'y5:2': 1.5429e-01, 'y5:3': 1.4943e-01, 'y5:4': 1.4672e-01, 'y5:5': 1.4716e-01, 'y5:6': 1.5130e-01, 'y5:7': 1.5001e-01, 'y5:8': 1.4950e-01, 'y5:9': 1.4723e-01}
ABC INFO: t: 4, eps: 1.48050303e+01.
ABC INFO: Accepted: 1000 / 10622 = 9.4144e-02, ESS: 5.9561e+02.
ABC.Distance DEBUG: Scale weights[5] = {'y1': 4.0584e-01, 'y2': 4.0947e-03, 'y3:0': 5.3349e-03, 'y3:1': 5.3433e-03, 'y3:2': 5.4532e-03, 'y3:3': 5.2179e-03, 'y4': 6.3040e+00, 'y5:0': 1.4880e-01, 'y5:1': 1.5016e-01, 'y5:2': 1.4834e-01, 'y5:3': 1.4874e-01, 'y5:4': 1.4767e-01, 'y5:5': 1.4637e-01, 'y5:6': 1.5000e-01, 'y5:7': 1.4701e-01, 'y5:8': 1.4916e-01, 'y5:9': 1.4870e-01}
ABC INFO: t: 5, eps: 1.41383700e+01.
ABC INFO: Accepted: 1000 / 17487 = 5.7185e-02, ESS: 1.9548e+02.
ABC.Predictor INFO: Fitted <LinearPredictor predictor=LinearRegression()> in 0.01s
ABC.Predictor INFO: Pearson correlations: 1.000 0.952 0.858 0.100
ABC.Predictor DEBUG: Linear regression coefficients (n_target, n_feature):
[[ 9.99527667e-01 2.77857512e-04 5.36876059e-04 -1.68168917e-04
9.69310087e-05 -2.05926422e-04 -3.42453359e-05 -3.84918591e-04
-1.30413605e-04 1.40054597e-04 3.65928537e-04 7.19093022e-05
2.33989326e-04 -4.13467674e-04 1.57383685e-04 -1.40151970e-04
1.33729847e-04]
[-4.31930024e-03 9.51459762e-01 3.06195289e-03 3.69854817e-04
1.66411240e-03 4.88186617e-03 3.21282619e-03 -1.08077935e-03
1.38752338e-03 7.83199668e-04 -4.20321959e-05 3.34944988e-03
-7.87894002e-04 -2.21970135e-03 -2.78464110e-03 -1.88726674e-03
-3.66002190e-03]
[-2.02147953e-02 2.17961067e-02 2.85439014e-01 2.95077432e-01
2.88521977e-01 2.80940280e-01 2.69106385e-03 -3.57860026e-03
-2.09958258e-03 -7.63777867e-03 -3.48141307e-03 -3.52275828e-03
1.85912656e-03 -9.30106640e-03 -3.34854476e-03 1.19108322e-03
4.89654548e-04]
[-4.32315645e-02 -4.54547394e-02 -4.10164233e-03 2.27187728e-02
1.50613743e-02 -3.77747609e-03 6.91002451e-02 1.07832973e-03
-7.73794797e-03 1.10017483e-02 -5.14681986e-03 -6.30359097e-03
-3.90858296e-03 1.32229437e-02 -1.04124795e-02 -7.31994650e-06
-1.16847157e-03]]
ABC.Distance DEBUG: Scale weights[6] = {'s_p1': 1.3968e+00, 's_p2': 1.4649e+00, 's_p3': 1.7281e+00, 's_p4': 1.4842e+01}
ABC INFO: t: 6, eps: 3.56026862e+00.
ABC INFO: Accepted: 1000 / 3212 = 3.1133e-01, ESS: 4.7024e+02.
ABC.Distance DEBUG: Scale weights[7] = {'s_p1': 1.4682e+00, 's_p2': 1.5160e+00, 's_p3': 1.6948e+00, 's_p4': 1.5711e+01}
ABC INFO: t: 7, eps: 2.98109037e+00.
ABC INFO: Accepted: 1000 / 3956 = 2.5278e-01, ESS: 8.0031e+02.
ABC.Distance DEBUG: Scale weights[8] = {'s_p1': 2.0374e+00, 's_p2': 1.9748e+00, 's_p3': 1.6233e+00, 's_p4': 1.7408e+01}
ABC INFO: t: 8, eps: 2.89987216e+00.
ABC INFO: Accepted: 1000 / 4663 = 2.1445e-01, ESS: 6.1422e+02.
ABC.Distance DEBUG: Scale weights[9] = {'s_p1': 2.5535e+00, 's_p2': 2.2357e+00, 's_p3': 1.8566e+00, 's_p4': 1.9539e+01}
ABC INFO: t: 9, eps: 2.80289548e+00.
ABC INFO: Accepted: 1000 / 4973 = 2.0109e-01, ESS: 6.9491e+02.
ABC.Distance DEBUG: Scale weights[10] = {'s_p1': 3.9393e+00, 's_p2': 2.3730e+00, 's_p3': 1.9666e+00, 's_p4': 2.0336e+01}
ABC INFO: t: 10, eps: 2.67588839e+00.
ABC INFO: Accepted: 1000 / 6451 = 1.5501e-01, ESS: 6.3239e+02.
ABC.Distance DEBUG: Scale weights[11] = {'s_p1': 5.2674e+00, 's_p2': 2.7002e+00, 's_p3': 1.9823e+00, 's_p4': 1.9937e+01}
ABC INFO: t: 11, eps: 2.48366508e+00.
ABC INFO: Accepted: 1000 / 7633 = 1.3101e-01, ESS: 4.8692e+02.
ABC.Distance DEBUG: Scale weights[12] = {'s_p1': 7.6135e+00, 's_p2': 2.9702e+00, 's_p3': 2.0205e+00, 's_p4': 2.0528e+01}
ABC INFO: t: 12, eps: 2.39778162e+00.
ABC INFO: Accepted: 1000 / 9812 = 1.0192e-01, ESS: 5.9664e+02.
ABC.Distance DEBUG: Scale weights[13] = {'s_p1': 1.0522e+01, 's_p2': 2.9627e+00, 's_p3': 1.9834e+00, 's_p4': 2.0199e+01}
ABC INFO: t: 13, eps: 2.17302874e+00.
ABC INFO: Accepted: 1000 / 12583 = 7.9472e-02, ESS: 5.6421e+02.
ABC.Distance DEBUG: Scale weights[14] = {'s_p1': 1.4501e+01, 's_p2': 3.1206e+00, 's_p3': 2.0128e+00, 's_p4': 1.9761e+01}
ABC INFO: t: 14, eps: 1.99101473e+00.
ABC INFO: Accepted: 1000 / 16972 = 5.8921e-02, ESS: 4.5603e+02.
ABC.Distance DEBUG: Scale weights[15] = {'s_p1': 1.9828e+01, 's_p2': 3.2828e+00, 's_p3': 2.1231e+00, 's_p4': 2.0252e+01}
ABC INFO: Stop: Total simulations budget.
ABC.History INFO: Done <ABCSMC id=3, duration=0:02:07.613343, end_time=2022-03-01 22:30:16>
ABC.Sampler INFO: Parallelize sampling on 4 processes.
ABC.History INFO: Start <ABCSMC id=4, start_time=2022-03-01 22:30:16>
ABC INFO: Calibration sample t = -1.
L1+Ada.+MAD+StatLR+P4
ABC.Distance DEBUG: Scale weights[0] = {'y1': 2.7301e-01, 'y2': 2.8598e-03, 'y3:0': 2.8602e-03, 'y3:1': 2.7538e-03, 'y3:2': 2.9544e-03, 'y3:3': 2.8587e-03, 'y4': 4.5365e+00, 'y5:0': 1.4590e-01, 'y5:1': 1.5560e-01, 'y5:2': 1.4777e-01, 'y5:3': 1.4560e-01, 'y5:4': 1.5037e-01, 'y5:5': 1.4025e-01, 'y5:6': 1.4648e-01, 'y5:7': 1.5637e-01, 'y5:8': 1.5794e-01, 'y5:9': 1.4908e-01}
ABC.Population INFO: Recording also rejected particles: True
ABC.Population INFO: Recording also rejected particles: True
ABC INFO: t: 0, eps: 1.89369290e+01.
ABC INFO: Accepted: 1000 / 2151 = 4.6490e-01, ESS: 1.0000e+03.
ABC.Distance DEBUG: Scale weights[1] = {'y1': 2.8968e-01, 'y2': 2.8892e-03, 'y3:0': 2.8432e-03, 'y3:1': 2.7813e-03, 'y3:2': 2.8680e-03, 'y3:3': 2.8110e-03, 'y4': 4.5767e+00, 'y5:0': 1.5001e-01, 'y5:1': 1.5527e-01, 'y5:2': 1.4435e-01, 'y5:3': 1.4876e-01, 'y5:4': 1.4710e-01, 'y5:5': 1.4302e-01, 'y5:6': 1.4970e-01, 'y5:7': 1.4820e-01, 'y5:8': 1.4631e-01, 'y5:9': 1.5175e-01}
ABC INFO: t: 1, eps: 1.64852309e+01.
ABC INFO: Accepted: 1000 / 3005 = 3.3278e-01, ESS: 7.4528e+02.
ABC.Distance DEBUG: Scale weights[2] = {'y1': 3.4557e-01, 'y2': 3.3545e-03, 'y3:0': 3.8228e-03, 'y3:1': 3.6598e-03, 'y3:2': 3.8026e-03, 'y3:3': 3.6251e-03, 'y4': 5.5368e+00, 'y5:0': 1.5246e-01, 'y5:1': 1.4430e-01, 'y5:2': 1.5403e-01, 'y5:3': 1.5308e-01, 'y5:4': 1.4364e-01, 'y5:5': 1.4518e-01, 'y5:6': 1.4606e-01, 'y5:7': 1.4537e-01, 'y5:8': 1.4799e-01, 'y5:9': 1.5178e-01}
ABC INFO: t: 2, eps: 1.63290007e+01.
ABC INFO: Accepted: 1000 / 4413 = 2.2660e-01, ESS: 7.7998e+02.
ABC.Distance DEBUG: Scale weights[3] = {'y1': 3.5896e-01, 'y2': 3.4710e-03, 'y3:0': 4.1708e-03, 'y3:1': 4.1406e-03, 'y3:2': 4.2032e-03, 'y3:3': 4.1327e-03, 'y4': 5.8933e+00, 'y5:0': 1.4684e-01, 'y5:1': 1.4973e-01, 'y5:2': 1.4611e-01, 'y5:3': 1.4659e-01, 'y5:4': 1.4848e-01, 'y5:5': 1.5063e-01, 'y5:6': 1.5018e-01, 'y5:7': 1.5276e-01, 'y5:8': 1.4972e-01, 'y5:9': 1.5006e-01}
ABC INFO: t: 3, eps: 1.54057968e+01.
ABC INFO: Accepted: 1000 / 6850 = 1.4599e-01, ESS: 5.7495e+02.
ABC.Distance DEBUG: Scale weights[4] = {'y1': 3.6718e-01, 'y2': 3.6559e-03, 'y3:0': 4.9476e-03, 'y3:1': 5.0767e-03, 'y3:2': 5.0433e-03, 'y3:3': 4.9906e-03, 'y4': 5.8281e+00, 'y5:0': 1.4814e-01, 'y5:1': 1.4396e-01, 'y5:2': 1.4800e-01, 'y5:3': 1.5039e-01, 'y5:4': 1.4888e-01, 'y5:5': 1.4729e-01, 'y5:6': 1.4752e-01, 'y5:7': 1.4578e-01, 'y5:8': 1.4882e-01, 'y5:9': 1.4958e-01}
ABC INFO: t: 4, eps: 1.48215051e+01.
ABC INFO: Accepted: 1000 / 10667 = 9.3747e-02, ESS: 6.9717e+02.
ABC.Distance DEBUG: Scale weights[5] = {'y1': 4.0169e-01, 'y2': 3.7610e-03, 'y3:0': 5.2485e-03, 'y3:1': 5.1891e-03, 'y3:2': 5.3267e-03, 'y3:3': 5.2498e-03, 'y4': 5.9238e+00, 'y5:0': 1.5100e-01, 'y5:1': 1.4852e-01, 'y5:2': 1.5031e-01, 'y5:3': 1.5294e-01, 'y5:4': 1.4890e-01, 'y5:5': 1.4915e-01, 'y5:6': 1.4684e-01, 'y5:7': 1.4759e-01, 'y5:8': 1.4931e-01, 'y5:9': 1.4833e-01}
ABC INFO: t: 5, eps: 1.41767456e+01.
ABC INFO: Accepted: 1000 / 15784 = 6.3355e-02, ESS: 6.1544e+02.
ABC.Predictor INFO: Fitted <LinearPredictor predictor=LinearRegression()> in 0.02s
ABC.Predictor INFO: Pearson correlations: 1.000 0.953 0.855 0.074 0.080 0.040 0.189 0.920 0.876 0.828 0.681 0.079 0.076 0.052 0.221 0.869
ABC.Predictor DEBUG: Linear regression coefficients (n_target, n_feature):
[[ 9.99510768e-01 -4.60738685e-04 1.08625808e-04 1.58691810e-04
-1.98663652e-04 1.53535344e-04 -3.86133701e-05 1.87568095e-04
3.02111648e-04 -1.73577437e-04 3.68225536e-04 8.32443061e-05
-2.30071573e-05 -1.99091970e-04 3.61359436e-05 -1.25270152e-04
5.30494681e-04]
[-2.71651931e-03 9.52676069e-01 6.05641758e-03 -6.48824790e-03
3.35541527e-03 -1.74615352e-03 -1.56981984e-03 8.08272033e-04
4.59887190e-03 3.64368195e-03 -3.85431682e-03 1.09284542e-03
1.48778343e-03 1.19918108e-04 -1.76965296e-03 2.21681897e-03
4.38477781e-06]
[ 9.55890113e-03 -1.00455529e-02 2.95898110e-01 2.82928588e-01
2.79904779e-01 2.86633945e-01 -2.39756035e-03 5.08740517e-03
-1.31256846e-04 2.56362437e-03 6.97985539e-03 5.52809853e-03
4.79922298e-03 5.20053926e-03 1.79696177e-04 -3.68929696e-03
-9.17605856e-03]
[ 1.55306028e-03 -4.48347840e-02 -4.13360255e-03 1.15973150e-02
2.10992272e-03 2.65642118e-02 4.24285803e-02 -8.73863838e-04
-5.35031385e-03 -7.02477676e-03 2.04264402e-03 -9.43458929e-03
4.96108657e-03 -1.37339651e-02 3.48955093e-03 -3.76988253e-03
-1.08385414e-02]
[-1.70605121e-02 -1.27341583e-02 1.33222883e-02 2.00364433e-02
1.64097986e-02 5.61843314e-04 6.16404605e-02 -2.30958667e-03
-8.08117853e-03 8.13189780e-03 -1.35364334e-02 4.68372814e-04
1.29031331e-02 -3.44619473e-03 5.76340530e-03 1.36955172e-02
-6.67887987e-03]
[ 1.72303944e-02 1.77755568e-02 -3.70042159e-03 5.74465016e-03
1.55238336e-02 1.68778619e-03 -4.04296507e-03 -7.43196496e-03
9.71264800e-04 1.01714125e-02 3.95506992e-03 -9.15925486e-03
-2.00936965e-03 7.25733448e-03 1.08787867e-02 1.18587833e-02
-9.58381352e-03]
[ 2.87629241e-02 9.09732075e-03 -5.79602784e-02 -6.42016620e-02
-6.36871296e-02 -4.82742391e-02 -6.38538030e-02 -1.90528926e-03
9.37107175e-03 -3.43178045e-03 -8.12701939e-03 -6.67535713e-03
-1.86426045e-02 -8.66684852e-03 2.43254929e-03 -4.36882962e-03
8.32521525e-03]
[-6.25800699e-04 -1.96334806e-03 -2.66840166e-03 -3.30753519e-03
1.14286573e-04 1.42988276e-03 9.20154633e-01 -3.83632241e-04
5.46510882e-04 1.47669643e-05 3.47002210e-03 7.60685270e-04
8.14322877e-04 1.75581191e-03 2.02053714e-04 4.49624018e-03
1.14995481e-03]
[ 8.76314523e-01 1.30352656e-03 -4.69437902e-03 7.49942526e-04
-1.48872719e-03 -6.03239254e-03 -2.39451275e-03 -2.19554384e-03
-2.01688359e-04 2.31708774e-04 -2.44563009e-03 8.29519086e-03
-2.62431774e-03 -2.92991287e-03 1.47890186e-03 2.79280019e-03
6.80411424e-03]
[ 5.96778951e-03 8.27451435e-01 1.77694286e-02 -3.71081787e-03
1.62718929e-02 3.85922880e-03 -5.82495968e-03 -1.90075288e-03
9.06013609e-03 -6.87577774e-04 3.02224131e-03 8.21232872e-03
-4.77693034e-03 2.06468373e-03 -4.13968304e-03 -7.57001501e-03
-5.20581926e-03]
[ 1.27199175e-02 -7.30998753e-03 2.32044136e-01 2.33487791e-01
2.22610332e-01 2.23465981e-01 1.35978016e-02 3.39062449e-03
-1.58535378e-03 -1.17487462e-03 4.37696249e-03 1.31478691e-02
1.53600241e-02 6.38439244e-03 8.18357619e-03 -1.60247368e-03
-2.01272116e-02]
[ 1.33973198e-02 -3.91112176e-02 -8.21646648e-03 1.12698430e-02
9.95368929e-04 2.71698773e-02 5.45530264e-02 5.57162223e-03
-4.27513577e-03 -2.72563490e-03 5.49279283e-03 -1.00586833e-02
-3.28606415e-03 -1.47122086e-02 7.35787206e-03 -3.41086081e-03
-7.42568943e-03]
[-1.39739981e-02 -8.94928206e-03 1.32888438e-02 1.46829964e-02
1.68172673e-02 -1.24626527e-03 6.17375206e-02 -2.97136398e-03
-5.77341470e-03 3.33107486e-03 -1.14728614e-02 1.30632635e-03
1.02864363e-02 -3.34803612e-03 8.80772550e-03 1.40247213e-02
-5.18279102e-03]
[ 2.68656764e-02 2.45820570e-02 -9.76984045e-04 1.13583035e-02
1.41258982e-02 2.46462949e-03 -3.76243932e-03 -5.85395000e-03
1.03532290e-03 8.16963711e-03 6.26811076e-03 -8.60308436e-03
-6.37585666e-03 8.49752609e-03 1.01030555e-02 1.60698751e-02
-1.22366322e-02]
[ 1.71741385e-02 4.95248890e-03 -7.15597051e-02 -7.82607234e-02
-7.32612181e-02 -6.02245586e-02 -5.54288381e-02 -2.24264410e-03
8.93001781e-03 -3.12149040e-04 -9.52643686e-03 -7.98770418e-03
-2.14061739e-02 -8.38043420e-03 -3.88531504e-03 -1.45558399e-03
1.58920242e-02]
[-1.06294074e-03 -2.65078230e-03 -2.84533633e-03 -5.82971025e-03
1.97754633e-03 6.40923888e-03 8.68422108e-01 1.61949398e-04
-2.34579792e-03 -7.58774828e-04 7.82382684e-04 1.10468113e-03
-2.00109223e-03 2.79841331e-03 -3.45871951e-03 5.57442247e-03
2.77604751e-03]]
ABC.Distance DEBUG: Scale weights[6] = {'s_p1^1': 1.3690e+00, 's_p2^1': 1.4478e+00, 's_p3^1': 1.6919e+00, 's_p4^1': 2.0158e+01, 's_p1^2': 1.8840e+01, 's_p2^2': 3.6457e+01, 's_p3^2': 7.6750e+00, 's_p4^2': 1.7680e+00, 's_p1^3': 1.5612e+00, 's_p2^3': 1.6691e+00, 's_p3^3': 2.1190e+00, 's_p4^3': 1.9001e+01, 's_p1^4': 2.0079e+01, 's_p2^4': 2.8146e+01, 's_p3^4': 6.5311e+00, 's_p4^4': 1.8716e+00}
ABC INFO: t: 6, eps: 1.37234275e+01.
ABC INFO: Accepted: 1000 / 3691 = 2.7093e-01, ESS: 1.2819e+02.
ABC.Distance DEBUG: Scale weights[7] = {'s_p1^1': 1.5625e+00, 's_p2^1': 1.4516e+00, 's_p3^1': 1.7657e+00, 's_p4^1': 1.9478e+01, 's_p1^2': 1.9394e+01, 's_p2^2': 3.8006e+01, 's_p3^2': 7.8179e+00, 's_p4^2': 1.9016e+00, 's_p1^3': 1.7850e+00, 's_p2^3': 1.6766e+00, 's_p3^3': 2.2346e+00, 's_p4^3': 1.8747e+01, 's_p1^4': 2.1135e+01, 's_p2^4': 2.9855e+01, 's_p3^4': 6.7159e+00, 's_p4^4': 2.0246e+00}
ABC INFO: t: 7, eps: 1.23640875e+01.
ABC INFO: Accepted: 1000 / 6659 = 1.5017e-01, ESS: 7.0947e+02.
ABC.Distance DEBUG: Scale weights[8] = {'s_p1^1': 1.7313e+00, 's_p2^1': 1.6197e+00, 's_p3^1': 1.4878e+00, 's_p4^1': 2.0500e+01, 's_p1^2': 1.7966e+01, 's_p2^2': 3.6743e+01, 's_p3^2': 6.8386e+00, 's_p4^2': 1.6877e+00, 's_p1^3': 1.9756e+00, 's_p2^3': 1.8514e+00, 's_p3^3': 1.8647e+00, 's_p4^3': 1.8922e+01, 's_p1^4': 1.9282e+01, 's_p2^4': 2.8572e+01, 's_p3^4': 5.8604e+00, 's_p4^4': 1.7863e+00}
ABC INFO: t: 8, eps: 1.01883360e+01.
ABC INFO: Accepted: 1000 / 7433 = 1.3454e-01, ESS: 5.6557e+02.
ABC.Distance DEBUG: Scale weights[9] = {'s_p1^1': 1.9133e+00, 's_p2^1': 1.8905e+00, 's_p3^1': 1.9246e+00, 's_p4^1': 2.2856e+01, 's_p1^2': 2.0931e+01, 's_p2^2': 4.1685e+01, 's_p3^2': 8.7301e+00, 's_p4^2': 1.8820e+00, 's_p1^3': 2.1923e+00, 's_p2^3': 2.1916e+00, 's_p3^3': 2.4190e+00, 's_p4^3': 2.1628e+01, 's_p1^4': 2.1949e+01, 's_p2^4': 3.3985e+01, 's_p3^4': 7.4372e+00, 's_p4^4': 1.9971e+00}
ABC INFO: t: 9, eps: 1.03225060e+01.
ABC INFO: Accepted: 1000 / 9692 = 1.0318e-01, ESS: 6.5811e+01.
ABC.Distance DEBUG: Scale weights[10] = {'s_p1^1': 2.4984e+00, 's_p2^1': 2.1868e+00, 's_p3^1': 1.9821e+00, 's_p4^1': 2.4080e+01, 's_p1^2': 2.1430e+01, 's_p2^2': 4.2256e+01, 's_p3^2': 8.8317e+00, 's_p4^2': 1.9412e+00, 's_p1^3': 2.8393e+00, 's_p2^3': 2.5081e+00, 's_p3^3': 2.4766e+00, 's_p4^3': 2.2385e+01, 's_p1^4': 2.2458e+01, 's_p2^4': 3.3949e+01, 's_p3^4': 7.5083e+00, 's_p4^4': 2.0618e+00}
ABC INFO: t: 10, eps: 9.94845850e+00.
ABC INFO: Accepted: 1000 / 17522 = 5.7071e-02, ESS: 6.5216e+02.
ABC.Distance DEBUG: Scale weights[11] = {'s_p1^1': 2.5797e+00, 's_p2^1': 2.3304e+00, 's_p3^1': 1.6310e+00, 's_p4^1': 2.3434e+01, 's_p1^2': 1.9444e+01, 's_p2^2': 4.1726e+01, 's_p3^2': 7.4219e+00, 's_p4^2': 1.7194e+00, 's_p1^3': 2.9249e+00, 's_p2^3': 2.6776e+00, 's_p3^3': 2.0471e+00, 's_p4^3': 2.1381e+01, 's_p1^4': 2.0473e+01, 's_p2^4': 3.4299e+01, 's_p3^4': 6.3034e+00, 's_p4^4': 1.8240e+00}
ABC INFO: t: 11, eps: 8.46651838e+00.
ABC INFO: Accepted: 1000 / 14909 = 6.7074e-02, ESS: 6.1217e+02.
ABC.Distance DEBUG: Scale weights[12] = {'s_p1^1': 3.8851e+00, 's_p2^1': 2.4552e+00, 's_p3^1': 2.0971e+00, 's_p4^1': 2.5210e+01, 's_p1^2': 2.1455e+01, 's_p2^2': 4.3251e+01, 's_p3^2': 9.4128e+00, 's_p4^2': 1.8732e+00, 's_p1^3': 4.4449e+00, 's_p2^3': 2.8282e+00, 's_p3^3': 2.6400e+00, 's_p4^3': 2.2642e+01, 's_p1^4': 2.2430e+01, 's_p2^4': 3.6190e+01, 's_p3^4': 7.9923e+00, 's_p4^4': 1.9853e+00}
ABC INFO: Stop: Total simulations budget.
ABC.History INFO: Done <ABCSMC id=4, duration=0:01:56.427502, end_time=2022-03-01 22:32:13>
ABC.Sampler INFO: Parallelize sampling on 4 processes.
ABC.History INFO: Start <ABCSMC id=5, start_time=2022-03-01 22:32:13>
ABC INFO: Calibration sample t = -1.
L1+Ada.+MAD+SensiLR
ABC.Distance DEBUG: Scale weights[0] = {'y1': 3.0150e-01, 'y2': 2.7824e-03, 'y3:0': 2.7615e-03, 'y3:1': 2.9441e-03, 'y3:2': 2.8034e-03, 'y3:3': 2.8899e-03, 'y4': 4.5520e+00, 'y5:0': 1.4869e-01, 'y5:1': 1.5296e-01, 'y5:2': 1.5816e-01, 'y5:3': 1.4395e-01, 'y5:4': 1.5551e-01, 'y5:5': 1.4295e-01, 'y5:6': 1.4652e-01, 'y5:7': 1.5257e-01, 'y5:8': 1.4990e-01, 'y5:9': 1.4121e-01}
ABC.Population INFO: Recording also rejected particles: True
ABC.Population INFO: Recording also rejected particles: True
ABC INFO: t: 0, eps: 1.92772734e+01.
ABC INFO: Accepted: 1000 / 1975 = 5.0633e-01, ESS: 1.0000e+03.
ABC.Distance DEBUG: Scale weights[1] = {'y1': 2.8377e-01, 'y2': 2.9485e-03, 'y3:0': 2.7075e-03, 'y3:1': 2.8474e-03, 'y3:2': 2.8313e-03, 'y3:3': 2.8522e-03, 'y4': 4.3678e+00, 'y5:0': 1.5383e-01, 'y5:1': 1.4529e-01, 'y5:2': 1.5591e-01, 'y5:3': 1.4766e-01, 'y5:4': 1.5196e-01, 'y5:5': 1.4831e-01, 'y5:6': 1.5933e-01, 'y5:7': 1.5182e-01, 'y5:8': 1.4658e-01, 'y5:9': 1.4724e-01}
ABC INFO: t: 1, eps: 1.68839789e+01.
ABC INFO: Accepted: 1000 / 2930 = 3.4130e-01, ESS: 8.2233e+02.
ABC.Distance DEBUG: Scale weights[2] = {'y1': 3.2800e-01, 'y2': 3.5557e-03, 'y3:0': 3.6778e-03, 'y3:1': 3.6777e-03, 'y3:2': 3.5765e-03, 'y3:3': 3.6011e-03, 'y4': 5.1697e+00, 'y5:0': 1.5209e-01, 'y5:1': 1.5091e-01, 'y5:2': 1.4854e-01, 'y5:3': 1.4277e-01, 'y5:4': 1.4625e-01, 'y5:5': 1.5492e-01, 'y5:6': 1.4560e-01, 'y5:7': 1.4210e-01, 'y5:8': 1.4691e-01, 'y5:9': 1.4825e-01}
ABC INFO: t: 2, eps: 1.61392223e+01.
ABC INFO: Accepted: 1000 / 4488 = 2.2282e-01, ESS: 6.8232e+02.
ABC.Distance DEBUG: Scale weights[3] = {'y1': 3.4553e-01, 'y2': 3.6582e-03, 'y3:0': 4.0849e-03, 'y3:1': 4.2980e-03, 'y3:2': 4.2035e-03, 'y3:3': 4.2421e-03, 'y4': 5.7293e+00, 'y5:0': 1.4560e-01, 'y5:1': 1.4869e-01, 'y5:2': 1.4653e-01, 'y5:3': 1.4306e-01, 'y5:4': 1.5172e-01, 'y5:5': 1.4930e-01, 'y5:6': 1.5290e-01, 'y5:7': 1.4766e-01, 'y5:8': 1.4528e-01, 'y5:9': 1.4781e-01}
ABC INFO: t: 3, eps: 1.55209850e+01.
ABC INFO: Accepted: 1000 / 6284 = 1.5913e-01, ESS: 7.6418e+02.
ABC.Distance DEBUG: Scale weights[4] = {'y1': 3.8963e-01, 'y2': 3.6697e-03, 'y3:0': 4.7152e-03, 'y3:1': 4.6414e-03, 'y3:2': 4.6223e-03, 'y3:3': 4.7370e-03, 'y4': 6.0951e+00, 'y5:0': 1.5133e-01, 'y5:1': 1.5027e-01, 'y5:2': 1.4847e-01, 'y5:3': 1.4562e-01, 'y5:4': 1.4481e-01, 'y5:5': 1.5238e-01, 'y5:6': 1.5083e-01, 'y5:7': 1.5144e-01, 'y5:8': 1.4729e-01, 'y5:9': 1.4924e-01}
ABC INFO: t: 4, eps: 1.50008507e+01.
ABC INFO: Accepted: 1000 / 9432 = 1.0602e-01, ESS: 4.4768e+02.
ABC.Distance DEBUG: Scale weights[5] = {'y1': 4.2876e-01, 'y2': 4.0297e-03, 'y3:0': 5.2911e-03, 'y3:1': 5.2451e-03, 'y3:2': 5.3291e-03, 'y3:3': 5.4409e-03, 'y4': 5.9510e+00, 'y5:0': 1.4590e-01, 'y5:1': 1.5094e-01, 'y5:2': 1.4937e-01, 'y5:3': 1.4755e-01, 'y5:4': 1.4817e-01, 'y5:5': 1.4640e-01, 'y5:6': 1.4594e-01, 'y5:7': 1.4589e-01, 'y5:8': 1.4631e-01, 'y5:9': 1.4482e-01}
ABC INFO: t: 5, eps: 1.44154487e+01.
ABC INFO: Accepted: 1000 / 15471 = 6.4637e-02, ESS: 5.3378e+02.
ABC.Distance DEBUG: Scale weights[6] = {'y1': 4.2999e-01, 'y2': 4.0846e-03, 'y3:0': 5.3919e-03, 'y3:1': 5.3642e-03, 'y3:2': 5.3249e-03, 'y3:3': 5.3809e-03, 'y4': 6.4527e+00, 'y5:0': 1.4656e-01, 'y5:1': 1.4889e-01, 'y5:2': 1.4897e-01, 'y5:3': 1.4938e-01, 'y5:4': 1.4779e-01, 'y5:5': 1.4796e-01, 'y5:6': 1.4813e-01, 'y5:7': 1.4685e-01, 'y5:8': 1.4725e-01, 'y5:9': 1.5146e-01}
ABC.Predictor INFO: Fitted <LinearPredictor predictor=LinearRegression()> in 0.01s
ABC.Predictor INFO: Pearson correlations: 1.000 0.955 0.890 0.078
ABC.Predictor DEBUG: Linear regression coefficients (n_target, n_feature):
[[ 9.99505960e-01 5.06331385e-04 1.98724232e-04 6.49224356e-04
-9.27269004e-05 5.82686633e-05 3.31357444e-04 -1.79866552e-04
2.84368115e-04 1.19795829e-04 -8.35634747e-05 1.97516381e-04
-2.11504669e-04 -1.04215601e-04 -1.41961315e-04 9.57926735e-05
2.97832771e-05]
[ 7.97678398e-05 9.53973936e-01 -2.14726363e-03 -2.68573550e-03
-4.36075109e-03 -2.48727609e-03 -1.57920770e-03 4.47055467e-03
-6.51982765e-04 1.42307469e-03 4.29395830e-03 1.09203904e-05
-4.50272110e-03 3.04646042e-03 -1.05339461e-03 9.99928306e-04
-5.35562693e-05]
[ 1.03822406e-02 -2.36336729e-02 2.89615388e-01 2.83093363e-01
2.81648079e-01 2.75534880e-01 -9.87834276e-03 -3.76521626e-05
1.45724359e-03 -2.16436282e-03 -6.86677212e-03 -3.90985077e-04
3.93469411e-03 -4.24750038e-03 3.95140603e-03 6.06811872e-03
1.88138504e-03]
[-6.62215494e-03 3.40491371e-02 9.69950851e-03 2.59583742e-02
1.73689401e-03 2.90470203e-02 -2.32474569e-03 7.79645306e-03
2.69902759e-02 1.16379453e-02 -1.01635698e-02 -2.75373936e-02
-5.03465669e-04 7.95753133e-03 -1.73128732e-02 -5.38443607e-03
9.27345913e-03]]
ABC.Distance DEBUG: Optimal FD delta: [0.1 0.1 0.1 0.01 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1
0.1 0.1 0.1 ]
ABC.Distance DEBUG: Info weights[6] = {'y1': 1.0370e+00, 'y2': 1.1459e+00, 'y3:0': 2.8091e-01, 'y3:1': 3.4582e-01, 'y3:2': 2.4562e-01, 'y3:3': 3.5396e-01, 'y4': 1.8043e-02, 'y5:0': 3.7897e-02, 'y5:1': 1.1608e-01, 'y5:2': 5.1964e-02, 'y5:3': 5.2608e-02, 'y5:4': 1.1653e-01, 'y5:5': 9.8349e-03, 'y5:6': 4.0609e-02, 'y5:7': 7.7841e-02, 'y5:8': 2.9043e-02, 'y5:9': 4.0342e-02}
ABC INFO: t: 6, eps: 3.41013611e+00.
ABC INFO: Accepted: 1000 / 3945 = 2.5349e-01, ESS: 6.2250e+02.
ABC.Distance DEBUG: Scale weights[7] = {'y1': 4.8149e-01, 'y2': 4.1963e-03, 'y3:0': 5.7683e-03, 'y3:1': 5.8005e-03, 'y3:2': 5.6042e-03, 'y3:3': 5.7032e-03, 'y4': 6.3354e+00, 'y5:0': 1.4814e-01, 'y5:1': 1.4868e-01, 'y5:2': 1.5187e-01, 'y5:3': 1.4827e-01, 'y5:4': 1.4769e-01, 'y5:5': 1.4542e-01, 'y5:6': 1.4911e-01, 'y5:7': 1.4657e-01, 'y5:8': 1.4387e-01, 'y5:9': 1.4754e-01}
ABC INFO: t: 7, eps: 3.08936405e+00.
ABC INFO: Accepted: 1000 / 3571 = 2.8003e-01, ESS: 7.2115e+02.
ABC.Distance DEBUG: Scale weights[8] = {'y1': 9.2029e-01, 'y2': 6.8780e-03, 'y3:0': 5.2832e-03, 'y3:1': 5.3990e-03, 'y3:2': 5.4486e-03, 'y3:3': 5.2554e-03, 'y4': 5.4432e+00, 'y5:0': 1.5054e-01, 'y5:1': 1.4689e-01, 'y5:2': 1.4756e-01, 'y5:3': 1.4784e-01, 'y5:4': 1.5015e-01, 'y5:5': 1.5320e-01, 'y5:6': 1.4733e-01, 'y5:7': 1.4586e-01, 'y5:8': 1.4476e-01, 'y5:9': 1.4680e-01}
ABC INFO: t: 8, eps: 3.25680313e+00.
ABC INFO: Accepted: 1000 / 3967 = 2.5208e-01, ESS: 7.1272e+02.
ABC.Distance DEBUG: Scale weights[9] = {'y1': 1.3387e+00, 'y2': 8.0504e-03, 'y3:0': 5.7341e-03, 'y3:1': 5.7367e-03, 'y3:2': 5.5893e-03, 'y3:3': 5.5547e-03, 'y4': 5.5734e+00, 'y5:0': 1.5191e-01, 'y5:1': 1.4728e-01, 'y5:2': 1.5207e-01, 'y5:3': 1.4980e-01, 'y5:4': 1.4617e-01, 'y5:5': 1.4725e-01, 'y5:6': 1.4786e-01, 'y5:7': 1.5477e-01, 'y5:8': 1.4939e-01, 'y5:9': 1.4476e-01}
ABC INFO: t: 9, eps: 3.20569742e+00.
ABC INFO: Accepted: 1000 / 4924 = 2.0309e-01, ESS: 5.4541e+02.
ABC.Distance DEBUG: Scale weights[10] = {'y1': 1.9327e+00, 'y2': 8.4124e-03, 'y3:0': 5.6746e-03, 'y3:1': 5.7413e-03, 'y3:2': 5.5302e-03, 'y3:3': 5.6135e-03, 'y4': 5.4430e+00, 'y5:0': 1.4433e-01, 'y5:1': 1.5127e-01, 'y5:2': 1.4593e-01, 'y5:3': 1.5161e-01, 'y5:4': 1.5110e-01, 'y5:5': 1.4552e-01, 'y5:6': 1.5001e-01, 'y5:7': 1.4830e-01, 'y5:8': 1.5105e-01, 'y5:9': 1.4678e-01}
ABC INFO: t: 10, eps: 2.98950818e+00.
ABC INFO: Accepted: 1000 / 6007 = 1.6647e-01, ESS: 5.0535e+02.
ABC.Distance DEBUG: Scale weights[11] = {'y1': 2.9024e+00, 'y2': 9.3798e-03, 'y3:0': 5.5734e-03, 'y3:1': 5.3665e-03, 'y3:2': 5.5896e-03, 'y3:3': 5.4790e-03, 'y4': 5.1359e+00, 'y5:0': 1.4794e-01, 'y5:1': 1.4645e-01, 'y5:2': 1.4794e-01, 'y5:3': 1.4766e-01, 'y5:4': 1.4778e-01, 'y5:5': 1.5242e-01, 'y5:6': 1.4822e-01, 'y5:7': 1.5010e-01, 'y5:8': 1.4679e-01, 'y5:9': 1.4820e-01}
ABC INFO: t: 11, eps: 2.83424965e+00.
ABC INFO: Accepted: 1000 / 7445 = 1.3432e-01, ESS: 6.0355e+02.
ABC.Distance DEBUG: Scale weights[12] = {'y1': 4.5457e+00, 'y2': 8.9210e-03, 'y3:0': 5.6495e-03, 'y3:1': 5.6173e-03, 'y3:2': 5.7553e-03, 'y3:3': 5.5339e-03, 'y4': 5.3425e+00, 'y5:0': 1.5017e-01, 'y5:1': 1.4725e-01, 'y5:2': 1.4874e-01, 'y5:3': 1.4501e-01, 'y5:4': 1.4321e-01, 'y5:5': 1.4677e-01, 'y5:6': 1.4888e-01, 'y5:7': 1.4672e-01, 'y5:8': 1.5033e-01, 'y5:9': 1.4516e-01}
ABC INFO: t: 12, eps: 2.68180644e+00.
ABC INFO: Accepted: 1000 / 9886 = 1.0115e-01, ESS: 6.0227e+02.
ABC.Distance DEBUG: Scale weights[13] = {'y1': 6.3874e+00, 'y2': 9.2961e-03, 'y3:0': 5.7875e-03, 'y3:1': 5.6329e-03, 'y3:2': 5.7978e-03, 'y3:3': 5.8240e-03, 'y4': 5.4496e+00, 'y5:0': 1.4906e-01, 'y5:1': 1.4630e-01, 'y5:2': 1.4846e-01, 'y5:3': 1.4924e-01, 'y5:4': 1.4633e-01, 'y5:5': 1.4622e-01, 'y5:6': 1.4883e-01, 'y5:7': 1.4429e-01, 'y5:8': 1.4710e-01, 'y5:9': 1.5105e-01}
ABC INFO: t: 13, eps: 2.55883444e+00.
ABC INFO: Accepted: 1000 / 14493 = 6.8999e-02, ESS: 4.7837e+02.
ABC.Distance DEBUG: Scale weights[14] = {'y1': 7.8777e+00, 'y2': 9.9930e-03, 'y3:0': 5.8969e-03, 'y3:1': 5.8393e-03, 'y3:2': 5.8364e-03, 'y3:3': 5.7734e-03, 'y4': 5.4041e+00, 'y5:0': 1.4911e-01, 'y5:1': 1.4758e-01, 'y5:2': 1.4736e-01, 'y5:3': 1.4716e-01, 'y5:4': 1.4780e-01, 'y5:5': 1.4812e-01, 'y5:6': 1.4770e-01, 'y5:7': 1.4886e-01, 'y5:8': 1.4654e-01, 'y5:9': 1.5054e-01}
ABC INFO: t: 14, eps: 2.39374197e+00.
ABC INFO: Accepted: 1000 / 22574 = 4.4299e-02, ESS: 2.5037e+02.
ABC.Distance DEBUG: Scale weights[15] = {'y1': 9.2921e+00, 'y2': 9.8073e-03, 'y3:0': 6.0602e-03, 'y3:1': 6.0103e-03, 'y3:2': 6.0713e-03, 'y3:3': 5.9636e-03, 'y4': 5.5552e+00, 'y5:0': 1.4657e-01, 'y5:1': 1.4723e-01, 'y5:2': 1.4786e-01, 'y5:3': 1.4790e-01, 'y5:4': 1.4593e-01, 'y5:5': 1.4750e-01, 'y5:6': 1.4739e-01, 'y5:7': 1.4779e-01, 'y5:8': 1.4746e-01, 'y5:9': 1.4869e-01}
ABC INFO: Stop: Total simulations budget.
ABC.History INFO: Done <ABCSMC id=5, duration=0:02:00.293444, end_time=2022-03-01 22:34:13>
ABC.Sampler INFO: Parallelize sampling on 4 processes.
ABC.History INFO: Start <ABCSMC id=6, start_time=2022-03-01 22:34:13>
ABC INFO: Calibration sample t = -1.
L1+Ada.+MAD+SensiLR+P4
ABC.Distance DEBUG: Scale weights[0] = {'y1': 3.0537e-01, 'y2': 2.8441e-03, 'y3:0': 2.8437e-03, 'y3:1': 2.5888e-03, 'y3:2': 2.7542e-03, 'y3:3': 2.8230e-03, 'y4': 4.8179e+00, 'y5:0': 1.4825e-01, 'y5:1': 1.4457e-01, 'y5:2': 1.5298e-01, 'y5:3': 1.4922e-01, 'y5:4': 1.4718e-01, 'y5:5': 1.4664e-01, 'y5:6': 1.4813e-01, 'y5:7': 1.4736e-01, 'y5:8': 1.5168e-01, 'y5:9': 1.4919e-01}
ABC.Population INFO: Recording also rejected particles: True
ABC.Population INFO: Recording also rejected particles: True
ABC INFO: t: 0, eps: 1.92658228e+01.
ABC INFO: Accepted: 1000 / 2052 = 4.8733e-01, ESS: 1.0000e+03.
ABC.Distance DEBUG: Scale weights[1] = {'y1': 2.8694e-01, 'y2': 2.8590e-03, 'y3:0': 2.7455e-03, 'y3:1': 2.6707e-03, 'y3:2': 2.8282e-03, 'y3:3': 2.7406e-03, 'y4': 4.5965e+00, 'y5:0': 1.4733e-01, 'y5:1': 1.5229e-01, 'y5:2': 1.4179e-01, 'y5:3': 1.4988e-01, 'y5:4': 1.4262e-01, 'y5:5': 1.4545e-01, 'y5:6': 1.4481e-01, 'y5:7': 1.4590e-01, 'y5:8': 1.5127e-01, 'y5:9': 1.4650e-01}
ABC INFO: t: 1, eps: 1.65432416e+01.
ABC INFO: Accepted: 1000 / 2805 = 3.5651e-01, ESS: 8.2051e+02.
ABC.Distance DEBUG: Scale weights[2] = {'y1': 3.2767e-01, 'y2': 3.4156e-03, 'y3:0': 3.6100e-03, 'y3:1': 3.5093e-03, 'y3:2': 3.6095e-03, 'y3:3': 3.5531e-03, 'y4': 5.7057e+00, 'y5:0': 1.5678e-01, 'y5:1': 1.4765e-01, 'y5:2': 1.4942e-01, 'y5:3': 1.4357e-01, 'y5:4': 1.4967e-01, 'y5:5': 1.4921e-01, 'y5:6': 1.5103e-01, 'y5:7': 1.5058e-01, 'y5:8': 1.4552e-01, 'y5:9': 1.5508e-01}
ABC INFO: t: 2, eps: 1.63761812e+01.
ABC INFO: Accepted: 1000 / 4243 = 2.3568e-01, ESS: 7.7280e+02.
ABC.Distance DEBUG: Scale weights[3] = {'y1': 3.5603e-01, 'y2': 3.6163e-03, 'y3:0': 4.2498e-03, 'y3:1': 4.1430e-03, 'y3:2': 4.2602e-03, 'y3:3': 4.3458e-03, 'y4': 5.8988e+00, 'y5:0': 1.4953e-01, 'y5:1': 1.5370e-01, 'y5:2': 1.4384e-01, 'y5:3': 1.4341e-01, 'y5:4': 1.5126e-01, 'y5:5': 1.5254e-01, 'y5:6': 1.4732e-01, 'y5:7': 1.4856e-01, 'y5:8': 1.4938e-01, 'y5:9': 1.4586e-01}
ABC INFO: t: 3, eps: 1.54018211e+01.
ABC INFO: Accepted: 1000 / 6935 = 1.4420e-01, ESS: 6.4810e+02.
ABC.Distance DEBUG: Scale weights[4] = {'y1': 3.9915e-01, 'y2': 3.7827e-03, 'y3:0': 4.7571e-03, 'y3:1': 4.8260e-03, 'y3:2': 4.7792e-03, 'y3:3': 4.8713e-03, 'y4': 6.2252e+00, 'y5:0': 1.5107e-01, 'y5:1': 1.4890e-01, 'y5:2': 1.4624e-01, 'y5:3': 1.4770e-01, 'y5:4': 1.4854e-01, 'y5:5': 1.4669e-01, 'y5:6': 1.4648e-01, 'y5:7': 1.5307e-01, 'y5:8': 1.4702e-01, 'y5:9': 1.4776e-01}
ABC INFO: t: 4, eps: 1.46632824e+01.
ABC INFO: Accepted: 1000 / 10959 = 9.1249e-02, ESS: 6.2475e+02.
ABC.Distance DEBUG: Scale weights[5] = {'y1': 4.2331e-01, 'y2': 3.9298e-03, 'y3:0': 5.3554e-03, 'y3:1': 5.3350e-03, 'y3:2': 5.3235e-03, 'y3:3': 5.3094e-03, 'y4': 6.4311e+00, 'y5:0': 1.4832e-01, 'y5:1': 1.4896e-01, 'y5:2': 1.4839e-01, 'y5:3': 1.4820e-01, 'y5:4': 1.4770e-01, 'y5:5': 1.5282e-01, 'y5:6': 1.4834e-01, 'y5:7': 1.4746e-01, 'y5:8': 1.4865e-01, 'y5:9': 1.4801e-01}
ABC INFO: t: 5, eps: 1.41605699e+01.
ABC INFO: Accepted: 1000 / 16573 = 6.0339e-02, ESS: 4.8160e+02.
ABC.Distance DEBUG: Scale weights[6] = {'y1': 4.5506e-01, 'y2': 4.1339e-03, 'y3:0': 5.7586e-03, 'y3:1': 5.7424e-03, 'y3:2': 5.8308e-03, 'y3:3': 5.6785e-03, 'y4': 6.1080e+00, 'y5:0': 1.4924e-01, 'y5:1': 1.4785e-01, 'y5:2': 1.4912e-01, 'y5:3': 1.4667e-01, 'y5:4': 1.4854e-01, 'y5:5': 1.4818e-01, 'y5:6': 1.4686e-01, 'y5:7': 1.5071e-01, 'y5:8': 1.4947e-01, 'y5:9': 1.4931e-01}
ABC.Predictor INFO: Fitted <LinearPredictor predictor=LinearRegression()> in 0.02s
ABC.Predictor INFO: Pearson correlations: 0.999 0.953 0.850 0.124 0.081 0.077 0.082 0.925 0.867 0.827 0.683 0.121 0.079 0.086 0.094 0.875
ABC.Predictor DEBUG: Linear regression coefficients (n_target, n_feature):
[[ 9.99467799e-01 -6.08456786e-05 -5.96816711e-05 -9.77668494e-05
2.39617680e-04 -8.92999377e-05 4.22410927e-05 1.69655282e-04
-4.44860924e-05 1.86195873e-04 5.05596525e-04 3.79109863e-04
1.75463461e-04 -7.70261189e-05 4.95800243e-04 -1.72393054e-04
-2.52460553e-04]
[-3.94005909e-03 9.52625808e-01 -1.51758187e-03 -1.29099679e-03
2.78521627e-03 -7.00666584e-03 1.34386758e-04 -1.61199310e-03
2.52300175e-03 5.72280693e-03 -3.80291728e-03 6.88461560e-04
-2.19637490e-03 -2.95666180e-05 3.04181510e-03 6.69539045e-03
3.15937444e-05]
[ 1.50406124e-02 -1.19775493e-02 2.94759912e-01 2.89398576e-01
2.83188362e-01 2.84292226e-01 -1.79451966e-03 3.10644248e-03
-7.93030482e-03 -5.34238685e-03 3.36709745e-03 -5.27885998e-04
1.33833095e-03 -4.90404382e-03 4.91586834e-03 2.48544435e-03
6.65773906e-03]
[-4.29727247e-02 6.78809588e-02 -3.75164640e-02 -2.73419876e-02
-2.94059185e-02 -2.33241195e-02 6.92171987e-03 5.98383634e-03
-7.92166171e-03 -7.71487811e-03 -4.53864121e-03 -1.02870134e-02
2.72949986e-03 -1.61452844e-03 -1.35030681e-03 -5.27953011e-03
8.75565996e-03]
[-5.46831600e-02 -3.89667658e-02 4.51058656e-03 1.49600678e-03
3.59513775e-02 4.19507023e-04 2.17130079e-02 7.94630106e-03
-2.38982129e-03 6.63414175e-03 7.16806574e-03 -1.08874688e-02
-4.08254398e-03 -5.91218983e-03 -9.18105497e-04 2.66562589e-03
-1.11885793e-02]
[-4.33170926e-02 -5.84786903e-02 1.48555950e-02 -6.83098761e-03
-1.52933134e-02 1.51730784e-03 -3.75059409e-03 8.90618583e-05
-1.00873073e-02 -4.43976636e-03 -6.69733662e-03 2.65948070e-03
3.08437689e-03 -9.78588767e-03 -5.56294130e-03 2.38408934e-03
-1.15231963e-02]
[-6.87404238e-02 2.61986061e-02 -1.28884618e-02 -4.03627600e-03
-1.42847406e-02 1.63582062e-03 6.37321997e-03 4.91841683e-03
5.33362087e-03 -6.48723330e-03 5.54300308e-03 -3.21128457e-03
9.90302942e-03 2.87963380e-04 -1.66737282e-03 -1.72875830e-03
-6.80467921e-03]
[ 1.42623400e-03 -2.56332770e-03 3.46320867e-03 -6.06217730e-03
-9.62905690e-04 2.54571955e-03 9.24946371e-01 -3.42009163e-03
3.31886338e-04 1.49397518e-03 2.69706465e-03 -2.81370032e-03
-2.28303040e-03 -3.92370621e-03 4.12718816e-03 3.91637455e-03
-6.37193916e-04]
[ 8.67776728e-01 1.76252848e-02 -1.59609770e-02 -1.10021771e-03
-7.97046781e-03 -2.19411708e-03 1.67970973e-02 8.36228086e-03
-3.09691241e-04 -3.52425528e-03 -1.67665097e-03 5.33739631e-03
-3.20494515e-03 2.49444727e-03 6.48756694e-03 9.21098220e-04
2.02104810e-03]
[ 1.09887225e-02 8.26910701e-01 3.43621313e-03 -1.17949386e-03
-1.13712786e-03 5.32917668e-04 1.57859662e-02 4.16087998e-04
3.24019382e-03 8.08376849e-03 -8.69719681e-04 -2.08583364e-03
2.57782439e-03 -5.88183165e-03 1.34058318e-03 9.40357644e-04
1.49951536e-03]
[ 9.26484417e-03 -2.28874913e-02 2.31666085e-01 2.36306181e-01
2.31689570e-01 2.23534208e-01 -2.67149060e-02 4.46423506e-03
-1.52495534e-03 -6.65712901e-03 7.22358909e-04 -1.07947284e-03
6.94144197e-04 -7.03428292e-03 -3.75406598e-03 -2.87239767e-03
3.08042348e-03]
[-5.30008965e-02 6.90398917e-02 -3.46661020e-02 -1.86209478e-02
-2.68786349e-02 -2.19446837e-02 3.65099894e-03 5.90193471e-03
-5.48885782e-03 -3.09419571e-03 -3.62972114e-03 -7.84707926e-03
1.03394692e-03 -1.00174257e-03 -2.03988489e-03 -5.74044890e-04
1.06075451e-02]
[-5.15345592e-02 -4.19702203e-02 1.21489375e-03 4.20093667e-03
3.64183975e-02 -2.70566105e-03 1.76865186e-02 -4.32720328e-04
1.83833456e-03 6.71729366e-03 3.78006425e-03 -1.29479534e-02
-6.70705751e-03 -4.27785521e-03 -7.21097946e-03 4.21828780e-03
-1.14203158e-02]
[-4.48414873e-02 -6.87463587e-02 1.35411494e-02 -6.93410175e-03
-1.49342063e-02 -1.33310349e-03 -5.32773253e-03 2.47821474e-03
-9.93593523e-03 -2.46717612e-04 -9.68411133e-03 4.67410859e-03
2.55456077e-03 -9.86139778e-03 -4.56848597e-03 7.08141059e-03
-1.07708475e-02]
[-7.07037518e-02 4.26174467e-02 -1.38803772e-02 -2.92668410e-03
-1.30301447e-02 -1.01603196e-02 9.75524943e-03 3.74548101e-03
3.88866090e-03 -9.72738112e-03 4.86491845e-03 -3.37795309e-03
1.56065048e-02 1.12536529e-04 1.58877810e-03 -2.32148916e-03
-1.62344213e-03]
[-7.99673387e-04 -6.88926811e-03 5.91311630e-03 -1.21272789e-02
2.71141710e-03 -1.96249299e-03 8.74733027e-01 -2.78978613e-03
3.25261770e-03 1.44845562e-03 2.53036895e-03 -5.09965067e-03
-2.33374550e-03 -1.03569847e-03 6.06843672e-03 4.24396614e-03
-2.57304341e-03]]
ABC.Distance DEBUG: Optimal FD delta: [0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1]
ABC.Distance DEBUG: Info weights[6] = {'y1': 3.9726e+00, 'y2': 3.8846e+00, 'y3:0': 1.0378e+00, 'y3:1': 7.8093e-01, 'y3:2': 1.2633e+00, 'y3:3': 7.1563e-01, 'y4': 2.2445e+00, 'y5:0': 1.5802e-01, 'y5:1': 2.2227e-01, 'y5:2': 2.3051e-01, 'y5:3': 2.2248e-01, 'y5:4': 2.5606e-01, 'y5:5': 2.2768e-01, 'y5:6': 1.7441e-01, 'y5:7': 1.4169e-01, 'y5:8': 1.3541e-01, 'y5:9': 3.3218e-01}
ABC INFO: t: 6, eps: 1.42162817e+01.
ABC INFO: Accepted: 1000 / 4370 = 2.2883e-01, ESS: 4.7835e+02.
ABC.Distance DEBUG: Scale weights[7] = {'y1': 5.0399e-01, 'y2': 4.1726e-03, 'y3:0': 5.8194e-03, 'y3:1': 5.8915e-03, 'y3:2': 5.7821e-03, 'y3:3': 5.7917e-03, 'y4': 6.6940e+00, 'y5:0': 1.4639e-01, 'y5:1': 1.4925e-01, 'y5:2': 1.4917e-01, 'y5:3': 1.4544e-01, 'y5:4': 1.4720e-01, 'y5:5': 1.5076e-01, 'y5:6': 1.4693e-01, 'y5:7': 1.4707e-01, 'y5:8': 1.4493e-01, 'y5:9': 1.4968e-01}
ABC INFO: t: 7, eps: 1.31240624e+01.
ABC INFO: Accepted: 1000 / 3994 = 2.5038e-01, ESS: 7.5321e+02.
ABC.Distance DEBUG: Scale weights[8] = {'y1': 9.0430e-01, 'y2': 6.0646e-03, 'y3:0': 5.3165e-03, 'y3:1': 5.3546e-03, 'y3:2': 5.2800e-03, 'y3:3': 5.1171e-03, 'y4': 6.4189e+00, 'y5:0': 1.4621e-01, 'y5:1': 1.4882e-01, 'y5:2': 1.5259e-01, 'y5:3': 1.4952e-01, 'y5:4': 1.4771e-01, 'y5:5': 1.4516e-01, 'y5:6': 1.5138e-01, 'y5:7': 1.4365e-01, 'y5:8': 1.4591e-01, 'y5:9': 1.4701e-01}
ABC INFO: t: 8, eps: 1.33303433e+01.
ABC INFO: Accepted: 1000 / 4567 = 2.1896e-01, ESS: 7.5763e+02.
ABC.Distance DEBUG: Scale weights[9] = {'y1': 1.2052e+00, 'y2': 6.9590e-03, 'y3:0': 5.5645e-03, 'y3:1': 5.4895e-03, 'y3:2': 5.3951e-03, 'y3:3': 5.5151e-03, 'y4': 6.8727e+00, 'y5:0': 1.5124e-01, 'y5:1': 1.4969e-01, 'y5:2': 1.4966e-01, 'y5:3': 1.4703e-01, 'y5:4': 1.4538e-01, 'y5:5': 1.4690e-01, 'y5:6': 1.4796e-01, 'y5:7': 1.4861e-01, 'y5:8': 1.4932e-01, 'y5:9': 1.4850e-01}
ABC INFO: t: 9, eps: 1.29931588e+01.
ABC INFO: Accepted: 1000 / 4872 = 2.0525e-01, ESS: 6.8956e+02.
ABC.Distance DEBUG: Scale weights[10] = {'y1': 1.8756e+00, 'y2': 8.2983e-03, 'y3:0': 5.4316e-03, 'y3:1': 5.4255e-03, 'y3:2': 5.4896e-03, 'y3:3': 5.3516e-03, 'y4': 6.6145e+00, 'y5:0': 1.4891e-01, 'y5:1': 1.4981e-01, 'y5:2': 1.4863e-01, 'y5:3': 1.4950e-01, 'y5:4': 1.5153e-01, 'y5:5': 1.4800e-01, 'y5:6': 1.4474e-01, 'y5:7': 1.4879e-01, 'y5:8': 1.4695e-01, 'y5:9': 1.4912e-01}
ABC INFO: t: 10, eps: 1.26075916e+01.
ABC INFO: Accepted: 1000 / 6034 = 1.6573e-01, ESS: 4.1603e+02.
ABC.Distance DEBUG: Scale weights[11] = {'y1': 2.7631e+00, 'y2': 8.9946e-03, 'y3:0': 5.2964e-03, 'y3:1': 5.3974e-03, 'y3:2': 5.3363e-03, 'y3:3': 5.3291e-03, 'y4': 6.6998e+00, 'y5:0': 1.4870e-01, 'y5:1': 1.4891e-01, 'y5:2': 1.4816e-01, 'y5:3': 1.5172e-01, 'y5:4': 1.5236e-01, 'y5:5': 1.5108e-01, 'y5:6': 1.4806e-01, 'y5:7': 1.4726e-01, 'y5:8': 1.4946e-01, 'y5:9': 1.4760e-01}
ABC INFO: t: 11, eps: 1.19787516e+01.
ABC INFO: Accepted: 1000 / 8894 = 1.1244e-01, ESS: 7.2872e+02.
ABC.Distance DEBUG: Scale weights[12] = {'y1': 4.2505e+00, 'y2': 8.7372e-03, 'y3:0': 5.0485e-03, 'y3:1': 5.2558e-03, 'y3:2': 5.0977e-03, 'y3:3': 5.0384e-03, 'y4': 6.3470e+00, 'y5:0': 1.4950e-01, 'y5:1': 1.4874e-01, 'y5:2': 1.4968e-01, 'y5:3': 1.4879e-01, 'y5:4': 1.4772e-01, 'y5:5': 1.5010e-01, 'y5:6': 1.4699e-01, 'y5:7': 1.4370e-01, 'y5:8': 1.5178e-01, 'y5:9': 1.4991e-01}
ABC INFO: t: 12, eps: 1.10735248e+01.
ABC INFO: Accepted: 1000 / 9583 = 1.0435e-01, ESS: 3.1046e+02.
ABC.Distance DEBUG: Scale weights[13] = {'y1': 6.0975e+00, 'y2': 9.5923e-03, 'y3:0': 5.5449e-03, 'y3:1': 5.4214e-03, 'y3:2': 5.4842e-03, 'y3:3': 5.4284e-03, 'y4': 6.8227e+00, 'y5:0': 1.4732e-01, 'y5:1': 1.4950e-01, 'y5:2': 1.4835e-01, 'y5:3': 1.4972e-01, 'y5:4': 1.4826e-01, 'y5:5': 1.4887e-01, 'y5:6': 1.4885e-01, 'y5:7': 1.5098e-01, 'y5:8': 1.4742e-01, 'y5:9': 1.4688e-01}
ABC INFO: t: 13, eps: 1.08800857e+01.
ABC INFO: Accepted: 1000 / 17057 = 5.8627e-02, ESS: 6.3732e+02.
ABC.Distance DEBUG: Scale weights[14] = {'y1': 7.5116e+00, 'y2': 8.9793e-03, 'y3:0': 5.2835e-03, 'y3:1': 5.2556e-03, 'y3:2': 5.2562e-03, 'y3:3': 5.2838e-03, 'y4': 6.7129e+00, 'y5:0': 1.5036e-01, 'y5:1': 1.4885e-01, 'y5:2': 1.4743e-01, 'y5:3': 1.4917e-01, 'y5:4': 1.4989e-01, 'y5:5': 1.4812e-01, 'y5:6': 1.4792e-01, 'y5:7': 1.4949e-01, 'y5:8': 1.4977e-01, 'y5:9': 1.5026e-01}
ABC INFO: Stop: Total simulations budget.
ABC.History INFO: Done <ABCSMC id=6, duration=0:01:37.675742, end_time=2022-03-01 22:35:51>
CPU times: user 4min 20s, sys: 20.9 s, total: 4min 41s
Wall time: 11min 23s
While overall all approaches would benefit from a continued analysis, the approaches L1+Ada.+MAD+StatLR+P4 and L1+Ada.+MAD+SensiLR+P4 employing scale normalization, accounting for informativeness, and using augmented regression targets, approximate the true posterior distribution best. Using only scale normalization captures the overall dynamics, however gives large uncertainties, as unnecessary emphasis is put on \(y_5\). Approaches only using \(\theta\) as regression targets however fail to capture the dynamics of \(\theta_4\), as the regression model cannot unravel a meaningful relationship between data and parameters.
[12]:
fig, axes = plt.subplots(ncols=len(prior_bounds), figsize=(16, 4))
# plot ground truth
for i_par, par in enumerate(gt_par.keys()):
# define parameter-simulation transformation
p_to_y = lambda p: p
if par == "p4":
p_to_y = lambda p: p**2
# observed data corresponding to parameter
y_obs = p_to_y(gt_par[par])
# bounds
xmin, xmax = prior_bounds[par]
# standard deviation
sigma = sigmas[par]
# pdf as function of only p
pdf = partial(
unnorm_1d_normal_pdf,
y_obs=y_obs,
sigma=sigma,
p_to_y=p_to_y,
)
# integrate density
norm = sp.integrate.quad(pdf, xmin, xmax)[0]
# plot density
xs = np.linspace(xmin, xmax, 300)
axes[i_par].plot(
xs,
pdf(xs) / norm,
linestyle="dashed",
color="grey",
label="ground truth",
)
# plot ABC approximations
for i_par, par in enumerate(prior_bounds.keys()):
for distance_id, h in zip(distances.keys(), hs):
pyabc.visualization.plot_kde_1d_highlevel(
h,
x=par,
xname=par,
xmin=prior_bounds[par][0],
xmax=prior_bounds[par][1],
ax=axes[i_par],
label=distance_id,
kde=pyabc.GridSearchCV() if par == "p4" else None,
numx=500,
)
# prettify
for ax in axes[1:]:
ax.set_ylabel(None)
fig.tight_layout(rect=(0, 0.1, 1, 1))
axes[-1].legend(
bbox_to_anchor=(1, -0.2), loc="upper right", ncol=len(distances) + 1
)
ABC.Transition INFO: Best params: {'scaling': 0.2875}
ABC.Transition INFO: Best params: {'scaling': 0.05}
ABC.Transition INFO: Best params: {'scaling': 0.05}
ABC.Transition INFO: Best params: {'scaling': 0.05}
ABC.Transition INFO: Best params: {'scaling': 0.05}
ABC.Transition INFO: Best params: {'scaling': 0.2875}
[12]:
<matplotlib.legend.Legend at 0x7f51d3e90fd0>
While the scale weights accurately depict the scales the various model output types vary on, the sensitivity weights are high for \(y_1\) through \(y_4\), with low weights assigned to \(y_5\). Very roughly, the sum of sensitivity weights for the four model outputs \(y_3\) is roughly equal to e.g. the sensitivity weight assigned to \(y_2\), as desirable. However, the weights assigned are now completely homogeneous, indicating that an increased training sample or more complex regression model may be preferable.
[13]:
# plot weights
fig, axes = plt.subplots(nrows=2, ncols=1, figsize=(8, 8))
# scale weights
scale_distance_ids = [
distance_id
for distance_id in distances.keys()
if "Ada." in distance_id and "Stat" not in distance_id
]
scale_log_files = []
for i_dist, distance_id in enumerate(scale_distance_ids):
scale_log_files.append(f"{scale_log_file}_{distance_id}.json")
pyabc.visualization.plot_distance_weights(
scale_log_files,
labels=scale_distance_ids,
colors=[colors[distance_id] for distance_id in scale_distance_ids],
xlabel="Model output",
title="Scale weights",
ax=axes[0],
keys=dict2arrlabels(data, keys=data.keys()),
)
# info weights
info_distance_ids = [
distance_id for distance_id in distances.keys() if "Sensi" in distance_id
]
info_log_files = []
for i_dist, distance_id in enumerate(info_distance_ids):
info_log_files.append(f"{info_log_file}_{distance_id}.json")
pyabc.visualization.plot_distance_weights(
info_log_files,
labels=info_distance_ids,
colors=[colors[distance_id] for distance_id in info_distance_ids],
xlabel="Model output",
title="Sensitivity weights",
ax=axes[1],
keys=dict2arrlabels(data, keys=data.keys()),
)
fig.tight_layout()
[14]:
# plot flow diagram
fig = pyabc.visualization.plot_sensitivity_sankey(
info_sample_log_file=f"{info_sample_log_file}_L1+Ada.+MAD+SensiLR+P4",
t=f"{info_log_file}_L1+Ada.+MAD+SensiLR+P4.json",
h=hs[-1],
predictor=LinearPredictor(),
par_trafo=ParTrafo(trafos=par_trafos, trafo_ids=trafo_ids),
height=900,
)
# here just showing a non-interactive plot to reduce storage
img_file = tempfile.mkstemp(suffix=(".svg"))[1]
fig.write_image(img_file)
display(SVG(img_file))
ABC.Predictor INFO: Fitted <LinearPredictor predictor=LinearRegression()> in 0.02s
ABC.Predictor INFO: Pearson correlations: 0.999 0.953 0.850 0.124 0.081 0.077 0.082 0.925 0.867 0.827 0.683 0.121 0.079 0.086 0.094 0.875
ABC.Predictor DEBUG: Linear regression coefficients (n_target, n_feature):
[[ 9.99467799e-01 -6.08456786e-05 -5.96816711e-05 -9.77668494e-05
2.39617680e-04 -8.92999377e-05 4.22410927e-05 1.69655282e-04
-4.44860924e-05 1.86195873e-04 5.05596525e-04 3.79109863e-04
1.75463461e-04 -7.70261189e-05 4.95800243e-04 -1.72393054e-04
-2.52460553e-04]
[-3.94005909e-03 9.52625808e-01 -1.51758187e-03 -1.29099679e-03
2.78521627e-03 -7.00666584e-03 1.34386758e-04 -1.61199310e-03
2.52300175e-03 5.72280693e-03 -3.80291728e-03 6.88461560e-04
-2.19637490e-03 -2.95666180e-05 3.04181510e-03 6.69539045e-03
3.15937444e-05]
[ 1.50406124e-02 -1.19775493e-02 2.94759912e-01 2.89398576e-01
2.83188362e-01 2.84292226e-01 -1.79451966e-03 3.10644248e-03
-7.93030482e-03 -5.34238685e-03 3.36709745e-03 -5.27885998e-04
1.33833095e-03 -4.90404382e-03 4.91586834e-03 2.48544435e-03
6.65773906e-03]
[-4.29727247e-02 6.78809588e-02 -3.75164640e-02 -2.73419876e-02
-2.94059185e-02 -2.33241195e-02 6.92171987e-03 5.98383634e-03
-7.92166171e-03 -7.71487811e-03 -4.53864121e-03 -1.02870134e-02
2.72949986e-03 -1.61452844e-03 -1.35030681e-03 -5.27953011e-03
8.75565996e-03]
[-5.46831600e-02 -3.89667658e-02 4.51058656e-03 1.49600678e-03
3.59513775e-02 4.19507023e-04 2.17130079e-02 7.94630106e-03
-2.38982129e-03 6.63414175e-03 7.16806574e-03 -1.08874688e-02
-4.08254398e-03 -5.91218983e-03 -9.18105497e-04 2.66562589e-03
-1.11885793e-02]
[-4.33170926e-02 -5.84786903e-02 1.48555950e-02 -6.83098761e-03
-1.52933134e-02 1.51730784e-03 -3.75059409e-03 8.90618583e-05
-1.00873073e-02 -4.43976636e-03 -6.69733662e-03 2.65948070e-03
3.08437689e-03 -9.78588767e-03 -5.56294130e-03 2.38408934e-03
-1.15231963e-02]
[-6.87404238e-02 2.61986061e-02 -1.28884618e-02 -4.03627600e-03
-1.42847406e-02 1.63582062e-03 6.37321997e-03 4.91841683e-03
5.33362087e-03 -6.48723330e-03 5.54300308e-03 -3.21128457e-03
9.90302942e-03 2.87963380e-04 -1.66737282e-03 -1.72875830e-03
-6.80467921e-03]
[ 1.42623400e-03 -2.56332770e-03 3.46320867e-03 -6.06217730e-03
-9.62905690e-04 2.54571955e-03 9.24946371e-01 -3.42009163e-03
3.31886338e-04 1.49397518e-03 2.69706465e-03 -2.81370032e-03
-2.28303040e-03 -3.92370621e-03 4.12718816e-03 3.91637455e-03
-6.37193916e-04]
[ 8.67776728e-01 1.76252848e-02 -1.59609770e-02 -1.10021771e-03
-7.97046781e-03 -2.19411708e-03 1.67970973e-02 8.36228086e-03
-3.09691241e-04 -3.52425528e-03 -1.67665097e-03 5.33739631e-03
-3.20494515e-03 2.49444727e-03 6.48756694e-03 9.21098220e-04
2.02104810e-03]
[ 1.09887225e-02 8.26910701e-01 3.43621313e-03 -1.17949386e-03
-1.13712786e-03 5.32917668e-04 1.57859662e-02 4.16087998e-04
3.24019382e-03 8.08376849e-03 -8.69719681e-04 -2.08583364e-03
2.57782439e-03 -5.88183165e-03 1.34058318e-03 9.40357644e-04
1.49951536e-03]
[ 9.26484417e-03 -2.28874913e-02 2.31666085e-01 2.36306181e-01
2.31689570e-01 2.23534208e-01 -2.67149060e-02 4.46423506e-03
-1.52495534e-03 -6.65712901e-03 7.22358909e-04 -1.07947284e-03
6.94144197e-04 -7.03428292e-03 -3.75406598e-03 -2.87239767e-03
3.08042348e-03]
[-5.30008965e-02 6.90398917e-02 -3.46661020e-02 -1.86209478e-02
-2.68786349e-02 -2.19446837e-02 3.65099894e-03 5.90193471e-03
-5.48885782e-03 -3.09419571e-03 -3.62972114e-03 -7.84707926e-03
1.03394692e-03 -1.00174257e-03 -2.03988489e-03 -5.74044890e-04
1.06075451e-02]
[-5.15345592e-02 -4.19702203e-02 1.21489375e-03 4.20093667e-03
3.64183975e-02 -2.70566105e-03 1.76865186e-02 -4.32720328e-04
1.83833456e-03 6.71729366e-03 3.78006425e-03 -1.29479534e-02
-6.70705751e-03 -4.27785521e-03 -7.21097946e-03 4.21828780e-03
-1.14203158e-02]
[-4.48414873e-02 -6.87463587e-02 1.35411494e-02 -6.93410175e-03
-1.49342063e-02 -1.33310349e-03 -5.32773253e-03 2.47821474e-03
-9.93593523e-03 -2.46717612e-04 -9.68411133e-03 4.67410859e-03
2.55456077e-03 -9.86139778e-03 -4.56848597e-03 7.08141059e-03
-1.07708475e-02]
[-7.07037518e-02 4.26174467e-02 -1.38803772e-02 -2.92668410e-03
-1.30301447e-02 -1.01603196e-02 9.75524943e-03 3.74548101e-03
3.88866090e-03 -9.72738112e-03 4.86491845e-03 -3.37795309e-03
1.56065048e-02 1.12536529e-04 1.58877810e-03 -2.32148916e-03
-1.62344213e-03]
[-7.99673387e-04 -6.88926811e-03 5.91311630e-03 -1.21272789e-02
2.71141710e-03 -1.96249299e-03 8.74733027e-01 -2.78978613e-03
3.25261770e-03 1.44845562e-03 2.53036895e-03 -5.09965067e-03
-2.33374550e-03 -1.03569847e-03 6.06843672e-03 4.24396614e-03
-2.57304341e-03]]
ABC.Distance DEBUG: Optimal FD delta: [0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1]
This was a little introduction to approaches accounting for heterogeneous data scales by adaptive scale normalization, and data informativeness by using regression models to either construct low-dimensional summary statistics, or inform sensitivity weights. Beyond linear regression employed in this notebook, various other regression methods are possible, including e.g. Gaussian processes or neural networks to capture non-linear relationships better. These are also implemented in pyABC.
It should be noted that the use of regression models may be sensitive to in particular training sample size, especially for more complex regression models, such that some care may need to be taken there. For example, also model selection with out-of-sample validation is provided via ModelSelectionPredictor.