pyabc.external

External simulators

This module can be used to easily interface pyABC with model simulations, summary statistics calculators and distance functions written in programming languages other than Python.

The class pyabc.external.ExternalHandler, as well as derived Model, SumStat, and Distance classes, allow the use of arbitrary languages, with communication via file i/o. It has been successfully used with models written in e.g. R, Java, or C++.

Further, pyABC provides efficient interfaces to R via the class pyabc.external.r.R via the rpy2 package, and to Julia via the class pyabc.external.julia.Julia via the pyjulia package.

class pyabc.external.ExternalDistance(executable: str, file: str, fixed_args: List = None, suffix: str = None, prefix: str = 'dist_', dir: str = None, show_stdout: bool = False, show_stderr: bool = True, raise_on_error: bool = False, timeout: float = None)

Bases: object

Use script and sumstat output files to compute the distance.

Format:

{executable} {file} sumstat_0={sumstat_0} sumstat_1={sumstat_1} target={loc}

The distance is written to a file, which is then read in (it must only contain a single float number).

__call__(sumstat_0, sumstat_1)

Call self as a function.

__init__(executable: str, file: str, fixed_args: List = None, suffix: str = None, prefix: str = 'dist_', dir: str = None, show_stdout: bool = False, show_stderr: bool = True, raise_on_error: bool = False, timeout: float = None)

class pyabc.external.ExternalHandler(executable: str, file: str = None, fixed_args: List = None, create_folder: bool = False, suffix: str = None, prefix: str = None, dir: str = None, show_stdout: bool = False, show_stderr: bool = True, raise_on_error: bool = False, timeout: float = None)

Bases: object

Handler for calls to external scripts.

This class bundles repeated functionality.

__init__(executable: str, file: str = None, fixed_args: List = None, create_folder: bool = False, suffix: str = None, prefix: str = None, dir: str = None, show_stdout: bool = False, show_stderr: bool = True, raise_on_error: bool = False, timeout: float = None)

Parameters:

• executable – Name of the executable to call, e.g. bash, java or Rscript. The executable may be parameterized, e.g. appearances of {loc} in the string are replaced at runtime by the location of the output.

• file – Path to the file to be executed, e.g. a .sh, .java or .r file, or also a .xml file depending on the executable.

• fixed_args – Argument string to use every time.

• create_folder – Whether the function should create a temporary directory. If False, only one temporary file is created.

• suffix – Suffix, prefix, and base directory for the created temporary files.

• prefix – Suffix, prefix, and base directory for the created temporary files.

• dir – Suffix, prefix, and base directory for the created temporary files.

• show_stdout – Whether to show or hide the stdout and stderr streams.

• show_stderr – Whether to show or hide the stdout and stderr streams.

• raise_on_error – Whether to raise when an error in the execution of the external script occurs, or just continue.

• timeout – Maximum execution time in seconds, after which the executable is stopped.

create_executable(loc)

Parse and return executable.

Replaces instances of {loc} by the location loc.

create_loc()

Create temporary file or folder.

Returns:

loc – Path of the created file or folder.

Return type:

str

run(args: List[str] = None, cmd: str = None, loc: str = None)

Run the script for the given arguments.

Parameters:

• args (List[str], optional) – Arguments to pass to the external program, e.g. parameters.

• cmd (str, optional) – If this is not None, then it is assumed to contain the full command to be executed via the shell (then args is ignored). Be aware of possible security implications of shell injection.

• loc (str, optional) – Location for the output. If None is passed, one is created.

class pyabc.external.ExternalModel(executable: str, file: str, fixed_args: List = None, create_folder: bool = False, suffix: str = None, prefix: str = 'modelsim_', dir: str = None, show_stdout: bool = False, show_stderr: bool = True, raise_on_error: bool = False, timeout: float = None, name: str = 'ExternalModel')

Bases: Model

Interface to a model that is called via an external simulator.

Parameters are passed to the model as named command line arguments in the form

{executable} {file} {par1}={val1} {par2}={val2} … target={loc}

Here, {file} is the script that performs the model simulation, and {loc} is the name of a temporary file or folder that was created to store the simulated data.

Note

The generated temporary files are not automatically deleted, unless by the system e.g. in the /tmp directory upon restart.

__call__(pars: Parameter)

Call self as a function.

__init__(executable: str, file: str, fixed_args: List = None, create_folder: bool = False, suffix: str = None, prefix: str = 'modelsim_', dir: str = None, show_stdout: bool = False, show_stderr: bool = True, raise_on_error: bool = False, timeout: float = None, name: str = 'ExternalModel')

Initialize the model.

Parameters:

• name (str, optional (default = "ExternalModel")) – As in pyabc.Model.name.

• ExternalHandler. (All other parameters as in) –

eval_param_limits(limits)

evaluate single parameter’s boundary value on computation time.

Parameters:

limits (dict) – the lower and upper boundary values of parameters. The key would be the parameter name and the value would be a list of the lower and upper limit of parameter value, e.g., [lower, upper].

Returns:

time_eval_dict – a dictionary that contains the parameter names as key and a list as a value. The list contains the computation time when using lower and upper limits, e.g., [lower, upper].

Return type:

dict

eval_param_limits_matrix(limits)

evaluate two paramters’ boundary values on computation time.

Parameters:

limits (dict) – the lower and upper boundary values of parameters. The key would be the parameter name and the value would be a list of the lower and upper limit of parameter value, e.g., [lower, upper].

Returns:

• time_eval_mat_df_lower (df) –

  a dataframe for the computation time measured when using the lower

  limit value of parameters.

• time_eval_mat_df_upper (df) –

  a dataframe for the computation time measured when using the upper

  limit value of parameters.

sample(pars)

Return a sample from the model evaluated at parameters pars. This can be raw data, or already summarized statistics thereof.

This method has to be implemented by any subclass.

Parameters:

pars (Parameter) – Dictionary of parameters.

Returns:

sample – The sampled data.

Return type:

any

class pyabc.external.ExternalSumStat(executable: str, file: str, fixed_args: List = None, create_folder: bool = False, suffix: str = None, prefix: str = 'sumstat_', dir: str = None, show_stdout: bool = False, show_stderr: bool = True, raise_on_error: bool = False, timeout: float = None)

Bases: object

Interface to an external calculator that takes the simulated model output and writes to file the summary statistics.

Format:

{executable} {file} model_output={model_output} target={loc}

Here, {file} is the path to the summary statistics computation script, {model_output} is the path to the previously generated model output, and {loc} is the destination to write te summary statistics to.

__call__(model_output)

Create summary statistics from the model_output generated by the model.

__init__(executable: str, file: str, fixed_args: List = None, create_folder: bool = False, suffix: str = None, prefix: str = 'sumstat_', dir: str = None, show_stdout: bool = False, show_stderr: bool = True, raise_on_error: bool = False, timeout: float = None)

pyabc.external.create_sum_stat(loc: str = '', returncode: int = 0)

Create a summary statistics dictionary, as returned by the ExternalModel.

Can be used to encode the measured summary statistics, or also create a dummy summary statistic.

Parameters:

• loc (str, optional (default = '')) – Location of the summary statistics file or folder.

• returncode (int, optional (default = 0)) – Defaults to 0, indicating correct execution. Should usually not be changed.

Returns:

• A dictionary with keys ‘loc’ and ‘returncode’ of the given

• parameters.