Reference

DeepPumas.DataDrivenDomain Method

julia

DataDrivenDomain(
    basis::DataDrivenDiffEq.Basis,
    output::Int64;
    ...
) -> DataDrivenDomain{Float64, B, _A, _B, Nothing} where {B<:DataDrivenDiffEq.Basis, _A, _B}
DataDrivenDomain(
    basis::DataDrivenDiffEq.Basis,
    output::Int64,
    prior::Union{Nothing, Prior};
    ...
) -> DataDrivenDomain{Float64}
DataDrivenDomain(
    basis::DataDrivenDiffEq.Basis,
    output::Int64,
    prior::Union{Nothing, Prior},
    coeff_type::Type{T};
    rng,
    init,
    kwargs...
) -> DataDrivenDomain

Constructs a DataDrivenDomain from a given Basis with output dimensions output. The argument output defines the number of output variables, the argument prior defines the (sparsifying) prior to use. Additionally, the eltype of the coefficients can be provided via via coeff_type which defaults to Float64. Initial values can be provided via the keyword init in the form of either an AbstractVector of length output*length(basis) or as an AbstractMatrix with size (output, length(basis).

DeepPumas.DataDrivenDomain Type

A parameter which represents a linear combination of a library of candidate functions.

julia

struct DataDrivenDomain{T, M<:DataDrivenDiffEq.AbstractBasis, O, L, P, V} <: Pumas.Domain

The DataDrivenDomain enables sparse identification within DeepPumas. It allows the user to wrap a Basis inside a PumasModel and identify the coefficients, penalized via a sufficient prior.

Fields

basis: The candidate functions, represented by a Basis
prior: The (elementwise) Prior of the model.
init: Possible initial values for the coefficients.

Possible Priors

Additionally, a MixtureModel can be used to implement different priors, e.g. the SpikeNSlab prior via:

julia

MixtureModel(Normal[Normal(0.0, 10.0), Normal(0.0, 1e-5)])

Note

DataDrivenDomain is a wrapper type for a DataDrivenDiffEq.Basis, which assumes that all randomeffects are modeled as control variables of the Basis.

Methods

DeepPumas.GridSearch Type

julia

GridSearch(bounds=(; lambda=(1e-7, 1e2)); kwargs...)

Return a GridSearch which specifies hyperopt search space and options.

Keyword arguments

criteria = Pumas.KFold(K = 5) - The evaluation criteria for the hyperopt. Defaults to a K-fold cross-validation.
scale::Union{Symbol,NamedTuple} = :log10 - Specify how to divide up the parameter search space. A single symbol applies to all bounds, while a named tuple can specify the scale independently for different hyperparameters - e.g., scale = (; lambda=:log10). Valid options are :linear, :log, and :log10.
resolution::Union{Int,NamedTuple} = 20 - The resolution at which the search space is divided. An Int specifies the total number of search points while a named tuple specifies the number of search points per hyperparameter and the total number of search points is the product of those.
ensemblealg = EnsembleThreads() - Specify how to parallelize the search. Available options are EnsembleSerial() for non-paralellized, EnsembleThreads for threaded (within a single CPU), and EnsembleDistributed for distributed (across CPUs).
verbosity = 1 - Specify the verbosity of the printout. Set to 0 to disable output.
optim_alg = nothing - Specify the optimization algorithm. Only Adam is supported for all neural networks backends. Other algorithms from Optimisers.jl can be used, but only when calling hyperopt on a Flux-based machine learning model. If nothing is passed then a fresh hyperopt will use Adam while a continued optimization (where you've passed a HyperoptResult) will re-use the previous algorithm.
optim_options - A NamedTuple of options to be passed on to the fitting. Valid options:
- rng - a random number generator.
- loss - the loss function, available options are l1 (default) and l2.
- lambda - the regularization constant. Overridden if lambda is part of the hyperparameters to optimize. Defaults to 0 or to any regularization or prior specified in the model passed to hyperopt.
- alpha - the regularization mode. 0 (default) for L2 regularization and 1 for L1.
- verbosity - tune the amount of printouts during hyperoptimization. Defaults to 1. Set to 0 to avoid printouts.
- batch_size - Specify the batch sizes for the optimization algorithm (typically Adam). Batching determines how many data points to use each time the gradient is calculated and a step in parameter space is taken. Batching introduces an often beneficial stochasticity in the optimization. This is good for avoiding local optima but can be more costly compared to full batch for the same number of epochs. Defaults to including all data in one batch.
- epochs - the number of iterations to optimize for. Defaults to 10_000.

DeepPumas.HoldoutValidation Type

julia

HoldoutValidation(training_fraction; kwargs...)

Arguments

training_fraction - the fraction of the data that should be included in the training. The remainder is withheld for validation.

Keyword arguments

shuffle = true
optim_alg = nothing - Specify the optimization algorithm. Only Adam is supported for all neural network backends. Other algorithms from Optimisers.jl can be used when calling fit on Flux-based machine learning models. If nothing is passed then a fresh hyperopt will use Adam while a continued optimization (where you've passed a HyperoptResult) will re-use the previous algorithm.
optim_options - A NamedTuple of options to be passed on to the fitting. Valid options:
- rng - a random number generator.
- loss - the loss function, available options are l1 (default) and l2.
- lambda - the regularization constant. Defaults to 0 or to any regularization or prior specified in the model passed to fit.
- alpha - the regularization mode. 0 for L2 regularization and 1 for L1. Defaults to 0 or to any regularization or prior specified in the model passed to fit.
- verbosity - tune the amount of printouts during fitting. Defaults to 1. Set to 0 to avoid printouts.
- batch_size - Specify the batch sizes for the optimization algorithm (typically Adam). Batching determines how many data points to use each time the gradient is calculated and a step in parameter space is taken. Batching introduces an often beneficial stochasticity in the optimization. This is good for avoiding local optima but can be more costly compared to full batch for the same number of epochs. Defaults to including all data in one batch.
- epochs - the number of iterations to optimize for. Defaults to 10_000.

DeepPumas.L1 Type

julia

L1(λ=1; input=true, output=true, bias=true)

Construct an L1 regularization type.

Adds a regularization of λ⋅∑ⱼ|βⱼ| where βⱼ is a regularized parameter and λ is a regularization constant. Such a regularization is proportional to having a Laplace(0., 1/λ) prior.

λ::Number - the regularization constant.
input::Bool - toggle regularization of the input layer.
output::Bool - toggle regularization of the output layer.
bias::Bool - toggle regularization of bias parameters.

The regularization of the input and output layer bias parameters follow an AND rule where they're only regularized if input==true AND bias==true.

For now, the input, output and bias specifications are only respected during the fitting of a full PumasModel, not during hyperoptimization or fitting in the augment workflow where everything is regularized.

DeepPumas.L2 Type

julia

L2(λ=1; input=true, output=true, bias=true)

Construct an L2 regularization type.

Adds a regularization of λ⋅∑ⱼ(βⱼ²) where βⱼ is a regularized parameter and λ is a regularization constant. Such a regularization is proportional to having a Normal(0., 1/√(2λ)) prior.

λ::Number - the regularization constant.
input::Bool = true - toggle regularization of the input layer.
output::Bool = true - toggle regularization of the output layer.
bias::Bool = false - toggle regularization of bias parameters.

The regularization of the input and output layer bias parameters follow an AND rule where they're only regularized if input==true AND bias==true.

For now, the input, output and bias specifications are only respected during the fitting of a full PumasModel, not during hyperoptimization or fitting in the augment workflow where everything is regularized.

DeepPumas.Prior Type

julia

Prior(distribution; input=true, output=true, bias=true)

Specify a prior parameter distribution for a neural network embedded in a PumasModel.

Arguments:

distribution::UnivariateDistribution - the prior distribution according to which the neural network weights will be regularized
input::Bool = true - toggle regularization of the input layer
output::Bool = true - toggle regularization of the output layer
bias::Bool = true - toggle regularization of bias parameters

For now, the input, output and bias specifications are only respected during the fitting of a full PumasModel, not during hyperoptimization or fitting in the augment workflow where everything is regularized.

Examples:

julia

using Distributions

d1 = Normal(0.0, 1.0)
prior_1 = Prior(d1)

d2 = Uniform(0.0, 1.0)
prior_2 = Prior(d2; output=false, bias=false)

DeepPumas.MLPDomain Method

julia

MLPDomain(input, layers...; kwargs...)

Construct a, possibly regularized, MLP.

Arguments

input::Int - the number of inputs to the neural network.
layers... - Each additional argument after input specifies a new neural network layer. Layers can either be specified by an Int specifying the length of the layer or by a Tuple (length, [act,] [bias]) which allows you to optionally specify an activation function, act, and whether the layer should have bias parameters, bias. Such a tuple of inputs overrides the default values that are passed as keyword arguments to MLPDomain. The output layer should typically be different from the rest - the identity activation function often makes sense and bias may be undesirable.

Keyword arguments

act::Function = tanh - the default activation function for the layers. Can be overridden by tuples in layers.... tanh has proven effective for small and shallow MLPDomains but relu and leakyrelu are the go-to activation functions for anything deeper.
bias::Bool = true - the default specification of whether layers should have bias parameters. Can be overridden by tuples in layers....
reg::Union{Nothing, L1, L2, Prior} = nothing - specify what, if any, regularization to use. Available options are L1, L2, a Prior, or nothing for no regularization. When the MLPDomain is embedded in a PumasModel then the regularization is equivalent to a prior on the neural network weights and will only be applied during fitting when using an algorithm that supports parameter priors, such as MAP or JointMAP.
backend::Symbol=:flux - specify which machine learning package to use in the backend. Available options are (:flux, :simplechains, :staticflux). :flux is flexible and stable. :staticflux is much faster for small networks on a CPU but can take a long time to compile for anything too large. :simplechains is fast but should be considered experimental and is not fully featured. Defaults to :flux unless the default has been modified using set_mlp_backend.

Examples

julia

MLPDomain(3, 5, 1)

3 inputs, a hidden layer of length 5 and an output layer of length 1. All using tanh activation, all with bias parameters, and none of the parameters regularized.

julia

MLPDomain(3, 5, 5, (1, identity, false); reg = L1(1.))

3 inputs, two hidden layers of length 5 with tanh activation and bias parameters, an output layer with a single output with no activation function and no bias. All weights have an L1 regularization when fitted with a prior supporting fitting function such as Pumas.MAP.

julia

MLPDomain(3, (5, tanh, true), 5, (1, identity); act = softplus, bias=false, backend=:staticflux, reg=Prior(Normal(0., 1.), output=false))

Construct a multi-layer perceptron that is equivalent to a Flux model, but backed by StaticArrays.MArrays with 3 inputs, one hidden layer of length 5 with tanh activation and bias parameters (as specified in the tuple), one hidden layer of length 5 with softplus activation and no bias parameters (as defined in act and bias), and one output layer of length 1 with no activation function (identity in the tuple) and no bias (bias kwarg). All weights except those of the output layer have a Normal(0, 1) prior when fitted with an algorithm that supports parameter priors.

julia

target = preprocess(fpm)
nn = MLPDomain(numinputs(target), 4, (numoutputs(target), identity))

Construct an MLPDomain to match the input-output dimension of a target for the augment workflow. See numinputs, numoutputs.

DeepPumas.NeuralDomain Function

julia

NeuralDomain(model, prior=nothing, init=init_params(model))

Construct a Domain with a neural network, associated parameter priors and initial values.

See also MLPDomain, which constructs the same kind of object with simpler syntax while covering the most common use-cases.

Arguments

ml - a FluxModel machine learning model.
prior::Union{Prior, Nothing} = nothing - the prior distribution of neural network parameters used for regularization (see Setting up Priors & Regularization). nothing means that no prior/regularization is applied.
init::Vector = init_params(ml) - the initial parameters of the resulting domain. Defaults to seeded (deterministic) Glorot Normal distributed values of the weights and 0 for biases. See init_params, sample_params.

Example

using DeepPumas
using Flux

NeuralDomain(
  Chain(
    Dense(5, 5),
    Dense(5, 5)
  ),
  Prior(Laplace(0., 1.); bias=false),
  rand(80)
)

DeepPumas.PresetTimeCallback Method

julia

PresetTimeCallback(tstops, user_affect!;
    initialize = DiffEqBase.INITIALIZE_DEFAULT,
    filter_tstops = true,
    kwargs...)

A callback that adds callback affect! calls at preset times. No playing around with tstops or anything is required: this callback adds the triggers for you to make it automatic.

Arguments

tstops: the times for the affect! to trigger at.
user_affect!: an affect!(integrator) function to use at the time points.

Keyword Arguments

filter_tstops: Whether to filter out tstops beyond the end of the integration timespan. Defaults to true. If false, then tstops can extend the interval of integration.

DeepPumas.augment Function

julia

augment(fpm, fitted_ml[, prior][, paramname])

Return an ML-augmented FittedPumasModel.

Insert fitted_ml between the @random and @pre block of the PumasModel in fpm. fitted_ml is evaluated for each Subject and the resulting output is additively combined with the random effects to form the 'augmented random effects' where part of the inter-subject variation is captured by the random effects and part by the fitted_ml. These augmented random effects replace the ordinary random effects used in the @pre, @dosecontrol, @derived and @observed blocks of the fpm model.

fitted_ml will be added as a Domain to the @param block, just like any other parameter, and it will have the name paramname. Since the same name cannot be used twice, you may have to pass a different paramname symbol if there is a conflict.

The resulting model will initialize its parameter values to the fitted coefficients of fpm and the fitted parameter values of fitted_ml such that init_params often results in very good parameters.

The returned model is a normal FittedPumasModel that can be used for any Pumas workflow. Its fitted coefficients are the same of fpm with the addition of the fitted parameter values of fitted_ml, often resulting in a good fit already.

Arguments

fpm::FittedPumasModel - a fitted @model to be augmented.
fitted_ml - the machine learning model to do the augmentation. This fitted_ml could be the output of either fit or hyperopt or AugmentML.
prior::Prior (optional) - specify the prior associated with the parameters of the embedded machine learning model. Defaults to the prior used in fit or the best prior coming out of hyperopt depending on how fitted_ml was created.
paramname::Symbol = :nn - A name to identify the embedded ML within the augmented model.

DeepPumas.cost Function

julia

cost(model, pop, params[, randeffs][, loss]; kwargs...)

Computes the cost of model applied to population pop using params using the loss function loss.

Arguments

model::PumasModel - the model being evaluated.
pop::Population - the Population on which to evaluate the cost.
params::NamedTuple - the parameters of the model.
randeffs::Union{Vector{NamedTuple}, Nothing} = nothing a vector of random effects, indicating particular random effects values for each subject. A value of nothing indicates that, if ipred == true, the empirical Bayes estimates are computed and individual predictions are used for cost calculation.
loss = mae - a function that to computes the distance between a single observation, ŷ and the corresponding prediction, y. See mae and mse for pre-defined functions to calculate mean absolute and mean squared error.

Keyword arguments

field::Union{Symbol, Int} = 1 - the fieldname of the observation for which to calculate the cost. This field should either be a symbol corresponding on one of the observations in your data or an integer picks the observation based on their order in the Subjects.
ipred::Bool - toggle whether to use individual predictions (ipred = true) rather than population predictions (ipred = false). Defaults to false when randeffs is set to nothing and true otherwise.
ytransform - a transformation that is applied to both the data, ŷ, and the prediction y before the cost is calculated. Must be a single-argument function.

DeepPumas.fix Method

julia

fix(f, args...)

Return a functor where the first args... arguments of f are flattened, stored, and fixed.

This is conceptually similar to, but faster than, defining x -> f([args..., x]).

DeepPumas.hyperopt Function

julia

hyperopt(result::HoldoutValidationResult[, alg])

Start a hyperoptimization from where a single fit finished.

The fit target is inherited from result and the hyperoptimization will start all its fits from the parameters in result.

Arguments

result::HoldoutValidationResult - the result of a single fit
alg::HyperoptAlg = GridSearch() - A specification of the hyperoptimization algorithm and options. Currently, only GridSearch is available.

DeepPumas.hyperopt Function

julia

hyperopt(ml::FluxDomain, target::FitTarget[, alg])
hyperopt(ml::FluxModel, target::FitTarget[, alg])

Optimize the parameters and hyperparameters of ml.

Arguments

ml - Typically a FluxDomain resulting from a call to MLPDomain but plain FluxModels are also supported.
target::FitTarget - the target input-output mapping for the optimization. Typically the result of preprocess.
alg (optional) - a specification of how to conduct the hyperparameter optimization. Currently, only GridSearch is supported. Defaults to GridSearch().

Returns

HyperoptResult which can be used to augment a PumasModel (see augment)

DeepPumas.l1 Method

julia

l1(ŷ, y)

An ℓ1 loss specification for use in fitting or hyperparameter optimization.

Results in losses of |ŷ[i] - y[i]| for every index i of the target and model values ŷ and y, respectively.

DeepPumas.l2 Method

julia

l2(ŷ, y)

An ℓ2 loss specification for use in fitting or hyperparameter optimization.

Results in losses of (ŷ[i] - y[i])^2 for every index i of the target and model values ŷ and y, respectively.

DeepPumas.numinputs Method

julia

numinputs(data::Union{Population, Subject}, covs)

Get the number of input nodes a machine learning model needs to process the covariates covs. This flattens and vector or matrix-valued covariates.

Arguments

data::Union{Subject, Population} - data in the form of a Subject or a Population.
covs::Tuple a tuple of the covariate names to use.

DeepPumas.numinputs Method

julia

numinputs(target::FitTarget)

Get the number of inputs in the input-output mapping specified in target.

DeepPumas.numoutputs Method

julia

numoutputs(model, ηs)

Get the flattened length of the specified ηs in a model.

This shows the number of output nodes a machine learning model needs in order to capture subject variability from the specified ηs.

Arguments

model::PumasModel - A model from @model.
ηs::Tuple - a tuple of symbols indicating which random effects to use.

DeepPumas.numoutputs Method

julia

numoutputs(target::FitTarget)

Get the number of outputs in the input-output mapping specified in target.

DeepPumas.plotgrid Method

julia

plotgrid(objs)
plotgrid!(objs)
plotgrid!(plt::Makie.Figure, objs)
plotgrid!(axes::Array{Makie.Axis}, objs)

Plot the objs (Population, results of simobs or predict) in a grid of subplots.

The plot can be customized by keyword arguments. Several of these keyword arguments can take NamedTuples that are passed on as attributes to the Makie.jl functions that do the plotting.

When used with a bang (!), plotgrid! will modify a previous plot. This is useful for plotting both predict and simobs results in the same plot, or plotting predictions from different models in the same subplots.

Arguments:

obsj::Vector A vector of the objects to plot.

Keyword arguments:

add_legend::Bool = true toggle whether to add the plotted objects to the legend.
axis::NamedTuple = (;) (plotgrid only) arguments to be passed to Makies Axis constructor. For full details, see Makie's documentation, but a common example might be axis = (; ylabel = "My Biomarker", xlims=(0., 1.))
data::Union{Bool, NamedTuple} = true toggle whether to plot data markers. NamedTuples will be passed as attributes to Makie.scatter! - see that documentation for full options. Common options are (; markersize=10, color=(:blue, 0.5), label="Trial Name"). Defaults to false for plotgrid!.
dose::Union{Bool, NamedTuple} = true toggle whether to plot vertical lines for doses. NamedTuples will be passed as attributes to Makie.vlines! - see that documentation for full options. Common options are (; linewidth=2, color=(:red, 0.2), linestyle=:dash) Defaults to false for plotgrid!.
figure::NamedTuple = (;) (plotgrid only) arguments to be passed to Makies Figure constructor. For full details, see Makie's documentation, but the most common example might be figure = (; size = (1000,800))
ipred::Union{Bool, NamedTuple} = true (only for plotting results of predict) toggle whether to plot the ipreds. NamedTuples will be passed as attributes to Makie.lines! - see that documentation for full options. Common options are (; linewidth=4, linestyle=:dash, color=(:blue, 0.8), label="My ipred").
layout::Tuple specify how many columns and rows to distribute the plots among. (nrows, ncolumns).
legend::NamedTuple = (;) arguments to be passed to Makies Legend constructor. For full details, see Makie's documentation, but a common example might be legend = (; orientation=:horizontal, framevisible=false)
linkx::Bool = true - link the x-axes of all the subplots. Not applicable with plotgrid!.
linky::Bool = true - link the y-axes of all the subplots. Not applicable with plotgrid!.
observation::Union{Int, Symbol} = 1 specify the variable to use for plotting. You can specify by number or symbol.
pred::Union{Bool, NamedTuple} = true (only for plotting results of predict) toggle whether to plot the ipreds. NamedTuples will be passed as attributes to Makie.lines! - see that documentation for full options. Common options are (; linewidth=4, linestyle=:dash, color=(:blue, 0.8), label="My ipred").
sim::Union{Bool, NamedTuple} = true (only for plotting results of simobs) toggle whether to plot the simobs result. NamedTuples will be passed as attributes to Makie.lines! and Makie.scatter! - see those documentations for full options. Common options are (; linewidth=4, linestyle=:dash, color=(:blue, 0.8), label="Simulated"). Turn off scatter with markersize=0 and/or turn of the connecting lines with linewidth=0.
title::Union{Function, String, Nothing} = nothing specify how to generate subplot titles. If you pass a function then it will be called like title(subject, i) whereupon it must return a string. Example title = (subject, i) -> "#$i, ID: $(subject.id)". If you pass a string then that string will be applied to all subplots. nothing sets a default behaviour which plots the first 10 characters of the patient ID unless the axis already has a title defined.
xlabel::String = "Time" specify the x-label.
xtransform::Function = identity a single-argument function to transform the x-data (typically time).
ylabel::String specify the single y-label that's placed to the left of the grid of subplots. Defaults to your output name. If you want a ylabel for each of the subplots then pass axis=(; ylabel="My Label") instead.
ytransform::Function = identity a single-argument transform function to apply to the output before plotting. ytransform = x -> 2*exp(x)
kwargs... Any keyword argument in excess of the above list is passed unmodified to Makies scatter or lines commands (depending on what's being plotted.).

Example usage

Let pop, sim, pred be a Population, a result from simobs or from predict, respectively.

julia

using CairoMakie

plt = plotgrid(pop)
plotgrid!(pred)
plotgrid!(simobs)
save("my_file_name.png", plt)

plotgrid(
  pred; 
  pred=(; label = "MyPredLabel", color=:blue, linewidth = 10),
  ipred=false,
  data=(; markersize=10, color=:red),
)
plotgrid!(sim, color=(:green, 0.3)) 
plotgrid!(sim, color=(:green, 0.3), add_legend=false) 

plotgrid(
  pred[1:12], 
  figure=(; size=(1000,400)),
  linewidth = 10, # applies to both `pred` and `ipred`
  layout=(6,2),
)

fig = Figure()
ax1 = Axis(fig[1,1])
scatter!(ax1, rand(10), rand(10))
ax_group = GridLayout(fig[1,2])
axes = [Axis(ax_group[i,j]) for i in 1:2, j in 1:2]
plotgrid!(axes, pred[1:4])

DeepPumas.preprocess Method

julia

preprocess(fpm; ηs, covs)
preprocess(model, pop, param, approx; ηs, covs)

Preprocess covariates and posterior η approximations into a target x→y mapping for fitting.

x is a standardized matrix of the covariate values for each Subject. y is derived from empirical bayes estimates (EBEs) that have been standardized and then rescaled such that their standard deviations approximate the sensitivity of the loglikelihood with respect to the given EBE. This informs the trade-off that a fit or a hyperopt needs to make when it is impossible to fit everything at once. Some random effects impact patient outcomes more than others - better to emphasize getting them right.

The resulting FitTarget contains these computed inputs (x) and outputs (y) for the target mapping as well as the transformations (pre and post) that facilitates going back and forth between this transformed mapping and the Subject→ηs mapping.

Arguments

Either

fpm::FittedPumasModel - a fitted pumas model. The embedded data will be used to generate the input data and some version of the posterior η distribution will be used to generate the target output.

or

model::AbstractPumasModel - a Pumas model
pop::Population - a Population of training data.
param::NamedTuple - fitted parameters.
approx::AbstractLikelihoodApproximation - a likelihood approximation, e.g., Pumas.FOCE().

Keyword arguments

ηs - specify the posterior ηs to be used as targets. This should be an iterator of Symbols. For example ηs = (:η_CL, :η_Ka). (η is typed by \eta<tab>)
covs - specify which covariates of fpm.data.covariates() to use when generating input data. This should be an iterator of Symbols. For example covs = (:AGE, :SEX).

Returns

FitTarget for use with fit or hyperopt.

Example

Let model be a PumasModel, pop be a Population.

julia

fpm = fit(model, pop, sample_params(model), MAP(FOCE()))
target = preprocess(fpm)

DeepPumas.preprocess Method

julia

preprocess(X, Y; standardize = false)

process the X and Y matrices into a FitTarget for use with fit or hyperopt.

The number of columns in your matrices reflect the number of datapoints while the number of rows give the dimensionality of your input (X) and output (Y).

standardize toggles a z-score standardization of X. This transformation is stored and can be applied to new input data fed to a model fitted towards this generated FitTarget.

DeepPumas.set_mlp_backend Function

julia

set_mlp_backend(backend=:flux)

Set which package should serve as the backend for the MLPDomain constructor. Available options are (:flux, :simplechains, :staticflux).

DeepPumas.sparse_fit Method

julia

sparse_fit(
    augmentedmodel,
    basis,
    pop,
    optimizer,
    priors;
    initial_sparse_parameters,
    kwargs...
)

Tries to fit an augmented pumasmodel with a DataDrivenDomain in such a way that it sparsifies the result as much as possible over the different prior(s). Returns the sorted solutions to the model fit, sorted by either aicc, bic, aic or loglikelihood.

DeepPumas.sparse_fit Method

julia

sparse_fit(
    pumasmodel,
    pop,
    optimizer,
    priors;
    init,
    sort_by,
    kwargs...
)

Tries to fit a PumasModel which contains a DataDrivenDomain in such a way that it sparsifies the result as much as possible over the different prior(s). Returns the sorted solutions to the model fit, sorted by either aicc, bic, aic or loglikelihood.

DeepPumas.synthetic_data Method

julia

synthetic_data(model[, dose][, param]; kwargs...)

Generate synthetic data from model.

Arguments

model::AbstractPumasModel - the model from which to generate data.
dose::Union{Vector{DosageRegimen}, DosageRegimen}=DosageRegimen(1) - A DosageRegimen or Vector thereof. If a vector is supplied then one Subject will be generated for each DosageRegimen. If not, every subject will have the same DosageRegimen and the size of the population can be adjusted with the keyword argument nsubj.
param::NamedTuple = init_params(model) - The parameter values to use for the data generation.

Keyword arguments

covariates::NamedTuple = (;) - Specify the distributions from which to generate synthetic covariates. The effect of the covariates should be specified inside the model but here you can tune their values. The keys of the NamedTuple should match the covariates specified in the model and the values should either be a Distribution or a Vector/Tuple thereof. Scalars will be returned for univariate distributions and vectors will be returned for multivariate distributions or from vectors/tuples of univariate distributions. Example distributions = (; AGE = Normal(55,20), cov_vector = fill(Normal(), 3)). Any model covariate not specified here will default to a Normal(0., 1) distribution.
obstimes = 0:1.0:10 - an iterable (Range, Vector, Tuple, etc.) to specify the timepoints at which the synthetic data is sampled.
nsubj = 100 - the number of Subjects to generate. Not applicable if dose is a Vector.
rng::AbstractRNG = Random.default_rng() - a random number generator. Typically used to ensure reproducibility with rng = StableRNG(123).

Pumas.init Method

julia

init(d)

Return an initialized object from d, where d can be a domain, Distribution, ParamSet or Formula.

Examples:

julia

using Pumas
using DeepPumas

init(RealDomain(;))  
0.0

init(ParamSet((;a=RealDomain(;), b=RealDomain(;))
(a=0.0, 
b=0.0,)

NN = init(MLPDomain(1, 2, 1))
DeepPumas.FluxModelParam{Flux.Chain{Tuple{Flux.Dense{typeof(tanh), Matrix{Float64}, Vector{Float64}}, Flux.Dense{typeof(tanh), Matrix{Float64}, 
Vector{Float64}}}}, Vector{Float64}}(Chain(Dense(1 => 2, tanh), Dense(2 => 1, tanh)), [-0.6472278596494233, -1.052911126562834, 0.0, 0.0, 
-0.0563053509375365, 0.5064983064035643, 0.0])

typeof(NN(1.0))
Vector{Float64}

Pumas.init_params Method

julia

init_params(ml::Union{FluxDomain, FluxModel})

Get a Vector of initial parameters for the machine learning model ml.

Uses a Glorot Normal initialization of the weights and sets biases to 0. A seeded random number generator is used for reproducibility.

Pumas.sample_params Method

julia

sample_params(ml::Union{FluxDomain, FluxModel}; rng)

Get a vector of random initial parameters for the machine learning model ml.

Uses a Glorot Normal initialization of the weights and sets biases to 0. The optional rng keyword argument lets you pass a random number generator to be used.

See also init_params.

StatsAPI.fit Method

julia

fit(model, pop, alg; kwargs...)

Tries to fit an augmented PumasModel with a provided AbstractDataDrivenAlgorithm. See the DataDrivenDiffEq documentation for available packages, algorithms and options.

StatsAPI.fit Function

julia

fit(ml, target[, init]; kwargs...)

Fit a machine learning model, ml, to a target input-output mapping.

Arguments

ml::Union{FluxDomain, Chain, HoldoutValidationResult} - A machine learning model, typically coming out of a call to MLPDomain or a fit thereof.
target::FitTarget - a target input-output mapping, typically generated by preprocess.
init=sample_params(nn) - the initial parameters to start from. Defaults to a random sampling of a Glorot Normal distribution. see also init_params which also samples from a Glorot distribution but with a seed for reproducibility.

Keyword arguments

training_fraction = 1.0 - The fraction of data that should be used for training. The rest is withheld for validation.
shuffle = true - Should the data be shuffled before splitting into a training and a validation set?
optim_alg = nothing - Specify the optimization algorithm. Only Adam is supported for all neural networks backends. Flux-based models can use optimization algorithms from Optimisers.jl. If nothing is passed then a fresh fit will use Adam while a continued fit (where ml is the result of a previous fit) will re-use the previous algorithm.
optim_options - A NamedTuple of options to be passed on to the fitting. Valid options:
- rng - a random number generator.
- loss - the loss function, available options are l1 (default) and l2.
- lambda - the regularization constant. Defaults to 0 or to any regularization or prior specified in ml.
- alpha - the regularization mode. 0 for L2 regularization and 1 for L1. Defaults to 0 or to any Prior or regularization specified in ml.
- verbosity - tune the amount of printouts during hyperoptimization. Defaults to 1. Set to 0 to avoid printouts.
- batch_size - Specify the batch sizes for the optimization algorithm (typically Adam). Batching determines how many data points to use each time the gradient is calculated and a step in parameter space is taken. Batching introduces an often beneficial stochasticity in the optimization. This is good for avoiding local optima but is sometimes computationally costly. Defaults to including all data in one batch.
- epochs - the number of iterations to optimize for. Defaults to 10_000.

StatsAPI.predict Method

julia

predict(fit_result, data)

Arguments

fit_result::Union{HyperoptResult, HoldoutValidationResult} - A fitted machine learning model.
data::Union{Subject, Population} - a Pumas subject or population to predict the random effects of.