Results


Scoring and saved simulation output

Every time a simulation is run the simulated results are compared to the results provided and a score is calculated. Additionally the end point of the experimental output of all simulations is also stored. When performing the diagnostics function an MATLAB® representation of the entire run is also saved.

  • simd - Simulation results (MATLAB® representation)

  • st - Total score

To simplify representations the following correspondence has been used

\(score_{i,j,k} = \frac{1}{n} \sum_{i=1}^n \left(\frac{Y_{i,j,k}-y_{i,j,k}}{τ_{i,j,k}}\right)^2\)

if stg.useLog = 0

\(st(θ;Y,τ) = \sum_{k=1}^l \sum_{j=1}^m score_{i,j,k}\)

if stg.useLog = 1

\(st(θ;Y,τ) = \sum_{k=1}^l \sum_{j=1}^m log_{10}(score_{i,j,k})\)

if stg.useLog = 2

\(st(θ;Y,τ) = \sum_{k=1}^l log_{10}(\sum_{j=1}^m score_{i,j,k})\)

if stg.useLog = 3

\(st(θ;Y,τ) = log_{10}(\sum_{k=1}^l \sum_{j=1}^m score_{i,j,k})\)

  • se - Score of each experiment

    if stg.useLog = 0 or 3

    \[\begin{split}se(θ;Y,τ) = \begin{bmatrix} \sum_{j=1}^m score_{i,j,1} \\ \sum_{j=1}^m score_{i,j,2} \\ ... \\ \sum_{j=1}^m score_{i,j,k} \end{bmatrix}\end{split}\]

    if stg.useLog = 1

    \[\begin{split}se(θ;Y,τ) = \begin{bmatrix} \sum_{j=1}^m log_{10}(score_{i,j,1}) \\ \sum_{j=1}^m log_{10}(score_{i,j,2}) \\ ... \\ \sum_{j=1}^m log_{10}(score_{i,j,k}) \end{bmatrix}\end{split}\]

    if stg.useLog = 2

    \[\begin{split}se(θ;Y,τ) = \begin{bmatrix} log_{10}(\sum_{j=1}^m score_{i,j,1}) \\ log_{10}(\sum_{j=1}^m score_{i,j,2})\\ ... \\ log_{10}(\sum_{j=1}^m score_{i,j,k}) \end{bmatrix}\end{split}\]

  • sd - Score of each experimental outputs in all experiments

    if stg.useLog = 0,2, or 3

    \[\begin{split}sd(θ;Y,τ) = \begin{bmatrix} score_{i,1,1} & score_{i,2,1} & ... & score_{i,j,1}\\ score_{i,1,2} & score_{i,2,2} & ... & score_{i,j,2}\\ ... & ... & ... & ... \\ score_{i,1,k} & score_{i,2,k} & ... & score_{i,j,k} \end{bmatrix}\end{split}\]

    if stg.useLog = 1

    \[\begin{split}sd(θ;Y,τ) = \begin{bmatrix} log_{10}(score_{i,1,1}) & log_{10}(score_{i,2,1}) & ... & log_{10}(score_{i,j,1})\\ log_{10}(score_{i,1,2}) & log_{10}(score_{i,2,2}) & ... & log_{10}(score_{i,j,2})\\ ... & ... & ... & ... \\ log_{10}(score_{i,1,k}) & log_{10}(score_{i,2,k}) & ... & log_{10}(score_{i,j,k}) \end{bmatrix}\end{split}\]

  • xfinal - Value of each experimental outputs at the end of the simulation

    \[\begin{split}xfinal(θ;Y,τ) = \begin{bmatrix} y_{n,1,1} & y_{n,2,1} & ... & y_{n,j,1} \\ y_{n,1,2} & y_{n,2,2} & ... & y_{n,j,2} \\ ... & ... & ... & ... \\ y_{n,1,k} & y_{n,2,k} & ... & y_{n,j,k} \end{bmatrix}\end{split}\]

    • \(F =\) Objective function for Particle Swarm optimization

    • \(Y =\) Data provided for fitting

    • \(y =\) Simulation results of the updated model under parameterization \(θ\)

    • \(θ =\) New parameterization for \(y\)

    • \(τ =\) Allowed mismatch between the two simulation results, analogous to the standard deviation of a Gaussian noise model in data fitting

    • \(n/i =\) Number/index of points in a given experimental output

    • \(m/j =\) Number/index of experimental outputs

    • \(l/k =\) Number/index of experiments


Diagnostics

When running the diagnostics a struct gets created that stores all the oputputs of the f_sim_score function.

  • rst.diag.simd - Simulation results (MATLAB® representation)

  • rst.diag.st - Total score

  • rst.diag.se - Score per experiment

  • rst.diag.sd - Score per experimental outputs in all experiments

  • rst.diag.xfinal - x value of all the species being tested at the end of the simulation


Optimization

  • rst.opt.name - Name of optimizer that was used

  • rst.opt.x - Best parameter set found by the optimization

  • rst.opt.fval - Score for that best parameter set

  • rst.opt.exitflag - Diagnostics to see how the optimization went

  • rst.opt.output - Diagnostics to see how the optimization went


Sensitivity Analysis

The calculations performed to obtain these sensitivities where performed according to the equations described in Halnes et al 2009.

  • rst.SA.M1 - Matrix with (\(r*k\)) random numbers within the lower and upper bound ranges set for each parameter

    \[\begin{split}M_1 = \begin{bmatrix} x_{1}^{(1)} & x_{2}^{(1)} & ... & x_{k}^{(1)} \\ x_{1}^{(2)} & x_{2}^{(2)} & ... & x_{k}^{(2)} \\ ... & ... & ... & ... \\ x_{1}^{(r)} & x_{2}^{(r)} & ... & x_{k}^{(r)} \end{bmatrix}\end{split}\]

  • rst.SA.M2 - Same as rst.SA.M1 but different random initialization

    \[\begin{split}M_2 = \begin{bmatrix} x_{1}^{(1')} & x_{2}^{(1')} & ... & x_{k}^{(1')} \\ x_{1}^{(2')} & x_{2}^{(2')} & ... & x_{k}^{(2')} \\ ... & ... & ... & ... \\ x_{1}^{(r')} & x_{2}^{(r')} & ... & x_{k}^{(r')} \end{bmatrix}\end{split}\]

  • rst.SA.N - Matrix of size (\(r*k*k\)) with columns exchanged between M1 and M2 as follows:

    \[\begin{split}N_i = \begin{bmatrix} x_{1}^{(1')} & x_{2}^{(1')} & ... & x_{i}^{(1)} & ... & x_{k}^{(1')} \\ x_{1}^{(2')} & x_{2}^{(2')} & ... & x_{i}^{(2)} & ... & x_{k}^{(2')} \\ ... & ... & ... & ... & ... & ... \\ x_{1}^{(r')} & x_{2}^{(r')} & ... & x_{i}^{(r)} & ... & x_{k}^{(r')} \end{bmatrix}\end{split}\]

    • \(x =\) Parameters

    • \(k =\) Total number of parameters (stg.parnum)

    • \(r =\) Total number of Samples (stg.sansamples)

    • \(i =\) Index of each parameter

  • rst.SA.fM1 -

    \[\begin{split}fM_1 = \begin{bmatrix} f(M_1^{(1)}) \\ f(M_1^{(2)}) \\ ... \\ f(M_1^{(r)}) \end{bmatrix} = \begin{bmatrix} f(x_{1}^{(1)} & x_{2}^{(1)} & ... & x_{k}^{(1)}) \\ f(x_{1}^{(2)} & x_{2}^{(2)} & ... & x_{k}^{(2)}) \\ ... & ... & ... & ... \\ f(x_{1}^{(r)} & x_{2}^{(r)} & ... & x_{k}^{(r)}) \end{bmatrix}\end{split}\]

  • rst.SA.fM2 -

    \[\begin{split}fM_2 = \begin{bmatrix} f(M_2^{(1')}) \\ f(M_2^{(2')}) \\ ... \\ f(M_2^{(r')}) \end{bmatrix} = \begin{bmatrix} f(x_{1}^{(1')} & x_{2}^{(1')} & ... & x_{k}^{(1')}) \\ f(x_{1}^{(2')} & x_{2}^{(2')} & ... & x_{k}^{(2')}) \\ ... & ... & ... & ... \\ f(x_{1}^{(r')} & x_{2}^{(r')} & ... & x_{k}^{(r')}) \end{bmatrix}\end{split}\]

  • rst.SA.fN -

    \[\begin{split}fN_i = \begin{bmatrix} f(N_i^{(1)}) \\ f(N_i^{(2)}) \\ ... \\ f(N_i^{(r)}) \end{bmatrix} = \begin{bmatrix} f(x_{1}^{(1')} & x_{2}^{(1')} & ... & x_{i}^{(1)} & ... & x_{k}^{(1')}) \\ f(x_{1}^{(2')} & x_{2}^{(2')} & ... & x_{i}^{(2)} & ... & x_{k}^{(2')}) \\ ... & ... & ... & ... & ... & ... \\ f(x_{1}^{(r')} & x_{2}^{(r')} & ... & x_{i}^{(r)} & ... & x_{k}^{(r')}) \end{bmatrix}\end{split}\]

    • \(k =\) Total number of parameters (stg.parnum)

    • \(r =\) Total number of Samples (stg.sansamples)

    • \(i =\) Index of each parameter

  • rst.SA.SI - First order effects

    \(S_{i}=\frac{V_{Θ_{i}}(E_{Θ_{-i}}(Y|Θ_{i}))}{V(Y)}=\frac{U_{i}-E^2(Y)}{V(Y)}\)

    \(U_{i}=\frac{1}{n-1}\sum_{r=1}^nf(M_1^r)f(N_i^r)\)

    \(E^2(Y)=\frac{1}{n}\sum_{r=1}^nf(M_1^r)f(M_2^r)\)

    \(V(Y) = \frac{1}{n-1}f^2(M_1^r)-E^2(Y)\)

    • \(V =\) Variance

    • \(E(... |...) =\) Conditional expected value

    • \(Θ =\) Parameters of the model

    • \(Y =\) Scalar output from the model

    • \(n =\) Total number of Samples (stg.sansamples)

    • \(r =\) Index of the Samples

    • \(i =\) Index of each parameter

  • rst.SA.STI - Total order effects

    \(S_{Ti}=\frac{V(Y)-V_{Θ_{i}}(E_{Θ_{i}}(Y|Θ_{i}))}{V(Y)}=1-\frac{U_{-i}-E^2(Y)}{V_T(Y)}\)

    \(U_{-i}=\frac{1}{n-1}\sum_{r=1}^nf(M_2^r)f(N_i^r)\)

    \(E^2(Y)=\frac{1}{n}\sum_{r=1}^nf(M_1^r)f(M_2^r)\)

    \(V_T(Y) = \frac{1}{n-1}f^2(M_2^r)-E^2(Y)\)

    • \(V =\) Variance

    • \(E(... |...) =\) Conditional expected value

    • \(Θ =\) Parameters of the model

    • \(Y =\) Scalar output from the model

    • \(n =\) Total number of Samples (stg.sansamples)

    • \(r =\) Index of the Samples

    • \(i =\) Index of each parameter