|
| 1 | +# [Solving the chemical master equation using FiniteStateProjection.jl](@id finite-state_projection) |
| 2 | +```@raw html |
| 3 | +<details><summary><strong>Environment setup and package installation</strong></summary> |
| 4 | +``` |
| 5 | +The following code sets up an environment for running the code on this page. |
| 6 | +```julia |
| 7 | +using Pkg |
| 8 | +Pkg.activate(; temp = true) # Creates a temporary environment, which is deleted when the Julia session ends. |
| 9 | +Pkg.add("Catalyst") |
| 10 | +Pkg.add("FiniteStateProjection") |
| 11 | +Pkg.add("JumpProcesses") |
| 12 | +Pkg.add("OrdinaryDiffEqDefault") |
| 13 | +Pkg.add("OrdinaryDiffEqRosenbrock") |
| 14 | +Pkg.add("Plots") |
| 15 | +Pkg.add("SteadyStateDiffEq") |
| 16 | +``` |
| 17 | +```@raw html |
| 18 | +</details> |
| 19 | +``` |
| 20 | +```@raw html |
| 21 | +<details><summary><strong>Quick-start example</strong></summary> |
| 22 | +``` |
| 23 | +The following code provides a brief example of how to simulate the chemical master equation using the [FiniteStateProjection.jl](https://github.com/SciML/FiniteStateProjection.jl) package. |
| 24 | +```julia |
| 25 | +# Create reaction network model (a bistable switch). |
| 26 | +using Catalyst |
| 27 | +rs_bistable = @reaction_network begin |
| 28 | + hillr(Y, v, K, n), ∅ --> X |
| 29 | + hillr(X, v, K, n), ∅ --> Y |
| 30 | + d, (X,Y) --> 0 |
| 31 | +end |
| 32 | + |
| 33 | +# Create a FSPSystem. The second argument denotes species order in u0 and sol. |
| 34 | +using FiniteStateProjection |
| 35 | +fsp_sys = FSPSystem(rs_bistable, [:X, :Y]) |
| 36 | + |
| 37 | +# Create ODEProblem. Initial condition is the system's initial distribution. |
| 38 | +# Initial condition also designates projection space (outside of which solution should be very close to 0). |
| 39 | +u0 = zeros(20,20) |
| 40 | +u0[6,2] = 1.0 |
| 41 | +tspan = (0.0, 50.0) |
| 42 | +ps = [:v => 1.0, :K => 3.0, :n => 3, :d => 0.2] |
| 43 | +oprob = ODEProblem(fsp_sys, u0, tspan, ps) |
| 44 | + |
| 45 | +# Simulate ODE (it can be quite large, so consider performance options). |
| 46 | +# Plot solution as a heatmap at a specific time point. |
| 47 | +using OrdinaryDiffEqRosenbrock, Plots |
| 48 | +osol = solve(oprob, Rodas5P()) |
| 49 | +heatmap(0:19, 0:19, osol(50.0); xguide = "Y", yguide = "X") |
| 50 | +``` |
| 51 | +```@raw html |
| 52 | +</details> |
| 53 | +``` |
| 54 | + \ |
| 55 | + |
| 56 | +As previously discussed, [*stochastic chemical kinetics*](@ref math_models_in_catalyst_sck_jumps) models are mathematically given by jump processes that capture the exact times at which individual reactions occur, and the exact (integer) amounts of each chemical species at a given time. They represent a more microscopic model than [chemical Langevin equation SDE](@ref math_models_in_catalyst_cle_sdes) and [reaction rate equation ODE](@ref math_models_in_catalyst_rre_odes) models, which can be interpreted as approximations to stochastic chemical kinetics models in the large population limit. |
| 57 | + |
| 58 | +One can study the dynamics of stochastic chemical kinetics models by simulating the stochastic processes using Monte Carlo methods. For example, they can be [exactly sampled](@ref simulation_intro_jumps) using [Stochastic Simulation Algorithms](https://en.wikipedia.org/wiki/Gillespie_algorithm) (SSAs), which are also often referred to as Gillespie's method. To gain a good understanding of a system's dynamics, one typically has to carry out a large number of jump process simulations to minimize sampling error. To avoid such sampling error, an alternative approach is to solve ODEs for the *full probability distribution* that these processes have a given value at each time. Knowing this distribution, one can then calculate any statistic of interest that can be sampled via running many SSA simulations. |
| 59 | + |
| 60 | +[*The chemical master equation*](https://en.wikipedia.org/wiki/Master_equation) (CME) describes the time development of this probability distribution[^1], and is given by a (possibly infinite) coupled system of ODEs (with one ODE for each possible chemical state, i.e. number configuration, of the system). For a simple [birth-death model](@ref basic_CRN_library_bd) ($\varnothing \xrightleftharpoons[d]{p} X$) the CME looks like |
| 61 | +```math |
| 62 | +\begin{aligned} |
| 63 | +\frac{dP(X(t)=0)}{dt} &= d \cdot P(X(t)=1) - p \cdot P(X(t)=0) \\ |
| 64 | +\frac{dP(X(t)=1)}{dt} &= p \cdot P(X(t)=0) + d \cdot 2P(X(t)=2) - (p + d) P(X(t)=1) \\ |
| 65 | + &\vdots\\ |
| 66 | +\frac{dP(X(t)=i)}{dt} &= p \cdot P(X(t)=i-1) + d \cdot (i + 1)P(X(t)=i+1) - (p + D \cdot i) P(X(t)=i) \\ |
| 67 | + &\vdots\\ |
| 68 | +\end{aligned} |
| 69 | +``` |
| 70 | +A general form of the CME is provided [here](@ref math_models_in_catalyst_sck_jumps). For chemical reaction networks in which the total population is bounded, the CME corresponds to a finite set of ODEs. In contrast, for networks in which the system can (in theory) become unbounded, such as networks that include zero order reactions like $\varnothing \to X$, the CME will correspond to an infinite set of ODEs. Even in the finite case, the number of ODEs corresponds to the number of possible state vectors (i.e. vectors with components representing the integer populations of each species in the network), and can become exceptionally large. Therefore, for even simple reaction networks there can be many more ODEs than can be represented in typical levels of computer memory, and it becomes infeasible to numerically solve the full system of ODEs that correspond to the CME. However, in many cases the probability of the system attaining species values outside some small range can become negligibly small. Here, a truncated, approximating, version of the CME can be solved practically. An approach for this is the *finite state projection method*[^2]. Below we describe how to use the [FiniteStateProjection.jl](https://github.com/SciML/FiniteStateProjection.jl) package to solve the truncated CME (with the package's [documentation](https://docs.sciml.ai/FiniteStateProjection/dev/) providing a more extensive description). While the CME approach can be very powerful, we note that even for systems with a few species, the truncated CME typically has too many states for it to be feasible to solve the full set of ODEs. |
| 71 | + |
| 72 | +## [Finite state projection simulation of single-species model](@id state_projection_one_species) |
| 73 | +For this example, we will use a simple [birth-death model](@ref basic_CRN_library_bd), where a single species ($X$) is created and degraded at constant rates ($p$ and $d$, respectively). |
| 74 | +```@example state_projection_one_species |
| 75 | +using Catalyst |
| 76 | +rs = @reaction_network begin |
| 77 | + (p,d), 0 <--> X |
| 78 | +end |
| 79 | +``` |
| 80 | +Next, we perform jump simulations (using the [ensemble simulation interface](@ref ensemble_simulations_monte_carlo)) of the model. Here, we can see how it develops from an initial condition and reaches a steady state distribution. |
| 81 | +```@example state_projection_one_species |
| 82 | +using JumpProcesses, Plots |
| 83 | +u0 = [:X => 5] |
| 84 | +tspan = (0.0, 10.0) |
| 85 | +ps = [:p => 20.0, :d => 0.5] |
| 86 | +jprob = JumpProblem(JumpInputs(rs, u0, tspan, ps)) |
| 87 | +eprob = EnsembleProblem(jprob) |
| 88 | +esol = solve(eprob, SSAStepper(); trajectories = 10) |
| 89 | +plot(esol; ylimit = (0.0, Inf)) |
| 90 | +Catalyst.PNG(plot(esol; ylimit = (0.0, Inf), fmt = :png, dpi = 200)) # hide |
| 91 | +``` |
| 92 | +Using chemical master equation simulations, we want to simulate how the *full probability distribution* of these jump simulations develops across the simulation time frame. |
| 93 | + |
| 94 | +As a first step, we import the FiniteStateProjection package. Next, we convert our [`ReactionSystem`](@ref) to a `FSPSystem` (from which we later will generate the ODEs that correspond to the truncated CME). |
| 95 | +```@example state_projection_one_species |
| 96 | +using FiniteStateProjection |
| 97 | +fsp_sys = FSPSystem(rs) |
| 98 | +nothing # hide |
| 99 | +``` |
| 100 | +Next, we set our initial condition. For normal simulations, $X$'s initial condition would be a single value. Here, however, we will simulate $X$'s probability distribution. Hence, its initial condition will also be a probability distribution. In FiniteStateProjection's interface, the initial condition is an array, where the $i$'th index is the probability that $X$ have an initial value of $i-1$. The total sum of all probabilities across the vector should be normalised to $1.0$. Here we assume that $X$'s initial conditions is known to be $5$ (hence the corresponding probability is $1.0$, and the remaining ones are $0.0$): |
| 101 | +```@example state_projection_one_species |
| 102 | +u0 = zeros(75) |
| 103 | +u0[6] = 1.0 |
| 104 | +bar(u0, label = "t = 0.0") |
| 105 | +``` |
| 106 | +We also plot the full distribution using the `bar` function. Finally, the initial condition vector defines the finite space onto which we project the CME. I.e. we will assume that, throughout the entire simulation, the probability of $X$ reaching values outside this initial vector is negligible. |
| 107 | + |
| 108 | +!!! warning |
| 109 | + This last bit is important. Even if the probability seems to be very small on the boundary provided by the initial condition, there is still a risk that probability will "leak". Here, it can be good to make simulations using different projections, ensuring that the results are consistent (especially for longer simulations). It is also possible to (at any time point) sum up the total probability density to gain a measure of how much has "leaked" (ideally, this sum should be as close to 1 as possible). While solving the CME over a very large space will ensure correctness, a too large a space comes with an unnecessary performance penalty. |
| 110 | + |
| 111 | +Now, we can finally create an `ODEProblem` using our `FSPSystem`, initial conditions, and the parameters declared previously. We can simulate this `ODEProblem` like any other ODE. |
| 112 | +```@example state_projection_one_species |
| 113 | +using OrdinaryDiffEqDefault |
| 114 | +oprob = ODEProblem(fsp_sys, u0, tspan, ps) |
| 115 | +osol = solve(oprob) |
| 116 | +nothing # hide |
| 117 | +``` |
| 118 | +Finally, we can plot $X$'s probability distribution at various simulation time points. Again, we will use the `bar` function to plot the distribution, and the interface described [here](@ref simulation_structure_interfacing_solutions) to access the simulation at specified time points. |
| 119 | +```@example state_projection_one_species |
| 120 | +bar(0:74, osol(1.0); bar_width = 1.0, linewidth = 0, alpha = 0.7, label = "t = 1.0") |
| 121 | +bar!(0:74, osol(2.0); bar_width = 1.0, linewidth = 0, alpha = 0.7, label = "t = 2.0") |
| 122 | +bar!(0:74, osol(5.0); bar_width = 1.0, linewidth = 0, alpha = 0.7, label = "t = 5.0") |
| 123 | +bar!(0:74, osol(10.0); bar_width = 1.0, linewidth = 0, alpha = 0.7, label = "t = 10.0", |
| 124 | + xguide = "X (copy numbers)", yguide = "Probability density") |
| 125 | +``` |
| 126 | + |
| 127 | +## [Finite state projection simulation of multi-species model](@id state_projection_multi_species) |
| 128 | +Next, we will consider a system with more than one species. The workflow will be identical, however, we will have to make an additional consideration regarding the initial condition, simulation performance, and plotting approach. |
| 129 | + |
| 130 | +For this example, we will consider a simple dimerisation model. In it, $X$ gets produced and degraded at constant rates, and can also dimerise to form $X₂$. |
| 131 | +```@example state_projection_multi_species |
| 132 | +using Catalyst # hide |
| 133 | +rs = @reaction_network begin |
| 134 | + (p,d), 0 <--> X |
| 135 | + (kB,kD), 2X <--> X₂ |
| 136 | +end |
| 137 | +``` |
| 138 | +Next, we will declare our parameter values and initial condition. In this case, the initial condition is a matrix where element $(i,j)$ denotes the initial probability that $(X(0),X₂(0)) = (i-1,j-1)$. In this case, we will use an initial condition where we know that $(X(0),X₂(0)) = (5,0)$. |
| 139 | +```@example state_projection_multi_species |
| 140 | +ps = [:p => 1.0, :d => 0.2, :kB => 2.0, :kD => 5.0] |
| 141 | +u0 = zeros(25,25) |
| 142 | +u0[6,1] = 1.0 |
| 143 | +nothing # hide |
| 144 | +``` |
| 145 | +In the next step, however, we have to make an additional consideration. Since we have more than one species, we have to define which dimension of the initial condition (and hence also the output solution) corresponds to which species. Here we provide a second argument to `FSPSystem`, which is a vector listing all species in the order they occur in the `u0` array. |
| 146 | +```@example state_projection_multi_species |
| 147 | +using FiniteStateProjection # hide |
| 148 | +fsp_sys = FSPSystem(rs, [:X, :X₂]) |
| 149 | +nothing # hide |
| 150 | +``` |
| 151 | +Finally, we can simulate the model just like in the 1-dimensional case. As we are simulating an ODE with $25⋅25 = 625$ states, we need to make some considerations regarding performance. In this case, we will simply specify the `Rodas5P()` ODE solver (more extensive advice on performance can be found [here](@ref ode_simulation_performance)). Here, we perform a simulation with a long time span ($t = 100.0$), aiming to find the system's steady state distribution. Next, we plot it using the `heatmap` function. |
| 152 | +```@example state_projection_multi_species |
| 153 | +using Plots # hide |
| 154 | +using OrdinaryDiffEqRosenbrock |
| 155 | +oprob = ODEProblem(fsp_sys, u0, 100.0, ps) |
| 156 | +osol = solve(oprob, Rodas5P()) |
| 157 | +heatmap(0:24, 0:24, osol[end]; xguide = "X₂", yguide = "X") |
| 158 | +``` |
| 159 | + |
| 160 | +!!! warning |
| 161 | + The `heatmap` function "flips" the plot contrary to what many would consider intuitive. I.e. here the x-axis corresponds to the second species ($X₂$) and the y-axis to the first species ($X$). |
| 162 | + |
| 163 | +## [Finite state projection steady state simulations](@id state_projection_steady_state_sim) |
| 164 | +Previously, we have shown how the [SteadyStateDiffEq.jl](https://github.com/SciML/SteadyStateDiffEq.jl) package can be used to [find an ODE's steady state through forward simulation](@ref steady_state_stability). The same interface can be used for ODEs generated through FiniteStateProjection. Below, we use this to find the steady state of the dimerisation example studied in the last example. |
| 165 | +```@example state_projection_multi_species |
| 166 | +using SteadyStateDiffEq, OrdinaryDiffEqRosenbrock |
| 167 | +ssprob = SteadyStateProblem(fsp_sys, u0, ps) |
| 168 | +sssol = solve(ssprob, DynamicSS(Rodas5P())) |
| 169 | +heatmap(0:24, 0:24, sssol; xguide = "X₂", yguide = "X") |
| 170 | +``` |
| 171 | +Finally, we can also approximate this steady state through Monte Carlo jump simulations. |
| 172 | +```@example state_projection_multi_species |
| 173 | +using JumpProcesses # hide |
| 174 | +jprob = JumpProblem(JumpInputs(rs, [:X => 0, :X₂ => 0], (0.0, 100.0), ps)) |
| 175 | +output_func(sol, i) = (sol[end], false) |
| 176 | +eprob = EnsembleProblem(jprob; output_func) |
| 177 | +esol = solve(eprob, SSAStepper(); trajectories = 10000) |
| 178 | +ss_jump = zeros(25,25) |
| 179 | +for endpoint in esol |
| 180 | + ss_jump[endpoint[1] + 1, endpoint[2] + 1] += 1 |
| 181 | +end |
| 182 | +heatmap(0:24, 0:24, ss_jump ./length(esol); xguide = "X₂", yguide = "X") |
| 183 | +``` |
| 184 | +Here we used an ensemble [output function](@ref activation_time_distribution_measurement) to only save each simulation's final state (and plot these using `heatmap`). |
| 185 | + |
| 186 | + |
| 187 | +--- |
| 188 | +## References |
| 189 | +[^1]: [Daniel T. Gillespie, *A rigorous derivation of the chemical master equation*, Physica A: Statistical Mechanics and its Applications (1992).](https://www.sciencedirect.com/science/article/abs/pii/037843719290283V) |
| 190 | +[^2]: [Brian Munsky, Mustafa Khammash, *The finite state projection algorithm for the solution of the chemical master equation*, Journal of Chemical Physics (2006).](https://pubs.aip.org/aip/jcp/article-abstract/124/4/044104/561868/The-finite-state-projection-algorithm-for-the?redirectedFrom=fulltext) |
0 commit comments