MIMIC-II IAC Patient Trajectory and Fate#
In the previous introduction tutorial, we explored the MIMIC-II IAC dataset, comprising electronic health records (EHR) of 1776 patients in 46 features, and identified patient group-specific clusters using ehrapy. Please go through the MIMIC-II IAC introduction before performing this tutorial to get familiar with the dataset.
As a next step, we want to determine patient trajectories and patient fate. The goal is to detect terminal states and the corresponding origins based on pseudotime. Real time very rarely reflects the actual progression of a disease. When measurements are done in a single snapshot or cross-sectional setting, some patients will show no sign of disease (e.g. healthy or recovered), some are at the onset of a specific disease and some are in a more severe stage or even at the height. For an appropriate analysis, we are interested in a continuous transition of states, such as from healthy to diseased to death, for which the real time is therefore not available or informative. Identification of transition states can be achieved by identifying source states (e.g. healthy) and then calculating pseudotime from this state. Based on Markov chain modelling, we uncover patient dynamics using CellRank. For more details, please read CellRank paper 1 and CellRank paper 2.
In this tutorial we will be using CellRank to:
Simulate patient trajectories with random walks.
Compute patient macrostates and infer fate probabilities towards predicted terminal states.
Identify potential driver features for each identified trajectory.
Visualize feature trends along specific patient states, while accounting for the continuous nature of fate determination.
Before performing this tutorial, we highly recommend to read the extensive and well written CellRank documentation, especially the general tutorial chapter is useful. If you are not familiar with single-cell data, do not be afraid and replace cells with patients visits and genes with features in your mind.
This tutorial requires cellrank to be installed. As this packages is not a dependency of ehrapy, it must be installed separately.
%env NUMBA_CPU_NAME=generic
Before we start with the patient fate analysis of the MIMIC-II IAC dataset, we set up our environment including the import of packages and preparation of the dataset.
Environment setup#
Ensure that the latest version of ehrapy is installed. A list of all dependency versions can be found at the end of this tutorial.
import anndata as ad
import cellrank as cr
import ehrapy as ep
import ehrdata as ed
import matplotlib.pyplot as plt
import numpy as np
We are ignoring a few warnings for readability reasons.
import warnings
warnings.filterwarnings("ignore")
We set a flag for numba to improve reproducibility across different machines:
Getting and preprocessing the MIMIC-II dataset#
This tutorial is based on the MIMIC-II IAC dataset which was previously introduced in the MIMIC-II IAC introduction tutorial.
edata = ed.dt.mimic_2()
edata
EHRData object with n_obs × n_vars × n_t = 1776 × 46 × 1
shape of .X: (1776, 46)
The MIMIC-II dataset has 1776 patients with 46 features.
Now that we have our EHRData object ready, we need to perform the standard preprocessing steps as performed in the introduction tutorial again before we can use ehrapy and CellRank for patient fate analysis.
ed.infer_feature_types(edata, binary_as="numeric")
! Feature was detected as categorical features stored numerically. Adjust using `ed.replace_feature_types` if needed.
Detected feature types for EHRData object with 1776 obs and 46 vars ╠══ 📅 Date features ╠══ 📐 Numerical features ║ ╠══ abg_count ║ ╠══ afib_flg ║ ╠══ age ║ ╠══ aline_flg ║ ╠══ bmi ║ ╠══ bun_first ║ ╠══ cad_flg ║ ╠══ censor_flg ║ ╠══ chf_flg ║ ╠══ chloride_first ║ ╠══ copd_flg ║ ╠══ creatinine_first ║ ╠══ day_28_flg ║ ╠══ day_icu_intime_num ║ ╠══ gender_num ║ ╠══ hgb_first ║ ╠══ hosp_exp_flg ║ ╠══ hospital_los_day ║ ╠══ hour_icu_intime ║ ╠══ hr_1st ║ ╠══ icu_exp_flg ║ ╠══ icu_los_day ║ ╠══ iv_day_1 ║ ╠══ liver_flg ║ ╠══ mal_flg ║ ╠══ map_1st ║ ╠══ mort_day_censored ║ ╠══ pco2_first ║ ╠══ platelet_first ║ ╠══ po2_first ║ ╠══ potassium_first ║ ╠══ renal_flg ║ ╠══ resp_flg ║ ╠══ sapsi_first ║ ╠══ sepsis_flg ║ ╠══ service_num ║ ╠══ sodium_first ║ ╠══ sofa_first ║ ╠══ spo2_1st ║ ╠══ stroke_flg ║ ╠══ tco2_first ║ ╠══ temp_1st ║ ╠══ wbc_first ║ ╚══ weight_first ╚══ 🗂️ Categorical features ╠══ day_icu_intime (7 categories) ╚══ service_unit (3 categories)
%%capture
edata = ep.pp.encode(edata, autodetect=True)
ep.pp.knn_impute(edata, n_neighbors=5, backend="scikit-learn", var_names = edata.var_names[edata.var["feature_type"] == "numeric"])
ep.pp.log_norm(edata, vars=["iv_day_1", "po2_first"], offset=1)
ep.pp.pca(edata, svd_solver="randomized", random_state=42)
ep.pp.neighbors(edata, transformer="sklearn", n_pcs=10)
ep.tl.umap(edata)
ep.tl.leiden(edata, resolution=0.3, key_added="leiden_0_3")
plt.rcParams["figure.figsize"] = (5, 4)
plt.rcParams["figure.dpi"] = 100
ep.pl.umap(edata, color=["leiden_0_3"], title="Leiden 0.3", size=20)
This UMAP embedding is exactly the same as previously computed in the MIMIC-II IAC introduction tutorial. Now we continue with the patient fate analysis.
Analysis using ehrapy and CellRank#
Depending on the data it may not always be possible to clearly define a cluster or specific patient visits as the origin or terminus of a trajectory. Working with single-cell data simplifies matters since the detection of stem cells generally signifies the start of cell differentiation.
In this tutorial, we will define a patient cluster as the origin (root cluster) and explore possible terminal states.
Pseudotime calculation#
As the root cluster for pseudotime calculation we choose cluster 0 since patients in that cluster do not show very severe comorbidities and features yet. Then we calculate the Diffusion Pseudotime.
edata = ad.AnnData(edata)
edata.uns["iroot"] = np.flatnonzero(edata.obs["leiden_0_3"] == "0")[0]
ep.tl.dpt(edata)
Now we define the kernel, compute the transition matrix and plot a projection onto the UMAP.
Determining patient fate with a PseudotimeKernel#
The PseudotimeKernel computes direct transition probabilities based on a KNN graph and pseudotime.
The KNN graph contains information about the (undirected) conductivities among patients, reflecting their similarity. Pseudotime can be used to either remove edges that point against the direction of increasing pseudotime, or to downweight them.
from cellrank.kernels import PseudotimeKernel
pk = PseudotimeKernel(edata, time_key="dpt_pseudotime")
pk.compute_transition_matrix(threshold_scheme="soft")
INFO Computing transition matrix based on pseudotime
100%|██████████| 1776/1776 [00:00<00:00, 4684.73cell/s]
INFO Finish (0.47s)
PseudotimeKernel[n=1776, dnorm=False, scheme='soft', b=10.0, nu=0.5]
plt.rcParams["figure.figsize"] = (5, 4)
plt.rcParams["figure.dpi"] = 100
pk.plot_projection(basis="umap", color="leiden_0_3")
INFO Projecting transition matrix onto 'umap'
INFO Adding `adata.obsm['T_fwd_umap']` (0.39s)
We observe two main trajectories originating from cluster 0 going to clusters 2, 3, and 5. Let’s check the metedata again.
ep.pl.umap(edata, color="censor_flg", title="Censored or Death (0 = death, 1 = censored)")
ep.pl.umap(
edata,
color="mort_day_censored",
title="Day post ICU admission of censoring or death",
)
Cluster 4 and 1 consist of patients that deceased, while cluster 5 includes patients with a high day post ICU admission.
Simulating transitions with random walks#
Cellrank makes it easy to simulate the behavior of random walks from specific clusters. This allows us to not only visualize where the patients end up, but also roughly how many in which clusters after a defined number of iterations. We can either just start walking…
pk.plot_random_walks(
seed=0,
n_sims=100,
start_ixs={"leiden_0_3": ["0"]},
legend_loc="right",
dpi=100,
show_progress_bar=False,
)
INFO Simulating 100 random walks of maximum length 444
INFO Finish (4.31s)
INFO Plotting random walks
… or set a number of required hits in one or more terminal clusters. Here, we require 50 hits in cluster 2 or 5.
pk.plot_random_walks(
seed=0,
n_sims=100,
start_ixs={"leiden_0_3": ["0"]},
stop_ixs={"leiden_0_3": ["2", "5"]},
successive_hits=50,
legend_loc="right",
dpi=100,
show_progress_bar=False,
)
INFO Simulating 100 random walks of maximum length 444
INFO Finish (3.83s)
INFO Plotting random walks
Black and yellow dots indicate random walk start and terminal patient visits, respectively.
Determining macrostates and terminal states#
To find the terminal states of cluster 0, well will use an estimator to predict the patient fates using the above calculated transition matrix. The main objective is to decompose the patient state space into a set of macrostates, that represent the slow-time scale dynamics of the process and predict terminal states. Here, we will use an Generalized Perron Cluster Cluster Analysis (GPCCA) estimator.
As a first step we try to identify macrostates in the data using the fit() function.
# Check if pseudotime was computed
print("Pseudotime key exists:", "dpt_pseudotime" in edata.obs)
if "dpt_pseudotime" in edata.obs:
print("Pseudotime range:", edata.obs["dpt_pseudotime"].min(), "to", edata.obs["dpt_pseudotime"].max())
print("Any NaN values:", edata.obs["dpt_pseudotime"].isna().sum())
Pseudotime key exists: True
Pseudotime range: 0.0 to 1.0
Any NaN values: 0
g = cr.estimators.GPCCA(pk)
g.fit(cluster_key="leiden_0_3")
g.macrostates_memberships
INFO Computing eigendecomposition of the transition matrix
INFO Adding `adata.uns['eigendecomposition_fwd']`
`.eigendecomposition`
Finish (0.13s)
WARNING Unable to import `petsc4py` or `slepc4py`. Using `method='brandts'`
WARNING For `method='brandts'`, dense matrix is required. Densifying
INFO Computing Schur decomposition
INFO Adding `adata.uns['eigendecomposition_fwd']`
`.schur_vectors`
`.schur_matrix`
`.eigendecomposition`
Finish (5.23s)
INFO Computing 3 macrostates
INFO Adding `.macrostates`
`.macrostates_memberships`
`.coarse_T`
`.coarse_initial_distribution
`.coarse_stationary_distribution`
`.schur_vectors`
`.schur_matrix`
`.eigendecomposition`
Finish (0.35s)
| 4 | 0 | 1 |
|---|---|---|
| 0.044765 | 0.630805 | 0.324430 |
| 0.038285 | 0.000002 | 0.961712 |
| 0.005882 | 0.000000 | 0.994117 |
| 0.080887 | 0.888674 | 0.030438 |
| 0.040122 | 0.000003 | 0.959876 |
| 0.017220 | 0.000001 | 0.982779 |
| 0.031671 | 0.000002 | 0.968327 |
| 0.042806 | 0.000004 | 0.957190 |
| 0.052732 | 0.000003 | 0.947265 |
| 0.081947 | 0.893222 | 0.024832 |
| ... | ... | ... |
| 0.012248 | 0.033155 | 0.954597 |
| 0.082040 | 0.894221 | 0.023739 |
| 0.372067 | 0.000023 | 0.627910 |
| 0.296290 | 0.000018 | 0.703692 |
| 0.073425 | 0.837972 | 0.088603 |
| 0.019602 | 0.000001 | 0.980397 |
| 0.077290 | 0.859582 | 0.063129 |
| 0.051682 | 0.685097 | 0.263221 |
| 0.073585 | 0.800063 | 0.126352 |
| 0.022674 | 0.021644 | 0.955682 |
1776 cells x 3 lineages
g.predict_terminal_states()
g.plot_macrostates(which="terminal")
INFO Adding `adata.obs['term_states_fwd']`
`adata.obs['term_states_fwd_probs']`
`.terminal_states`
`.terminal_states_probabilities`
`.terminal_states_memberships
Finish`
g.plot_macrostates(which="terminal", same_plot=False)
As a next step we will calculate the fate probabilities. For each patient visit, this computes the probability of being absorbed in any of the terminal states by aggregating over all random walks that start in a given patient visit and end in some terminal population.
g.compute_fate_probabilities(solver="direct", use_petsc=False)
g.plot_fate_probabilities()
INFO Computing fate probabilities
INFO Adding `adata.obsm['lineages_fwd']`
`.fate_probabilities`
Finish (0.36s)
The plot above combines fate probabilities towards all terminal states, each patient visit is colored according to its most likely fate, color intensity reflects the degree of fate priming.
g.plot_fate_probabilities(same_plot=False)
We can also visualize the fate probabilities jointly in a circular projection where each dot represents a patient visit, colored by cluster labels. Patient visits are arranged inside the circle according to their fate probabilities, fate biased visits are placed next to their corresponding corner while undetermined patient fates are placed in the middle.
edata.obsm.keys()
KeysView(AxisArrays with keys: X_pca, X_umap, X_diffmap, T_fwd_umap, schur_vectors_fwd, macrostates_fwd_memberships, term_states_fwd_memberships, lineages_fwd)
cr.pl.circular_projection(edata, keys="leiden_0_3", legend_loc="right")
Identification of driver features#
We uncover putative driver features by correlating fate probabilities with features using the compute_lineage_drivers() method. In other words, if a feature is systematically higher or lower in patient visits that are more or less likely to differentiate towards a given terminal state, respectively, then we call this feature a putative driver feature.
We calculate these driver features for our lineages:
%%capture
plt.rcParams["figure.figsize"] = (3, 3)
plt.rcParams["figure.dpi"] = 100
for lineage in g.macrostates_memberships._names:
drivers = g.compute_lineage_drivers(lineages=lineage)
edata.obs[f"fate_probs_{lineage}"] = g.fate_probabilities[lineage].X.flatten()
ep.pl.umap(
edata,
color=[f"fate_probs_{lineage}"] + list(drivers.index[:8]),
color_map="viridis",
s=50,
ncols=3,
vmax="p96",
)
INFO Adding `adata.varm['terminal_lineage_drivers']`
`.lineage_drivers`
Finish (2.42s)
INFO Adding `adata.varm['terminal_lineage_drivers']`
`.lineage_drivers`
Finish (0.01s)
INFO Adding `adata.varm['terminal_lineage_drivers']`
`.lineage_drivers`
Finish (0.00s)
The lineage 1_1 seems to have a lot of patients that deceased in hospital, are of high age and had a high platelet measurement, while lineage 1_2 consists of patients that deceased in hospital, had a high first SAPS I and SOFA score and lineage 5 consists of patients with a high number of days after ICU release.
Determining feature trends#
Given fate probabilities and a pseudotime, we can plot trajectory-specific feature trends. Specifically, we fit Generalized Additive Models (GAMs), weighting each observation’s (here patients) contribution to each trajectory according to its vector of fate probabilities. We start by initializing a model.
model = cr.models.GAM(edata)
With the model initialized, we can visualize feature dynamics along specific trajectories. Here, we have a closer look at a selection of features that were previously identified as lineage drivers.
cr.pl.gene_trends(
edata,
model,
[
"sapsi_first",
"sofa_first",
"age",
"weight_first",
"sodium_first",
"chloride_first",
"platelet_first",
"wbc_first",
],
time_key="dpt_pseudotime",
show_progress_bar=False,
same_plot=True,
ncols=2,
hide_cells=True,
figsize=(10, 12),
)
INFO Computing trends using 1 core(s)
INFO Finish (0.31s)
INFO Plotting trends
Conclusion#
In this tutorial we applied CellRank and ehrapy to identify patient visit trajectories from selected root clusters, computed macrostates of clusters and pointed out features that are driving those trajectories. Following that, we visualized feature trends across the pseudotime for the patient trajectories. In particular, we inspected trajectories of patients originating from cluster 0, which was defined by less severe features, and identified 3 major trajectories. Two trajectories (2_1 and 2_2) were terminated in a bad outcome cluster and were driven by severity features such as age, death and comorbidities.
As a next tutorial, we suggest to have a closer look at our survival analysis, continue with that tutorials or go back to our tutorial overview page.
References#
Raffa, J. (2016). Clinical data from the MIMIC-II database for a case study on indwelling arterial catheters (version 1.0). PhysioNet. https://doi.org/10.13026/C2NC7F.
Raffa J.D., Ghassemi M., Naumann T., Feng M., Hsu D. (2016) Data Analysis. In: Secondary Analysis of Electronic Health Records. Springer, Cham
Goldberger, A., Amaral, L., Glass, L., Hausdorff, J., Ivanov, P. C., Mark, R., … & Stanley, H. E. (2000). PhysioBank, PhysioToolkit, and PhysioNet: Components of a new research resource for complex physiologic signals. Circulation [Online]. 101 (23), pp. e215–e220.
Marius Lange, Volker Bergen, Michal Klein, Manu Setty, Bernhard Reuter, Mostafa Bakhti, Heiko Lickert, Meshal Ansari, Janine Schniering, Herbert B. Schiller, Dana Pe’er, and Fabian J. Theis. Cellrank for directed single-cell fate mapping. Nat. Methods, 2022. doi:10.1038/s41592-021-01346-6.
Lars Velten, Simon F. Haas, Simon Raffel, Sandra Blaszkiewicz, Saiful Islam, Bianca P. Hennig, Christoph Hirche, Christoph Lutz, Eike C. Buss, Daniel Nowak, Tobias Boch, Wolf-Karsten Hofmann, Anthony D. Ho, Wolfgang Huber, Andreas Trumpp, Marieke A. G. Essers, and Lars M. Steinmetz. Human haematopoietic stem cell lineage commitment is a continuous process. Nature Cell Biology, 19(4):271–281, 2017. doi:10.1038/ncb3493.
Bergen, V., Lange, M., Peidli, S. et al. Generalizing RNA velocity to transient cell states through dynamical modeling. Nat Biotechnol 38, 1408–1414 (2020). https://doi.org/10.1038/s41587-020-0591-3
Haghverdi, L., Büttner, M., Wolf, F. et al. Diffusion pseudotime robustly reconstructs lineage branching. Nat Methods 13, 845–848 (2016). https://doi.org/10.1038/nmeth.3971
Package versions#
!pip list