Note

This page was generated from mimic_2_fate.ipynb. Some tutorial content may look better in light mode.

MIMIC-II IAC Patient Fate

In the previous introduction tutorial, we explored the MIMIC-II IAC dataset, comprising electronic health records (EHR) of 1776 patients in 46 features, and identified patient group-specific clusters using ehrapy. Please go through the MIMIC-II IAC introduction before performing this tutorial to get familiar with the dataset.

As a next step, we want to determine patient fate. The goal is to detect terminal states and the corresponding origins based on pseudotime. Real time very rarely reflects the actual progression of a disease. When measurements are done on a certain day, some patients will show no sign of disease (e.g. healthy or recovered), some are at the onset of a specific disease and some are in a more severe stage or even at the height. For an appropriate analysis, we are interested in a continuous transition of states, such as from healthy to diseased to death, for which the real time is therefore not informative. Identification of transition states can be achieved by identifying source states (e.g. healthy) and then calculating pseudotime from this state. Based on Markov chain modelling, we uncover patient dynamics using CellRank. For more details, please read CellRank paper 1 and CellRank paper 2.

In this tutorial we will be using CellRank to:

1. Simulate patient trajectories with random walks.

2. Compute patient macrostates and infer fate probabilities towards predicted terminal states.

3. Identify potential driver features for each identified trajectory.

4. Visualize feature trends along specific patient states, while accounting for the continuous nature of fate determination.

Before performing this tutorial, we highly recommend to read the extensive and well written CellRank documentation, especially the general tutorial chapter is useful. If you are not familiar with single-cell data, do not be afraid and replace cells with patients visits and genes with features in your mind.

This tutorial requires cellrank to be installed. As this packages is not a dependency of ehrapy, it must be installed separately.

%%capture --no-display
!pip install cellrank

Before we start with the patient fate analysis of the MIMIC-II IAC dataset, we set up our environment including the import of packages and preparation of the dataset.

---

Environment setup

Ensure that the latest version of ehrapy is installed. A list of all dependency versions can be found at the end of this tutorial.

import ehrapy as ep
from matplotlib import rcParams
import cellrank as cr
import numpy as np

/home/zeth/miniconda3/envs/ehrapy/lib/python3.11/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm

We are ignoring a few warnings for readability reasons.

import warnings

warnings.filterwarnings("ignore")

---

Getting and preprocessing the MIMIC-II dataset

This tutorial is based on the MIMIC-II IAC dataset which was previously introduced in the MIMIC-II IAC introduction tutorial. We will load the encoded version of the dataset as an AnnData object, ehrapy’s default encoding is a simple one-hot encoding in this case.

%%capture
adata = ep.dt.mimic_2(encoded=True)
adata

2023-12-19 11:30:31,347 - root INFO - Transformed passed DataFrame into an AnnData object with n_obs x n_vars = `1776` x `46`.
2023-12-19 11:30:31,350 - root INFO - The original categorical values `['day_icu_intime', 'service_unit']` were added to uns.
2023-12-19 11:30:31,375 - root INFO - Encoding strings in X to save to .h5ad. Loading the file will reverse the encoding.
2023-12-19 11:30:31,381 - root INFO - Updated the original layer after encoding.
2023-12-19 11:30:31,410 - root INFO - The original categorical values `['day_icu_intime', 'service_unit']` were added to obs.

The MIMIC-II dataset has 1776 patients with 46 features. Now that we have our AnnData object ready, we need to perform the standard preprocessing steps as performed in the introduction tutorial again before we can use ehrapy and CellRank for patient fate analysis.

%%capture
ep.pp.knn_impute(adata, n_neighbours=5)
ep.pp.log_norm(adata, vars=["iv_day_1", "po2_first"], offset=1)
ep.pp.pca(adata)
ep.pp.neighbors(adata, n_pcs=10)
ep.tl.umap(adata)
ep.tl.leiden(adata, resolution=0.3, key_added="leiden_0_3")

2023-12-19 11:30:33,160 - root INFO - Values in columns ['iv_day_1', 'po2_first'] were replaced by [[5.28320373 7.71059732]
 [5.19739145 6.39859493]
 [5.70044357 7.6438663 ]
 ...
 [5.75700724 5.06575459]
 [4.44265126 3.1254439 ]
 [6.35437004 8.38228943]].

ep.settings.set_figure_params(figsize=(5, 4), dpi=100)
ep.pl.umap(adata, color=["leiden_0_3"], title="Leiden 0.3", size=20)

This UMAP embedding is exactly the same as previously computed in the MIMIC-II IA introduction tutorial. Now we continue with the patient fate analysis.

---

Analysis using ehrapy and CellRank

Depending on the data it may not always be possible to clearly define a cluster or specific patient visits as the origin or terminus of a trajectory. Working with single-cell data simplifies matters since the detection of stem cells generally signifies the start of cell differentiation.

In this tutorial, we will define a patient cluster as the origin (root cluster) and explore possible terminal states.

Pseudotime calculation

As the root cluster for pseudotime calculation we choose cluster 0 since patients in that cluster do not show very severe comorbidities and features yet. Then we calculate the Diffusion Pseudotime with the el.dpt() function.

adata.uns["iroot"] = np.flatnonzero(adata.obs["leiden_0_3"] == "0")[0]
ep.tl.dpt(adata)

WARNING: Trying to run `tl.dpt` without prior call of `tl.diffmap`. Falling back to `tl.diffmap` with default parameters.

Now we define the kernel, compute the transition matrix and plot a projection onto the UMAP.

Determining patient fate with a PseudotimeKernel

The PseudotimeKernel computes direct transition probabilities based on a KNN graph and pseudotime.

The KNN graph contains information about the (undirected) conductivities among observations (here patients), reflecting their similarity. Pseudotime can be used to either remove edges that point against the direction of increasing pseudotime, or to downweight them.

from cellrank.kernels import PseudotimeKernel

pk = PseudotimeKernel(adata, time_key="dpt_pseudotime")
pk.compute_transition_matrix()

 13%|█▎        | 223/1776 [00:00<00:03, 507.22cell/s]

100%|██████████| 1776/1776 [00:03<00:00, 518.37cell/s]

PseudotimeKernel[n=1776, dnorm=False, scheme='hard', frac_to_keep=0.3]

ep.settings.set_figure_params(figsize=(5, 4), dpi=100)
pk.plot_projection(basis="umap", color="leiden_0_3")

We observe two main trajectories originating from cluster 0 going to clusters 2 and 5. Let’s check the metadata again.

ep.pl.umap(
    adata, color="censor_flg", title="Censored or Death (0 = death, 1 = censored)"
)
ep.pl.umap(
    adata,
    color="mort_day_censored",
    title="Day post ICU admission of censoring or death",
)

Cluster 2 consists of patients that deceased and had severe comorbidities while cluster 5 includes patients with a high day post ICU admission.

Simulating transitions with random walks

Cellrank makes it easy to simulate the behavior of random walks from specific clusters. This allows us to not only visualize where the patients end up, but also roughly how many in which clusters after a defined number of iterations. We can either just start walking…

pk.plot_random_walks(
    seed=0,
    n_sims=100,
    start_ixs={"leiden_0_3": ["0"]},
    legend_loc="right",
    dpi=100,
    show_progress_bar=False,
)

… or set a number of required hits in one or more terminal clusters. Here, we require 50 hits in cluster 2 or 5.

pk.plot_random_walks(
    seed=0,
    n_sims=100,
    start_ixs={"leiden_0_3": ["0"]},
    stop_ixs={"leiden_0_3": ["2", "5"]},
    successive_hits=50,
    legend_loc="right",
    dpi=100,
    show_progress_bar=False,
)

Black and yellow dots indicate random walk start and terminal patient visits, respectively.

Determining macrostates and terminal states

To find the terminal states of cluster 0, well will use an estimator to predict the patient fates using the above calculated transition matrix. The main objective is to decompose the patient state space into a set of macrostates, that represent the slow-time scale dynamics of the process and predict terminal states. Here, we will use an Generalized Perron Cluster Cluster Analysis (GPCCA) estimator.

As a first step we try to identify macrostates in the data using the fit() function.

g = cr.estimators.GPCCA(pk)
g.fit(cluster_key="leiden_0_3")
g.macrostates_memberships

WARNING: Unable to import `petsc4py` or `slepc4py`. Using `method='brandts'`
WARNING: For `method='brandts'`, dense matrix is required. Densifying

2_1	2_2	4
0.269917	0.729932	0.000151
0.031573	0.197435	0.770992
0.032702	0.190194	0.777104
0.072695	0.927274	0.000031
0.034734	0.852121	0.113146
0.024950	0.434995	0.540056
0.026109	0.737378	0.236513
0.069814	0.868942	0.061244
0.030265	0.218082	0.751653
0.017000	0.982999	0.000001
...	...	...
0.084016	0.915038	0.000945
0.018946	0.981051	0.000003
0.007253	0.064232	0.928516
0.029628	0.131998	0.838374
0.138710	0.861214	0.000076
0.024621	0.482767	0.492612
0.101163	0.898778	0.000060
0.232216	0.767656	0.000128
0.010151	0.989821	0.000027
0.284933	0.709793	0.005274

1776 cells x 3 lineages

g.predict_terminal_states()
g.plot_macrostates(which="terminal")

g.plot_macrostates(which="terminal", same_plot=False)

As a next step we will calculate the fate probabilities. For each patient visit, this computes the probability of being absorbed in any of the terminal states by aggregating over all random walks that start in a given patient visit and end in some terminal population.

g.compute_fate_probabilities(preconditioner="ilu", tol=1e-15)
g.plot_fate_probabilities()

WARNING: Unable to import petsc4py. For installation, please refer to: https://petsc4py.readthedocs.io/en/stable/install.html.
Defaulting to `'gmres'` solver.

100%|██████████| 3/3 [02:06<00:00, 42.27s/]

WARNING: `3` solution(s) did not converge

The plot above combines fate probabilities towards all terminal states, each patient visit is colored according to its most likely fate, color intensity reflects the degree of fate priming.

g.plot_fate_probabilities(same_plot=False)

We can also visualize the fate probabilities jointly in a circular projection where each dot represents a patient visit, colored by cluster labels. Patient visits are arranged inside the circle according to their fate probabilities, fate biased visits are placed next to their corresponding corner while undetermined patient fates are placed in the middle.

cr.pl.circular_projection(adata, keys="leiden_0_3", legend_loc="right")

/home/zeth/miniconda3/envs/ehrapy/lib/python3.11/site-packages/scvelo/plotting/utils.py:63: DeprecationWarning: is_categorical_dtype is deprecated and will be removed in a future version. Use isinstance(dtype, pd.CategoricalDtype) instead
  return isinstance(c, str) and c in data.obs.keys() and cat(data.obs[c])
/home/zeth/miniconda3/envs/ehrapy/lib/python3.11/site-packages/scvelo/plotting/utils.py:63: DeprecationWarning: is_categorical_dtype is deprecated and will be removed in a future version. Use isinstance(dtype, pd.CategoricalDtype) instead
  return isinstance(c, str) and c in data.obs.keys() and cat(data.obs[c])

Identification of driver features

We uncover putative driver features by correlating fate probabilities with features using the compute_lineage_drivers() method. In other words, if a feature is systematically higher or lower in patient visits that are more or less likely to differentiate towards a given terminal state, respectively, then we call this feature a putative driver feature.

We calculate these driver features for our three lineages 2_1, 2_2 and 5.

%%capture
ep.settings.set_figure_params(figsize=(3, 3), dpi=100)

drivers_2_1 = g.compute_lineage_drivers(lineages="2_1")
adata.obs["fate_probs_2_1"] = g.fate_probabilities["2_1"].X.flatten()

ep.pl.umap(
    adata,
    color=["fate_probs_2_1"] + list(drivers_2_1.index[:8]),
    color_map="viridis",
    s=50,
    ncols=3,
    vmax="p96",
)

drivers_2_2 = g.compute_lineage_drivers(lineages="2_2")
adata.obs["fate_probs_2_2"] = g.fate_probabilities["2_2"].X.flatten()

ep.pl.umap(
    adata,
    color=["fate_probs_2_2"] + list(drivers_2_2.index[:8]),
    color_map="viridis",
    s=50,
    ncols=3,
    vmax="p96",
)

drivers_4 = g.compute_lineage_drivers(lineages="4")
adata.obs["fate_probs_4"] = g.fate_probabilities["4"].X.flatten()

ep.pl.umap(
    adata,
    color=["fate_probs_4"] + list(drivers_4.index[:8]),
    color_map="viridis",
    s=50,
    ncols=3,
    vmax="p96",
)

The lineage 2_1 seems to have a lot of patients that deceased in hospital, are of high age and had a high platelet measurement, while lineage 2_2 consists of patients that deceased in hospital, had a high first SAPS I and SOFA score and lineage 5 consists of patients with a high number of days after ICU release.

Determining feature trends

Given fate probabilities and a pseudotime, we can plot trajectory-specific feature trends. Specifically, we fit Generalized Additive Models (GAMs), weighting each observation’s (here patients) contribution to each trajectory according to its vector of fate probabilities. We start by initializing a model.

model = cr.models.GAM(adata)

With the model initialized, we can visualize feature dynamics along specific trajectories. Here, we have a closer look at a selection of features that were previously identified as lineage drivers.

cr.pl.gene_trends(
    adata,
    model,
    [
        "sapsi_first",
        "sofa_first",
        "age",
        "weight_first",
        "sodium_first",
        "chloride_first",
        "platelet_first",
        "wbc_first",
    ],
    time_key="dpt_pseudotime",
    show_progress_bar=False,
    same_plot=True,
    ncols=2,
    hide_cells=True,
    figsize=(10, 12),
)

  0%|          | 0/8 [00:00<?, ?gene/s]

100%|██████████| 8/8 [00:01<00:00,  6.14gene/s]

---

Conclusion

In this tutorial we applied CellRank and ehrapy to identify patient visit trajectories from selected root clusters, computed macrostates of clusters and pointed out features that are driving those trajectories. Following that, we visualized feature trends across the pseudotime for the patient trajectories. In particular, we inspected trajectories of patients originating from cluster 0, which was defined by less severe features, and identified 3 major trajectories. Two trajectories (2_1 and 2_2) were terminated in a bad outcome cluster and were driven by severity features such as age, death and comorbidities.

As a next tutorial, we suggest to have a closer look at our survival analysis, continue with that tutorials or go back to our tutorial overview page.

---

References

• Raffa, J. (2016). Clinical data from the MIMIC-II database for a case study on indwelling arterial catheters (version 1.0). PhysioNet. https://doi.org/10.13026/C2NC7F.

• Raffa J.D., Ghassemi M., Naumann T., Feng M., Hsu D. (2016) Data Analysis. In: Secondary Analysis of Electronic Health Records. Springer, Cham

• Goldberger, A., Amaral, L., Glass, L., Hausdorff, J., Ivanov, P. C., Mark, R., … & Stanley, H. E. (2000). PhysioBank, PhysioToolkit, and PhysioNet: Components of a new research resource for complex physiologic signals. Circulation [Online]. 101 (23), pp. e215–e220.

• Marius Lange, Volker Bergen, Michal Klein, Manu Setty, Bernhard Reuter, Mostafa Bakhti, Heiko Lickert, Meshal Ansari, Janine Schniering, Herbert B. Schiller, Dana Pe’er, and Fabian J. Theis. Cellrank for directed single-cell fate mapping. Nat. Methods, 2022. doi:10.1038/s41592-021-01346-6.

• Lars Velten, Simon F. Haas, Simon Raffel, Sandra Blaszkiewicz, Saiful Islam, Bianca P. Hennig, Christoph Hirche, Christoph Lutz, Eike C. Buss, Daniel Nowak, Tobias Boch, Wolf-Karsten Hofmann, Anthony D. Ho, Wolfgang Huber, Andreas Trumpp, Marieke A. G. Essers, and Lars M. Steinmetz. Human haematopoietic stem cell lineage commitment is a continuous process. Nature Cell Biology, 19(4):271–281, 2017. doi:10.1038/ncb3493.

• Bergen, V., Lange, M., Peidli, S. et al. Generalizing RNA velocity to transient cell states through dynamical modeling. Nat Biotechnol 38, 1408–1414 (2020). https://doi.org/10.1038/s41587-020-0591-3

• Haghverdi, L., Büttner, M., Wolf, F. et al. Diffusion pseudotime robustly reconstructs lineage branching. Nat Methods 13, 845–848 (2016). https://doi.org/10.1038/nmeth.3971

---

Package versions

%%capture --no-display
ep.print_versions()

Unable to fetch versions for one or more dependencies.