MorphoScout

MorphoScout is a BBBC021 morphology-mining pipeline for evaluating how cell selection strategies affect perturbation profiling. It provides one maintained command-line entrypoint for segmentation, crop embedding, profile aggregation, selection experiments, MorphoDiff-Typicality scoring, and thesis figure reproduction.

The maintained pipeline is centered on the BBBC021 experiment family:

• A0: baseline Cellpose-SAM crop extraction and profiling
• A1: random cell selection
• A2: foreground-fraction selection
• A3: MorphoDiff-Typicality top-K cell selection

diffmining/ is retained as frozen legacy support code for MorphoDiff-Typicality. New runs should start from pipeline/run_pipeline.py, the experiment configs, and the documentation below.

Repository Layout

Path	Purpose
morphoscout/	Maintained library code for segmentation, features, selection, normalization, aggregation, evaluation, and workflows.
pipeline/run_pipeline.py	Canonical CLI for all maintained workflows.
configs/	Dataset paths and canonical experiment configs.
scripts/runners/	SLURM wrappers for the maintained BBBC021 runs.
scripts/helpers/	Small helper utilities used by the maintained workflow.
artifacts/	Frozen run inputs, including the BBBC021 matched-subset manifest.
paper_artifacts/	Figure-generation code, notebook entrypoint, and bundled figure fixtures.
docs/	Detailed pipeline and configuration reference.
tests/	Maintained test suite.

Installation

MorphoScout expects Python 3.10. A Conda environment is the most direct setup because the pipeline uses PyTorch, Cellpose, and CUDA-aware packages:

git clone <repo-url>
cd MorphoScout
conda env create -f environment.yaml
conda activate morphoscout
export PYTHONPATH="$PWD:${PYTHONPATH:-}"

Alternatively, create a Python 3.10 virtual environment and install the pinned requirements:

python3.10 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
export PYTHONPATH="$PWD:${PYTHONPATH:-}"

Cellpose-SAM requires Cellpose 4. The SLURM runners default to .venvs/cellpose4_compare for Cellpose-specific steps, or use MORPHOSCOUT_CELLPOSE4_SITE_PACKAGES when Cellpose 4 is available from another environment.

Set PYTHONPATH from the repository root before running the CLI from a source checkout. The SLURM runners do this automatically.

Data Setup

Edit configs/paths.yaml so the dataset paths match your local BBBC021 layout:

datasets:
  bbbc021:
    images: <data>/bbbc021_by_compound
    metadata: <data>/bbbc021_metadata
    moa_mapping: <data>/bbbc021_metadata/BBBC021_v1_moa.csv

Input Data Contract

Full pipeline reproduction expects the MorphoDiff-style BBBC021 preprocessing: the original DNA, F-actin, and B-tubulin fluorescence channels are mapped into RGB PNG fields of view, following the image preparation described in the MorphoDiff paper and repository. Point datasets.bbbc021.images at the processed image root:

<data>/bbbc021_by_compound/
  dmso/
    DMSO_0.0_Week4_27861_D02_3683.png
  taxol/
    taxol_0.3_Week4_27861_F02_3763.png

Each processed image should be an RGB PNG with the BBBC021 channels arranged consistently with the maintained Cellpose-SAM config: actin, tubulin, and DAPI. The MorphoDiff paper reports standard preprocessing at 512 x 512 resolution, with image values scaled/normalized for Stable Diffusion and BBBC021 self-standardized by image channel. The default segmentation config uses channels [2, 3], corresponding to Cellpose's 1-based channel indexing for tubulin/cytoplasm-like signal and DAPI/nuclei signal in these RGB composites.

The processed image filename is parsed as:

<compound>_<concentration>_<week>_<plate>_<well>_<site>.png

The metadata directory should contain the BBBC021 metadata CSVs used to map images to perturbations and mechanisms of action:

<data>/bbbc021_metadata/
  BBBC021_v1_image.csv
  BBBC021_v1_moa.csv

The maintained experiments use the frozen matched-subset manifest:

artifacts/bbbc021_matched_subset.csv

This manifest is the reproducibility anchor for the paper-scale BBBC021 runs. It records the retained fields of view, compound/concentration perturbation keys, MoA labels, control status, and the processed image path used by the maintained experiments. Keep --matched-subset-csv artifacts/bbbc021_matched_subset.csv in full-run commands when reproducing the reported subset.

The subset was built to match the MorphoDiff-supported BBBC021 chemical space used by the typicality workflow. Retained active treatments have processed images, BBBC021 MoA annotations, and a usable MorphoDiff perturbation embedding derived from the available chemical representation; DMSO controls are retained as the baseline/control condition. BBBC021 treatments without usable embedding support, including compounds whose structures are unavailable or not represented in the MorphoDiff embedding table, are outside this maintained subset.

For environment-specific paths, prefer environment variables or local untracked config copies instead of changing the canonical experiment configs.

Pipeline Quick Start

All maintained workflows run through:

export PYTHONPATH="$PWD:${PYTHONPATH:-}"
python pipeline/run_pipeline.py --workflow <workflow> --config <config>

Baseline segmentation and embedding:

python pipeline/run_pipeline.py \
  --workflow standard \
  --config configs/experiments/A0.yaml \
  --output-dir <results>/A0 \
  --matched-subset-csv artifacts/bbbc021_matched_subset.csv \
  --phase all

Run only proposal detection:

python pipeline/run_pipeline.py \
  --workflow standard \
  --config configs/experiments/A0.yaml \
  --output-dir <results>/A0 \
  --matched-subset-csv artifacts/bbbc021_matched_subset.csv \
  --phase detect

Random selection:

python pipeline/run_pipeline.py \
  --workflow a1 \
  --config configs/experiments/A1.yaml \
  --output-dir <results>/A1/k40 \
  --matched-subset-csv artifacts/bbbc021_matched_subset.csv \
  --percent 40 \
  --seed 42

Foreground-fraction selection:

python pipeline/run_pipeline.py \
  --workflow a2 \
  --config configs/experiments/A2.yaml \
  --output-dir <results>/A2/k40 \
  --matched-subset-csv artifacts/bbbc021_matched_subset.csv \
  --percent 40 \
  --seed 42

MorphoDiff-Typicality selection:

python pipeline/run_pipeline.py \
  --workflow a3 \
  --config configs/experiments/A3.yaml \
  --output-dir <results>/A3/cell_mask/k40 \
  --matched-subset-csv artifacts/bbbc021_matched_subset.csv \
  --scoring-method cell_mask \
  --percent 40

Precompute the maintained population-null typicality score cache:

python pipeline/run_pipeline.py \
  --workflow typicality \
  --config configs/experiments/typicality/bbbc021/population_mc16_patch__tvncell_week.yaml \
  --matched-subset-csv artifacts/bbbc021_matched_subset.csv \
  --scores-only

Evaluate precomputed profiles:

python pipeline/run_pipeline.py \
  --workflow profiles \
  --config configs/experiments/A0.yaml \
  --profiles <profiles.csv> \
  --output-dir <results>/profiles_eval

Run baseline evaluation from precomputed embeddings:

python pipeline/run_pipeline.py \
  --workflow baseline \
  --config configs/experiments/A0.yaml \
  --embeddings-dir <results>/A0/extraction/embeddings/inception \
  --output-dir <results>/A0/baseline \
  --matched-subset-csv artifacts/bbbc021_matched_subset.csv

For the complete CLI flag reference, workflow requirements, config schema, and environment overrides, see docs/config-reference.md.

SLURM Runners

Cluster runs are wrapped in scripts/runners/. Start with:

less scripts/runners/README.md

Common overrides include:

• MORPHOSCOUT_RESULTS_ROOT
• MORPHOSCOUT_MATCHED_SUBSET_CSV
• MORPHOSCOUT_VENV
• MORPHOSCOUT_CELLPOSE4_VENV
• MORPHOSCOUT_SCORE_CACHE_DIR
• MORPHOSCOUT_SWEEP_PERCENTS
• MORPHOSCOUT_SWEEP_SEEDS

Thesis Figure Reproduction

paper_artifacts/ contains the figure-generation entrypoints and bundled fixtures.

Preview all active figures:

cd paper_artifacts
python build_paper_figures.py --all

Save one figure:

cd paper_artifacts
python build_paper_figures.py --figure foreground_typicality_agreement --save

The notebook entrypoint is paper_artifacts/paper_figures.ipynb. See paper_artifacts/README.md for the fixture contract, cached figure behavior, and supported path overrides.

Development Checks

Run the test suite from the repository root:

pytest

Useful documentation and hygiene checks:

python pipeline/run_pipeline.py --help
git status --short