Image Embeddings and Similar Cases

What it does

Image Embeddings and Similar Cases turns a medical image or series into a vector: a compact numerical representation in which images with similar content lie close together. Upload one image or one series; the bare model ID returns the embedding together with the encoder and its dimension, ready to store in a vector index.

Two related tools build on the same vectors. Similar-case search returns the closest cases from your organisation's own case library, each with its report, for comparison and teaching. Zero-shot labelling scores an image against a list of labels you write yourself, without training a classifier first.

Intended use

A building block for developers and researchers, and a reference aid for clinicians. Embeddings support search, clustering, few-shot classifiers and the curation of training data. Similar cases help a resident or reader see how comparable studies were reported.

Who it is for

Developers building search, triage or data-curation pipelines; researchers who need consistent image features across a cohort; departments that maintain teaching files and want to find comparable cases in their own archive.

Accepted input

• Images: one image or one series per request, from any modality MedRun accepts: radiographs, CT and MR volumes, ultrasound, fundus and other photographs, pathology tiles; also NIfTI, PNG and JPEG.
• Case library (similar-case search): the cases your organisation has uploaded, with their reports.
• Label list (zero-shot labelling): the labels as short text phrases.

Requirements

Requirement	Why
The same encoder and version for every vector you compare	Embedding spaces differ between encoders, so mixed vectors are not comparable
Re-indexing after an encoder upgrade	A new encoder version produces a new embedding space
One series per volume request	3D data embedded with a 2D encoder is pooled across slices of that series
A library of reported cases (similar-case search)	Retrieval quality depends on the library more than on the encoder

The modality is recognised from the series, and the encoder suited to it is chosen; the result names the encoder and version, so stored vectors stay traceable.

Optional context

For similar-case search, the number of neighbours to return can be set; for zero-shot labelling, the label list is the only context the run needs.

The result

Output	Content
Embedding	The vector, or a reference to it, with encoder name, version and dimension
Source	The image or series the vector was computed from, and any slice pooling applied

The vector is returned in the JSON result under embedding, so it can be written straight to a vector database.

Standards

• JSON: embedding with model, dimension and vector or reference; neighbors[] with case reference and similarity score for similar-case search; classifications[] for zero-shot labels.
• DICOM: Key Object Selection documents referencing the similar cases, so a PACS viewer can open them.
• Inputs: DICOM or the converted NIfTI, PNG and JPEG forms of the same images.

Choosing between the tools

Use the embedding when you build your own index or classifier and need raw features. Use similar-case search when you want comparable reported cases from your library without running your own index. Use zero-shot labelling to sort or filter images by labels you define before investing in a trained model.

Related tools

• universal/embed/similar-cases: top-k most similar cases from your organisation's library, with their reports and DICOM Key Object Selection references.
• universal/embed/classify: probabilities for a user-defined list of labels, without training.