Unmasking Face Embeddings:
Reading, Rendering and Naming with Foundation Models

European Conference on Computer Vision (ECCV) Workshops, 2026

Michigan State University

One precomputed linear map turns an FR identity embedding (e.g., ArcFace) into a foundation-model embedding (e.g., CLIP), fit once with no further training. The same template can then be read in natural language, rendered into a face, and matched to a name.

Abstract

Modern face recognition (FR) owes much of its success to deep neural networks that learn compact identity embeddings from face images. These embeddings are highly effective for biometric matching but largely opaque to semantic interpretation. In contrast, foundation models, pretrained on broad visual or vision-language tasks, provide rich interfaces for describing, retrieving, generating, and organizing visual content. This raises a question: what capabilities become available when face embeddings from domain-specific FR models are made interoperable with foundation models? Building on recent work on embedding compatibility, we use simple pre-computed linear transformations, estimated from paired embeddings alone, to connect existing FR models with off-the-shelf foundation models. Once aligned, a face embedding can be “unmasked” in multiple ways, without training or modifying either model: it can be read in natural language, enabling free-form text queries over a gallery of FR embeddings; rendered into a face image that recovers a person's appearance, using an unmodified diffusion decoder; and converted to a name, enabling identification even without an enrolled face gallery. In effect, one linear transformation turns an identity embedding into a rich embedding for web-scale foundation models, exposing face embeddings as semantically and visually rich representations, with implications for interpretability, retrieval, reconstruction, and template security.

Read: free-form text queries

Natural-language searches over a gallery of FR embeddings. After alignment, queries such as “a smiling young woman with blond hair” rank the gallery almost as well as querying the foundation model natively.

Render: a face from the embedding alone

A recognizable face decoded from the FR embedding alone, through an unmodified diffusion decoder. The reconstructions recover a person's appearance and retain identity.

Name: identity without a gallery

The template is matched against a text vocabulary of names, with no enrolled face gallery. Aligned FR templates name held-out people far above chance, while unaligned and random maps sit at chance.

BibTeX

@inproceedings{rubab2026unmasking,
  title     = {Unmasking Face Embeddings: Reading, Rendering and Naming with Foundation Models},
  author    = {Rubab, Fizza and Tong, Yiying and Ross, Arun},
  booktitle = {European Conference on Computer Vision Workshops (ECCVW)},
  year      = {2026}
}

This website is licensed under a Creative Commons Attribution 4.0 International License.

Unmasking Face Embeddings:Reading, Rendering and Naming with Foundation Models