tipsv1-so400m14

Model release Hugging Face 模型 · google Source language: Chinese

官方机构模型仓库创建于 2026-08-19。仓库创建时间不等于正式发布日期。任务类型:zero-shot-image-classification

Full text

TIPS — So400m/14 (v1)

TIPS (Text-Image Pre-training with Spatial awareness, ICLR 2025) is a family of contrastive vision-language models that produce spatially rich image features aligned with text embeddings. This is the original (v1) So400m/14 release with 413M vision params and 448M text params, converted from the official checkpoints.

Variant Vision params Text params Embed dim Resolution S/14 22M 34M 384 448 B/14 86M 110M 768 448 L/14 304M 184M 1024 448 So400m/14 413M 448M 1152 448 g/14 1.1B 389M 1536 448 g/14 low-res 1.1B 389M 1536 224

Usage

Load the model

Encode images

Images should be tensors in [0, 1] range (just ToTensor(), no ImageNet normalization).

The second CLS token (out.register_tokens) was trained on synthetic captions; the first (out.cls_token) on web alt-text, and is the one aligned with the text tower.

Encode text

Zero-shot classification

Visualize spatial features

Model details

ViT-So400m/14 vision encoder (27 layers, patch size 14, two CLS tokens) + 27-layer transformer text encoder Native resolution 448; other patch-multiple resolutions work via positional-embedding interpolation Preprocessing: images to [0, 1], no normalization; SentencePiece tokenizer, lowercased, max 64 tokens

License

Apache 2.0

Sources

← Back to the daily brief