nvidia/PixelUMM开源了新模型

NVIDIA nvidia/PixelUMM nvidia/PixelUMM https://huggingface.co/nvidia/PixelUMM

nvidia/PixelUMM · Hugging Face Hugging Face (https://huggingface.co/) Models (https://huggingface.co/models) Datasets (https://huggingface.co/datasets) Spaces (https://huggingface.co/spaces) Buckets new (https://huggingface.co/storage) Docs (https://huggingface.co/docs) Enterprise (https://huggingface.co/enterprise) Pricing (https://huggingface.co/pricing) Website Tasks (https://huggingface.co/tasks) HuggingChat (https://huggingface.co/chat) Collections (https://huggingface.co/collections) Languages (https://huggingface.co/languages) Organizations (https://huggingface.co/organizations) Community Blog (https://huggingface.co/blog) Posts (https://huggingface.co/posts) Daily Papers (https://huggingface.co/papers) Hardware (https://huggingface.co/hardware) Learn (https://huggingface.co/learn) Discord (https://huggingface.co/join/discord) Forum (https://discuss.huggingface.co/) GitHub (https://github.com/huggingface) Solutions Team & Enterprise (https://huggingface.co/enterprise) Hugging Face PRO (https://huggingface.co/pro) Enterprise Support (https://huggingface.co/support) Inference Providers (https://huggingface.co/inference/models) Inference Endpoints (https://huggingface.co/inference-endpoints) Storage Buckets (https://huggingface.co/storage) Log In (https://huggingface.co/login) Sign Up (https://huggingface.co/join) (https://huggingface.co/nvidia) nvidia (https://huggingface.co/nvidia) / PixelUMM (https://huggingface.co/nvidia/PixelUMM) Like 73 Follow NVIDIA 70.7k Image-to-Text (https://huggingface.co/models?pipeline_tag=image-to-text) multimodal (https://huggingface.co/models?other=multimodal) image-generation (https://huggingface.co/models?other=image-generation) video-generation (https://huggingface.co/models?other=video-generation) video-text-to-text (https://huggingface.co/models?other=video-text-to-text) License: nvidia-one-way-noncommercial-license Model card (https://huggingface.co/nvidia/PixelUMM) Files Files and versions xet (https://huggingface.co/nvidia/PixelUMM/tree/main) Community 3 (https://huggingface.co/nvidia/PixelUMM/discussions) Copy to bucket new PixelUMM (https://huggingface.co/nvidia/PixelUMM#pixelumm) Overview (https://huggingface.co/nvidia/PixelUMM#overview) Model Architecture (https://huggingface.co/nvidia/PixelUMM#model-architecture) Inputs and Outputs (https://huggingface.co/nvidia/PixelUMM#inputs-and-outputs) Installation (https://huggingface.co/nvidia/PixelUMM#installation) Intended Use (https://huggingface.co/nvidia/PixelUMM#intended-use) Limitations and Safety (https://huggingface.co/nvidia/PixelUMM#limitations-and-safety) License (https://huggingface.co/nvidia/PixelUMM#license) References (https://huggingface.co/nvidia/PixelUMM#references) (https://huggingface.co/nvidia/PixelUMM#pixelumm) PixelUMM PixelUMM: Encoder-Free Unified Image and Video Understanding and Generation PixelUMM is an NVIDIA-developed, encoder-free unified multimodal model for joint understanding and generation across text, images, and video directly in pixel space. (https://huggingface.co/nvidia/PixelUMM/blob/main/assets/pixelumm-teaser.png) PixelUMM is released for non-commercial research or evaluation purposes only. (https://huggingface.co/nvidia/PixelUMM#overview) Overview Images are represented as pixel patches instead of embeddings from a separate pretrained vision encoder, allowing understanding and generation to share a Transformer-based representation. PixelUMM supports: text-to-image generation; image-conditioned text generation; video-conditioned understanding; and video generation. (https://huggingface.co/nvidia/PixelUMM#model-architecture) Model Architecture Architecture: Decoder-only Transformer with raw-pixel patch embeddings and an iterative pixel-generation head. Language backbone: Qwen/Qwen3-8B (https://huggingface.co/Qwen/Qwen3-8B) , revision b968826d9c46dd6066d109eabc6255188de91218. Image representation: RGB images divided into 16-by-16 pixel patche