OpenMOSS-Team/Qwen3-30B-A3B-base-mla-topk8-rank128开源了新模型

OpenMOSS OpenMOSS-Team/Qwen3-30B-A3B-base-mla-topk8-rank128 OpenMOSS-Team/Qwen3-30B-A3B-base-mla-topk8-rank128 https://huggingface.co/OpenMOSS-Team/Qwen3-30B-A3B-base-mla-topk8-rank128

OpenMOSS-Team/Qwen3-30B-A3B-base-mla-topk8-rank128 · Hugging Face Hugging Face (https://huggingface.co/) Models (https://huggingface.co/models) Datasets (https://huggingface.co/datasets) Spaces (https://huggingface.co/spaces) Buckets new (https://huggingface.co/storage) Docs (https://huggingface.co/docs) Enterprise (https://huggingface.co/enterprise) Pricing (https://huggingface.co/pricing) Website Tasks (https://huggingface.co/tasks) HuggingChat (https://huggingface.co/chat) Collections (https://huggingface.co/collections) Languages (https://huggingface.co/languages) Organizations (https://huggingface.co/organizations) Community Blog (https://huggingface.co/blog) Posts (https://huggingface.co/posts) Daily Papers (https://huggingface.co/papers) Hardware (https://huggingface.co/hardware) Learn (https://huggingface.co/learn) Discord (https://huggingface.co/join/discord) Forum (https://discuss.huggingface.co/) GitHub (https://github.com/huggingface) Solutions Team & Enterprise (https://huggingface.co/enterprise) Hugging Face PRO (https://huggingface.co/pro) Enterprise Support (https://huggingface.co/support) Inference Providers (https://huggingface.co/inference/models) Inference Endpoints (https://huggingface.co/inference-endpoints) Storage Buckets (https://huggingface.co/storage) Log In (https://huggingface.co/login) Sign Up (https://huggingface.co/join) (https://huggingface.co/OpenMOSS-Team) OpenMOSS-Team (https://huggingface.co/OpenMOSS-Team) / Qwen3-30B-A3B-base-mla-topk8-rank128 (https://huggingface.co/OpenMOSS-Team/Qwen3-30B-A3B-base-mla-topk8-rank128) like 0 Follow OpenMOSS 1.18k Safetensors (https://huggingface.co/models?library=safetensors) qwen3_moe (https://huggingface.co/models?other=qwen3_moe) qwen3 (https://huggingface.co/models?other=qwen3) mla (https://huggingface.co/models?other=mla) mlafication (https://huggingface.co/models?other=mlafication) delta-weights (https://huggingface.co/models?other=delta-weights) License: apache-2.0 Model card (https://huggingface.co/OpenMOSS-Team/Qwen3-30B-A3B-base-mla-topk8-rank128) Files Files and versions xet (https://huggingface.co/OpenMOSS-Team/Qwen3-30B-A3B-base-mla-topk8-rank128/tree/main) Community (https://huggingface.co/OpenMOSS-Team/Qwen3-30B-A3B-base-mla-topk8-rank128/discussions) Copy to bucket new Qwen3-30B-A3B-base-mla-topk8-rank128 (https://huggingface.co/OpenMOSS-Team/Qwen3-30B-A3B-base-mla-topk8-rank128#qwen3-30b-a3b-base-mla-topk8-rank128) Variant (https://huggingface.co/OpenMOSS-Team/Qwen3-30B-A3B-base-mla-topk8-rank128#variant) Loading (https://huggingface.co/OpenMOSS-Team/Qwen3-30B-A3B-base-mla-topk8-rank128#loading) Files (https://huggingface.co/OpenMOSS-Team/Qwen3-30B-A3B-base-mla-topk8-rank128#files) License (https://huggingface.co/OpenMOSS-Team/Qwen3-30B-A3B-base-mla-topk8-rank128#license) (https://huggingface.co/OpenMOSS-Team/Qwen3-30B-A3B-base-mla-topk8-rank128#qwen3-30b-a3b-base-mla-topk8-rank128) Qwen3-30B-A3B-base-mla-topk8-rank128 This repository contains the attention-only incremental weights for an MLAfication conversion of Qwen/Qwen3-30B-A3B-Base (https://huggingface.co/Qwen/Qwen3-30B-A3B-Base) . It is not a standalone full-model checkpoint. (https://huggingface.co/OpenMOSS-Team/Qwen3-30B-A3B-base-mla-topk8-rank128#variant) Variant Field Value Base model Qwen/Qwen3-30B-A3B-Base Base revision 1b75feb79f60b8dc6c5bc769a898c206a1c6a4f9 MLA latent rank 128 RoPE dimensions per KV head 8 Training checkpoint step 8000 Delta tensors 288 Delta size 0.80 GiB SHA-256 8cc402cd6191ca2dd6491379305b722cb17b60f81b2d00085806d59d6b784de9 The delta contains only q_proj, k_r_proj, low-rank kv_proj, and QK-Norm parameters. Embeddings, MLP/MoE, output projection, LM head, and other base-model parameters are intentionally omitted. (https://huggingface.co/OpenMOSS-Team/Qwen3-30B-A3B-base-mla-topk8-rank128#loading) Loading Use the MHA2MLA-V2 (https://github.com/JT-Ushio/MHA2MLA-V2) implementation to instantiate the