mistralai/LIDstral-Arabic开源了新模型

mistralai/LIDstral-Arabic · Hugging Face Hugging Face (https://huggingface.co/) Models (https://huggingface.co/models) Datasets (https://huggingface.co/datasets) Spaces (https://huggingface.co/spaces) Buckets new (https://huggingface.co/storage) Docs (https://huggingface.co/docs) Enterprise (https://huggingface.co/enterprise) Pricing (https://huggingface.co/pricing) Website Tasks (https://huggingface.co/tasks) HuggingChat (https://huggingface.co/chat) Collections (https://huggingface.co/collections) Languages (https://huggingface.co/languages) Organizations (https://huggingface.co/organizations) Community Blog (https://huggingface.co/blog) Posts (https://huggingface.co/posts) Daily Papers (https://huggingface.co/papers) Hardware (https://huggingface.co/hardware) Learn (https://huggingface.co/learn) Discord (https://huggingface.co/join/discord) Forum (https://discuss.huggingface.co/) GitHub (https://github.com/huggingface) Solutions Team & Enterprise (https://huggingface.co/enterprise) Hugging Face PRO (https://huggingface.co/pro) Enterprise Support (https://huggingface.co/support) Inference Providers (https://huggingface.co/inference/models) Inference Endpoints (https://huggingface.co/inference-endpoints) Storage Buckets (https://huggingface.co/storage) Log In (https://huggingface.co/login) Sign Up (https://huggingface.co/join) (https://huggingface.co/mistralai) mistralai (https://huggingface.co/mistralai) / LIDstral-Arabic (https://huggingface.co/mistralai/LIDstral-Arabic) Like 20 Follow Mistral AI_ 20.1k Text Classification (https://huggingface.co/models?pipeline_tag=text-classification) fastText (https://huggingface.co/models?library=fasttext) 46 languages language-identification (https://huggingface.co/models?other=language-identification) dialect-identification (https://huggingface.co/models?other=dialect-identification) License: apache-2.0 Model card (https://huggingface.co/mistralai/LIDstral-Arabic) Files Files and versions xet (https://huggingface.co/mistralai/LIDstral-Arabic/tree/main) Community 1 (https://huggingface.co/mistralai/LIDstral-Arabic/discussions) Deploy Copy to bucket new Use this model Instructions to use mistralai/LIDstral-Arabic with libraries, inference providers, notebooks, and local apps. Follow these links to get started. Libraries fastText (https://huggingface.co/mistralai/LIDstral-Arabic?library=fasttext) How to use mistralai/LIDstral-Arabic with fastText: from huggingface_hub import hf_hub_download import fasttext model = fasttext.load_model(hf_hub_download("mistralai/LIDstral-Arabic", "model.bin")) Notebooks Google Colab (https://huggingface.co/mistralai/LIDstral-Arabic/colab) Kaggle (https://huggingface.co/mistralai/LIDstral-Arabic/kaggle) LIDstral Arabic (https://huggingface.co/mistralai/LIDstral-Arabic#lidstral-arabic) Method (https://huggingface.co/mistralai/LIDstral-Arabic#method) Evaluation (https://huggingface.co/mistralai/LIDstral-Arabic#evaluation) Usage (https://huggingface.co/mistralai/LIDstral-Arabic#usage) Install (https://huggingface.co/mistralai/LIDstral-Arabic#install) Use the model: (https://huggingface.co/mistralai/LIDstral-Arabic#use-the-model) Limitations (https://huggingface.co/mistralai/LIDstral-Arabic#limitations) License (https://huggingface.co/mistralai/LIDstral-Arabic#license) (https://huggingface.co/mistralai/LIDstral-Arabic#lidstral-arabic) LIDstral Arabic LIDstral Arabic is a fast language and dialect classifier for text written in Arabic script. It identifies Modern Standard Arabic, all Arabic dialects, and non-Arabic languages that use the same script. We include these non-Arabic languages in training to help the model distinguish languages that share a script and avoid classifying text as Arabic based on its script alone. (https://huggingface.co/mistralai/LIDstral-Arabic#method) Method The model combines a fastText one-vs-all (OVA) classifier, an MLP stacker, and per-class isotonic calibration: fastText

Mistral AI mistralai/LIDstral-Arabic mistralai/LIDstral-Arabic https://huggingface.co/mistralai/LIDstral-Arabic