amd/Llama-3.1-8B-Instruct-w4a16-llmcompressor-v0.12.0开源了新模型

AMD amd/Llama-3.1-8B-Instruct-w4a16-llmcompressor-v0.12.0 amd/Llama-3.1-8B-Instruct-w4a16-llmcompressor-v0.12.0 https://huggingface.co/amd/Llama-3.1-8B-Instruct-w4a16-llmcompressor-v0.12.0

amd/Llama-3.1-8B-Instruct-w4a16-llmcompressor-v0.12.0 · Hugging Face Hugging Face (https://huggingface.co/) Models (https://huggingface.co/models) Datasets (https://huggingface.co/datasets) Spaces (https://huggingface.co/spaces) Buckets new (https://huggingface.co/storage) Docs (https://huggingface.co/docs) Enterprise (https://huggingface.co/enterprise) Pricing (https://huggingface.co/pricing) Website Tasks (https://huggingface.co/tasks) HuggingChat (https://huggingface.co/chat) Collections (https://huggingface.co/collections) Languages (https://huggingface.co/languages) Organizations (https://huggingface.co/organizations) Community Blog (https://huggingface.co/blog) Posts (https://huggingface.co/posts) Daily Papers (https://huggingface.co/papers) Hardware (https://huggingface.co/hardware) Learn (https://huggingface.co/learn) Discord (https://huggingface.co/join/discord) Forum (https://discuss.huggingface.co/) GitHub (https://github.com/huggingface) Solutions Team & Enterprise (https://huggingface.co/enterprise) Hugging Face PRO (https://huggingface.co/pro) Enterprise Support (https://huggingface.co/support) Inference Providers (https://huggingface.co/inference/models) Inference Endpoints (https://huggingface.co/inference-endpoints) Storage Buckets (https://huggingface.co/storage) Log In (https://huggingface.co/login) Sign Up (https://huggingface.co/join) (https://huggingface.co/amd) amd (https://huggingface.co/amd) / Llama-3.1-8B-Instruct-w4a16-llmcompressor-v0.12.0 (https://huggingface.co/amd/Llama-3.1-8B-Instruct-w4a16-llmcompressor-v0.12.0) like 0 Follow AMD 3.07k Text Generation (https://huggingface.co/models?pipeline_tag=text-generation) Transformers (https://huggingface.co/models?library=transformers) Safetensors (https://huggingface.co/models?library=safetensors) English (https://huggingface.co/models?language=en) llama (https://huggingface.co/models?other=llama) quantized (https://huggingface.co/models?other=quantized) int4 (https://huggingface.co/models?other=int4) w4a16 (https://huggingface.co/models?other=w4a16) weight-only (https://huggingface.co/models?other=weight-only) 4-bit precision (https://huggingface.co/models?other=4-bit) llm-compressor (https://huggingface.co/models?other=llm-compressor) zendnn (https://huggingface.co/models?other=zendnn) compressed-tensors (https://huggingface.co/models?other=compressed-tensors) amd (https://huggingface.co/models?other=amd) cpu-inference (https://huggingface.co/models?other=cpu-inference) conversational (https://huggingface.co/models?other=conversational) text-generation-inference (https://huggingface.co/models?other=text-generation-inference) License: llama3.1 Model card (https://huggingface.co/amd/Llama-3.1-8B-Instruct-w4a16-llmcompressor-v0.12.0) Files Files and versions xet (https://huggingface.co/amd/Llama-3.1-8B-Instruct-w4a16-llmcompressor-v0.12.0/tree/main) Community (https://huggingface.co/amd/Llama-3.1-8B-Instruct-w4a16-llmcompressor-v0.12.0/discussions) Deploy Copy to bucket new Use this model Instructions to use amd/Llama-3.1-8B-Instruct-w4a16-llmcompressor-v0.12.0 with libraries, inference providers, notebooks, and local apps. Follow these links to get started. Libraries Transformers (https://huggingface.co/amd/Llama-3.1-8B-Instruct-w4a16-llmcompressor-v0.12.0?library=transformers) How to use amd/Llama-3.1-8B-Instruct-w4a16-llmcompressor-v0.12.0 with Transformers: # Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="amd/Llama-3.1-8B-Instruct-w4a16-llmcompressor-v0.12.0") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages) # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("amd/Llama-3.1-8B-Instruct-w4a16-llmcompressor-v0.12.0") model = AutoModelForCausalLM.from_pretrained("amd/Llama-3.1-8B-Instruct-w4a16-llmcompressor-v0.12.0", device_m