Popular repositories Loading

1. toncao toncao Public

   Config files for my GitHub profile.

2. GPTQModel GPTQModel Public

   Forked from ModelCloud/GPTQModel

   Production ready LLM model compression/quantization toolkit with hw accelerated inference support for both cpu/gpu via HF, vLLM, and SGLang.

   Python

3. llm-compressor llm-compressor Public

   Forked from vllm-project/llm-compressor

   Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

   Python

4. AQLM AQLM Public

   Forked from Vahe1994/AQLM

   Official Pytorch repository for Extreme Compression of Large Language Models via Additive Quantization https://arxiv.org/pdf/2401.06118.pdf and PV-Tuning: Beyond Straight-Through Estimation for Ext…

   Python

5. compressed-tensors compressed-tensors Public

   Forked from vllm-project/compressed-tensors

   A safetensors extension to efficiently store sparse quantized tensors on disk

   Python

6. TensorRT-Model-Optimizer TensorRT-Model-Optimizer Public

   Forked from NVIDIA/TensorRT-Model-Optimizer

   A unified library of state-of-the-art model optimization techniques like quantization, pruning, distillation, speculative decoding, etc. It compresses deep learning models for downstream deployment…

   Python