Home › Local-AI Stack Directory: 40 Self-Hosted LLM & Vector-DB Tools, Verified Specs › ExLlamaV2

ExLlamaV2

One row from Local-AI Stack Directory: 40 Self-Hosted LLM & Vector-DB Tools, Verified Specs. A verified table of 40 self-hosted/offline LLM & vector-DB tools with license, min RAM/GPU, offline capability, and a source URL for every claim — so you stop reading marketing pages to find what actually runs offline.

category

LLM runtime

license

MIT

min_ram_gpu

GPU required, 8GB+ VRAM

offline_capable

yes

maturity

active

source_url

https://github.com/turboderp/exllamav2

notes

Fast GPTQ/EXL2 quant inference

checked_on

2026-09-12