License Apache-2.0: 19 tools compared (category, min RAM GPU, offline capable)

19 of 40 rows from Local-AI Stack Directory: 40 Self-Hosted LLM & Vector-DB Tools, Verified Specs — every row carries the source it was checked against. Click a name for its notes.

tool name	category	min RAM GPU	offline capable	maturity	source URL
vLLM	LLM runtime	16GB+ GPU VRAM recommended	yes	mature, very active	source
text-generation-inference	LLM runtime	GPU required, 16GB+ VRAM	yes	mature, active	source
FastChat	LLM runtime/framework	GPU recommended, 16GB VRAM	yes	mature, moderate activity	source
MLC-LLM	LLM runtime	4GB+ RAM, mobile/GPU support	yes	mature, active	source
llamafile	LLM runtime	4GB+ RAM, CPU-only ok	yes	active	source
h2oGPT	LLM runtime/RAG	16GB RAM, GPU optional	yes	active	source
PrivateGPT	RAG framework	8GB RAM, GPU optional	yes	active	source
Haystack	RAG framework	depends on backend model	yes (with local models)	mature, active	source
Xinference	LLM runtime/serving	depends on model size	yes	active	source
OpenLLM	LLM serving	GPU recommended	yes	active	source
SGLang	LLM runtime	GPU required, 16GB+ VRAM	yes	active	source
Text Embeddings Inference	Embedding server	GPU optional, 4GB+ RAM	yes	active	source
Chroma	Vector DB	2GB+ RAM, no GPU needed	yes	mature, active	source
Qdrant	Vector DB	2GB+ RAM, no GPU needed	yes	mature, very active	source
Milvus	Vector DB	8GB+ RAM recommended	yes	mature, very active	source
Vespa	Vector DB/search engine	4GB+ RAM, scalable	yes	mature, active	source
Marqo	Vector search engine	4GB+ RAM, GPU optional	yes	active	source
Vald	Vector DB	scalable, k8s-based	yes	active	source
LanceDB	Vector DB	2GB+ RAM, no GPU needed	yes	active	source

The full verified table: Local-AI Stack Directory: 40 Self-Hosted LLM & Vector-DB Tools, Verified Specs — 40 rows, CSV + JSON, every row with a checked source. €14 See the product

More slices of this table