Oobabooga

The “text-generation-webui” is a Gradio-based web UI designed for Large Language Models, supporting various model backends including Transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), and Llama models.It features three interface modes: default (two columns), notebook, and chat.

Users can switch between models quickly via a dropdown menu and extend functionality with numerous built-in and user-contributed extensions, such as Coqui TTS for voice output and Whisper STT for voice input.

The tool supports chat with custom characters, precise templates for instruction-following models, and dynamic LoRA loading for custom training data.Additionally, it integrates with the Transformers library, enabling models to run in different precisions and provides an OpenAI-compatible API server for chat and completions.

Installation involves running platform-specific scripts, and Docker is supported for containerized environments.