Harbor

Effortlessly run LLM backends, APIs, frontends, and services with one command.

Harbor is a containerized LLM toolkit that helps you to run LLMs and satellites via a concise CLI - Convenience utils (manage models, config, debug, URLs, tunnels, more) - Built-in optimizing proxy and benchmarking - Service CLIs without install via Docker - Shared caches - Config profiles, history