Cognitora
AI Agent Compute Platform
FreeWorkflow automationDevOpsOpen Source42 impressions#1779 of its week
Cognitora is an open-source platform that orchestrates AI inference across heterogeneous engines using microVMs and KV-aware routing.
- for
- ML engineers and ops teams needing secure, low-latency inference on bare metal or cloud fleets.
- pricing
- open source
Key features
- KV-Aware Routing — BLAKE3 sequence digests route requests to nodes with cached prefixes, reducing redundant prefill work.
- Energy-Aware Scheduling — Power telemetry (Redfish, NVML) influences routing scores and enforces soft watt caps.
- Multi-Tier KV Cache — RAM (DashMap) and SSD (RocksDB) tiers with cross-node QUIC fetch accelerate token reuse.
- Engine-Agnostic Agent — Supervises vLLM, SGLang, llama.cpp, MLX, TRT-LLM and OpenAI-compatible servers via a single OpenAI gateway.
- Bare-Metal First Deploy — One curl install, six static binaries and systemd units run without Kubernetes.
- Cross-Cluster Federation — Routers forward requests to the lowest-latency peer cluster when local nodes lack the model.
- Native Inference Engine — First-party GGUF engine on Candle runs models directly behind the same OpenAI HTTP surface.
- Observability Dashboard — Zero-dependency web UI shows live request, token, latency, KV utilization and power metrics.
Use cases
- Run AI agents on Apple Silicon laptops with MLX while keeping KV-aware routing consistent.
- Serve heterogeneous GPU racks (NVIDIA, AMD) using a single curl install and systemd.
- Deploy reasoning models that benefit from split prefill/decode pools across specialized hardware.
- Cache large code-base contexts across RAM, SSD and peer nodes for fast code-generation.
- Cost-optimized serving with SLM→LLM cascade gating cheap queries.
Cognitora vs alternatives
| Cognitora |  n8n |  LangChain |  CrewAI |  Kubernetes | |
|---|---|---|---|---|---|
| Best for | Engine-agnostic inference orchestration | Workflow automation | LLM app framework | Multi-agent orchestration | Container orchestration |
| Pricing | Open source | Subscription | Free | Subscription | Free |
| DevHunt upvotes | 0 | 0 | 0 | 0 | 0 |
| Launched | Jan 2026 | — | — | — | — |
- Cognitora vs n8n: n8n focuses on general workflow automation, not low-latency inference orchestration across GPU fleets.
- Cognitora vs LangChain: LangChain is a library for building LLM apps, whereas Cognitora provides a production-grade inference control plane.
- Cognitora vs CrewAI: CrewAI orchestrates multi-agent systems at the application level; Cognitora orchestrates inference engines at the infrastructure level.
- Cognitora vs Kubernetes: Kubernetes is a generic container orchestrator; Cognitora adds AI-specific routing, KV-aware cache and energy-aware scheduling.
Cognitora FAQ
Does Cognitora replace existing inference engines?+
No, it coordinates multiple engines; you keep your preferred engine and Cognitora adds orchestration.
What platforms can I run Cognitora on?+
It works on bare-metal Linux with systemd, Kubernetes clusters, cloud VMs via Terraform, Docker, and local macOS with MLX.
How does routing latency compare to direct engine calls?+
Routing decisions take under 500 µs per vCPU and add less than 3 ms HTTP overhead (p99).
Is the control plane written in a specific language?+
The control plane is built in Rust and provides a single OpenAI-compatible gateway.
What security features are included?+
Cognitora uses mTLS everywhere, Cosign-verified releases, OIDC, API-key authentication and RBAC.
Can I use Cognitora with Kubernetes?+
Yes, an optional Helm chart and operator (CRDs) are provided for declarative deployments.
Summarized by DevHunt from cognitora.dev · Oct 4, 2026. Details may change; check the official site.
.png?width=64)
.png?width=64){kind=icon}
