Self hosted AI platform options for Kubernetes teams: A comparison
Self hosted AI platform options are compared by layer, from knowledge apps to retrieval databases and integrated RAG, with operating duties guiding the choice.
A Kubernetes team choosing self-hosted AI software first needs to select the layer it intends to operate. Compare a knowledge application, retrieval database, integrated RAG platform, or ML workflow platform through the remaining upgrade, restore, and access work. Hosting location and the service agreement then define the actual responsibility boundary.
Key takeaways
Short answer: evaluate the platform layer your team intends to operate.
- Evaluate an integrated RAG platform when you want document storage, parsing, retrieval and agents together. Retrieval-augmented generation (RAG) grounds generated answers in retrieved information; Seahorse Cloud combines storage, parsing, vector retrieval and managed agents, with on-premise installation or SaaS delivery on a Kubernetes-native architecture.
- Evaluate a knowledge application when employees need ready-made search and chat with connectors, such as Onyx.
- Evaluate a vector database such as Qdrant or Weaviate when your team will build the application, ingestion and access-control behavior around retrieval.
- Add distributed vector deployment or AI/ML workflow tooling when measured service needs call for those additional layers.
Compare the platform layers first
Start by naming the outcome your users need. A knowledge application supplies a working user-facing product. A retrieval database supplies a service your engineers connect to an application. An integrated RAG platform combines more of the document-to-agent path. An AI/ML workflow platform is designed for broader model and pipeline operations.
| Candidate or layer | What it supplies | Documented deployment path | Operating work the team should plan for |
|---|---|---|---|
| Onyx, knowledge application | Internal search, connectors, chat, agents, actions and deep research | Guided quickstart uses Docker Compose; Helm configurations are also available | Evaluate the chart and version, connected-source indexing needs, and applicable support route |
| Seahorse Cloud, integrated RAG | Object storage, vector database, autonomous parsing and managed agents | Kubernetes-native architecture; on-premise installation or SaaS subscription | Set the operator and lifecycle responsibilities for the selected delivery arrangement |
| Qdrant, retrieval database | Vector search for a RAG application your team builds | Community Helm chart or Enterprise Private Cloud Operator | Helm operations and support differ from the Enterprise Operator offer; verify the selected service boundary |
| Weaviate, retrieval database | Vector search with model integrations | Kubernetes; self-deployment or marketplace deployment | The team operates the selected deployment, integrations, data and releases |
| Distributed vector deployment | Multi-node retrieval, such as distributed Qdrant | Qdrant Helm chart supports distributed deployment | The team operates cluster configuration, load balancing, monitoring, backup and recovery |
| Kubeflow, AI/ML workflow platform | Components for the AI lifecycle, including pipelines and distributed training | Builds on Kubernetes | The team operates the selected model-workflow components and their dependencies |
These rows describe different platform layers, so they are not mutually exclusive. A team can use document processing with a retrieval database, operate an employee-facing knowledge application over its sources, or run model workflows alongside RAG. The sensible comparison asks which responsibilities each product removes from the team and which ones remain in its service boundary.
Kubernetes' StatefulSet documentation explains why stateful services need more than a pod specification: StatefulSet pods have stable identities and storage, and persistent-volume provisioning still depends on a storage class or an administrator. For an AI data service, plan storage ownership and recovery along with manifests and deployment automation.
When an integrated RAG platform fits
Evaluate Seahorse Cloud when the work spans document storage, document understanding, vector retrieval and managed agents, and your team wants those pieces within one RAG platform. The product includes object storage management, a vector database, autonomous parsing and managed agent systems. Seahorse Cloud's AI-powered storage combines S3-compatible object storage, semantic chunking, automatic vector database synchronization and autonomous document parsing.
That combination is useful when the team would otherwise connect and maintain separate storage, parsing, indexing and agent components. A shared document path can reduce repeated integration work between those layers. The team still needs to validate how source updates, access changes, parsing exceptions and deletes move through its chosen workflow; integrated components do not make those application rules disappear.
Seahorse Cloud uses a Kubernetes-native architecture and is available as an on-premise installation or a SaaS subscription. Hosting location and operator are separate decisions. An on-premise installation places the software in the organization's environment, while a SaaS subscription is a different delivery arrangement. For either choice, set the operating agreement for installation, upgrades, monitoring, backups, support and incident ownership before production use.
When a complete knowledge application fits
Evaluate Onyx when users need a working internal knowledge product. Onyx's self-hosted open-source core includes internal search, connectors, chat, agents, actions and deep research.
The infrastructure team should also be prepared to operate the application stack behind it. Onyx Standard adds a vector-and-keyword index, connector-sync background containers, inference servers for indexing and inference, object-store file storage, and Redis for performance optimizations. That is a larger service boundary than a database endpoint, so include those supporting components in architecture and recovery plans.
Onyx's guided quickstart uses Docker Compose, while its Kubernetes deployment guide documents Helm charts; the deployment overview also describes Helm Lite. Verify the chart and Onyx version, configuration, and support route against your Kubernetes requirements. Onyx Lite does not index connected sources, so evaluate Standard when connected-source indexing is required.
Onyx's GitHub connector requirements describe three access modes: Public shows every indexed document to all Onyx users, Private restricts documents to selected Onyx groups, and Auto Sync Permissions is documented to mirror each searcher's GitHub access.
Permission sync is a paid feature for the self-hosted Enterprise Edition and depends on the required edition, token and organization setup, user identity matching, and successful sync runs. Verify these conditions and test search results using accounts with different source permissions rather than assuming access stays in sync.
A packaged knowledge application fits when the desired deliverable is employee search and chat with supported connectors. A retrieval database fits when engineers want to own the user experience and application behavior. These routes can coexist: teams can build a separate application while retaining an internal knowledge service for another audience.
When Qdrant fits as a retrieval layer
Evaluate Qdrant when your developers plan to build the RAG application and want to operate the vector-retrieval service as one part of it. That leaves source ingestion, identity mapping, application permissions, user experience and model selection with the platform or application team. It suits an organization that already has those responsibilities assigned and wants a focused database component.
The Qdrant installation guide lists both a community-supported Helm chart and the Enterprise Private Cloud Operator. It recommends the Operator for Kubernetes deployments in an organization's own infrastructure without a cloud connection. Check the selected offer, chart or Operator version, and support terms before assigning lifecycle responsibilities.
With the community Helm chart, the team owns database operations including distributed scaling, monitoring, logging, upgrades, backup and disaster recovery. Do not generalize the chart's community-support scope to Qdrant's separate Enterprise Private Cloud Operator offer; verify the Operator's service and support terms separately.
Build a retrieval test from the team's documents, user questions, and query traffic. Record corpus size, vector dimension, filters, index settings, hardware, and concurrency with quality and latency results. Keep public ANN measurements as scoped references; a higher QPS at a different recall target is not a like-for-like product ranking. Use the same target recall and workload when comparing the intended retrieval layer.
When Weaviate fits as a self-managed retrieval layer
Evaluate Weaviate when the team wants to own a retrieval application and values a self-deployed Kubernetes service with configurable integrations. The Weaviate deployment overview presents its Kubernetes option for development through production, with customizable configurations, local inference containers and multimodal models. It offers self-deployment or marketplace deployment. That makes Weaviate relevant when engineers want retrieval and selected model integrations close to their own cluster and application stack.
Weaviate's Kubernetes installation guide uses Helm charts and a values.yaml file. Select the documented upgrade procedure for the actual chart and database versions, then rehearse it with persistent data and a tested recovery route. Version-specific StatefulSet replacement steps belong to their documented upgrade path, rather than a universal maintenance instruction.
Compare Qdrant and Weaviate using identical documents, embedding models, access filters, query traffic and hardware. Add exact identifiers and filtered searches when the application uses them, then compare retrieval quality, latency and operating results together.
When distributed vector search is worth operating
Treat distributed deployment as an architecture choice layered on a vector database, rather than as a separate AI application category. It becomes relevant when a measured workload needs the service split across multiple nodes for the team's availability, capacity or scaling targets.
Qdrant's production guide documents multi-node configuration and a load balancer for a highly available distributed setup. For the community Helm deployment, teams manage monitoring, logging, upgrades, scaling, backups and disaster recovery; confirm responsibilities against the selected Qdrant offer. Adopt this design only when workload tests and service targets justify the added work, with a named owner for each duty. A simpler deployment remains appropriate if it meets the tested target and recovery requirements.
Test expected data volume, ingestion bursts, concurrent searches, and node or storage failures. Measure latency and retrieval quality at normal load and during failure; verify restoration and return to the intended replica or shard state. Use these results to determine whether distribution addresses a measured need and whether the team can recover the service.
When an AI/ML workflow platform is a better fit than a RAG database
Evaluate Kubeflow when platform teams need reusable machine-learning workflows on Kubernetes. The official architecture guide describes Pipelines for AI lifecycle steps, Trainer for distributed training or LLM fine-tuning, and Notebooks for model development and interactive data science. These components support model lifecycle activity; a vector database serves application search by storing and retrieving vectors.
Kubeflow's architecture also includes Katib for model optimization and hyperparameter tuning, and the Spark Operator for data preparation and feature engineering. Identify whether experimentation, tuning, data preparation or distributed training belongs in the team's platform, then select components for those workflows.
Use Kubeflow alongside a retrieval service when the organization needs both model lifecycle workflows and vector search for applications. Assign a named owner, storage plan and upgrade procedure to each selected service before the pilot.
Use layer-appropriate acceptance tests
Use a controlled pilot aligned with the platform layer and the team's service requirements, applying only relevant checks.
For equivalent retrieval candidates, hold the document set, user roles, models, query mix and hardware constant, and compare them against the same retrieval-quality and latency targets. Assess Kubeflow on the machine-learning workflow outcomes the team needs, not as a substitute search application. Record user outcomes, permissions where applicable, and operating effort separately; a candidate passes when it meets its relevant requirements and the team can complete its procedures with named owners.
- Load representative data. Ingest the files, formats, metadata and document sizes expected in production. Include scanned pages, tables or complex layouts when those occur in the source collection. Inspect parsed output and verify that the application retrieves the intended content.
- Exercise updates and removal. After editing or deleting selected documents, confirm that updated content appears and removed content disappears from search. Record the time from source change to the expected result. Test the complete path rather than assuming that storage-to-index synchronization includes your permission and deletion rules.
- Test source permissions. Create users with distinct access in the source system and test each connector or retrieval path. Change one user's access and repeat the query. Record any edition, connector, identity or synchronization condition that controls the result.
- Interrupt a data job. Stop an ingestion or indexing task during a representative run, then resume it using the documented operating procedure. Check for missing records, duplicates and errors that require manual repair. Use this step to test retry and observability as well as ingestion.
- Restore persistent data. Restore a backup into an isolated environment and verify that the service starts with usable indexes and expected application state. Kubernetes volume snapshot guidance explains that snapshot creation and restoration depend on the required storage components, including a CSI driver. A snapshot object alone is not a tested recovery plan; test restoration with the cluster's actual storage provider and platform data.
- Rehearse upgrades and recovery. Upgrade a representative installation using the selected chart or deployment procedure. Test rollback only when the previous software version is compatible with the current data and schema; otherwise follow a tested restore plan, including version-specific migration steps. Rolling updates using Kubernetes' default OrderedReady policy can reach a state requiring manual intervention, so include operator repair steps in the runbook, as the StatefulSet guidance explains.
- Assign the service boundary. Name who owns the cluster, database, object storage, connectors, model inference, application permissions, upgrades, alert response and restore. For a SaaS subscription or other managed delivery, record which responsibilities remain with your team under the specific agreement. For an on-premise installation, document the operator and support path for the installation.
Keep the test record with the decision: input dataset, versions, configuration, user roles, measured results, restore evidence and unresolved tasks. This makes the choice reviewable when document volume, model use or team ownership changes.
FAQ
What is the best self-hosted AI?
Evaluate a knowledge application for ready-made search and chat, a retrieval database for a custom RAG application, an integrated RAG platform for document processing through agents, or workflow tooling for model pipelines and serving.
Can you self host your own AI?
Yes. A team can run AI software in its own environment, including an inference service for a model or an AI data platform for document processing and retrieval. Those are separate layers, so hosting a model does not by itself provide connectors, access-aware knowledge search, backups or an employee-facing application.
Does Kubernetes experience make a platform easy to operate?
No. Kubernetes experience helps with container deployment, but the team also needs procedures for data persistence, permissions, upgrades, monitoring and recovery. A restore test and an upgrade rehearsal show whether the wider service fits the team's operating capacity.
What is the best self-hosted AI platform?
Onyx fits a team seeking a complete internal knowledge product and its connector features; Seahorse Cloud fits a team seeking document storage, parsing, vector retrieval and managed agents in an integrated RAG platform. Qdrant or Weaviate fits a team that will build the application around its retrieval service. Distributed deployment fits a measured workload that justifies its added operating duties, and AI/ML workflow tooling fits a broader model-lifecycle need.
Evaluate the narrowest platform layer that meets the user requirement and passes the relevant acceptance tests. Consider a broader layer when it replaces integrations or operations the team would otherwise build and own. Confirm that the hosting arrangement, operating model, support and contract assign lifecycle work clearly.
Related background: Separate Hosting Location from Operating Responsibility in Enterprise AI
Try Seahorse Cloud
Evaluate this workflow with your own documents and acceptance criteria.
