# Requirements and sizing

> The infrastructure you need before self-hosting DataxLand (Kubernetes, Postgres 16, S3-compatible object storage, a shared volume, ingress and TLS), the default resources of each component and how to pick object storage on each cloud.

## Infrastructure you need

| Item | Requirement |
| --- | --- |
| Kubernetes | 1.27 or later, amd64 or arm64 nodes (images are multi-arch) |
| Postgres | Version 16, a managed service (RDS, Cloud SQL, Azure Database for PostgreSQL) or your own Postgres cluster, with backups |
| Databases | Two databases with the same user: `dx` (metadata) and `dx_lake` (warehouse catalog). The application creates `dx_lake` itself if the user has `CREATEDB` |
| Object storage | S3-compatible: two buckets, one for the warehouse (default `dx-lake`) and one for backups (default `dx-backups`) |
| Shared volume | A StorageClass that can provision `ReadWriteMany` PVCs (EFS, Filestore, Azure Files, NFS, CephFS). Single node: `ReadWriteOnce` is enough |
| Ingress | An ingress controller in the cluster and a domain for `web`, for example `dataxland.corp.example` |
| TLS | A certificate in a Secret, or cert-manager issuing one automatically |
| Email | An SMTP server for invitations, password resets and reports |
| AI model | An API key from a model provider, or an in-house OpenAI-compatible model (vLLM, Ollama, LM Studio, TGI) |

Optional: External Secrets Operator (reads secrets from your secret store), Prometheus Operator (collects metrics), a CNI that
supports NetworkPolicy (Calico, Cilium, Azure NPM, GKE Dataplane V2, AWS VPC CNI with policy enabled).

Pods run as a non-root user (uid 10001) with a read-only root filesystem, all capabilities dropped and
`seccompProfile: RuntimeDefault`. These settings match the Pod Security "restricted" standard; we have not yet tested on a
cluster that enforces it.

## Default resources

From the chart's `values.yaml`. "Request" is what Kubernetes reserves; "limit" is the memory ceiling of each pod.

| Component | Replicas | CPU request | Memory request | Memory limit |
| --- | --- | --- | --- | --- |
| `api` | 2 | 0.5 | 1 Gi | 3 Gi |
| `web` | 2 | 0.1 | 256 Mi | 768 Mi |
| `mcp` | 1 | 0.25 | 512 Mi | 2 Gi |
| `worker` | 1 | 1 | 2 Gi | 7 Gi |
| **Total** | | **2.45** | **5 Gi** | **16.5 Gi** |

Tune them with `api.resources`, `web.resources`, `mcp.resources` and `worker.resources`. Most of the memory of `api` and `mcp`
goes to read-only warehouses: `api.readers` (default `max: 8`, `memory: 256MB`) and `mcp.readers` (`max: 4`, `memory: 256MB`);
the product of the two numbers is the read-warehouse memory ceiling per pod. `worker.threads` (default 4) is the number of
threads used for builds.

## Sizing guide

The numbers below are **starting estimates**, not measurements of your workload. Watch memory and build times for the first
few weeks, then adjust.

| | Small | Medium | Large |
| --- | --- | --- | --- |
| Workspaces | up to 10 | 10 to 50 | over 50 |
| Kubernetes nodes | 3 × 4 vCPU, 16 GB | 3 × 8 vCPU, 32 GB | 4 or more × 8 vCPU, 32 GB |
| `worker` memory limit | 7 Gi (default) | 12 Gi | 16 Gi or more |
| `api` | 2 replicas (default) | enable `api.autoscaling` | enable `api.autoscaling`, raise `maxReplicas` |
| Postgres | 2 vCPU, 4 GB | 4 vCPU, 16 GB | 8 vCPU, 32 GB |
| Shared volume | 100 Gi (default) | 250 Gi | 500 Gi or more |

Object storage capacity depends on how much source data you have; compressed Parquet is usually much smaller than the
original. The backup bucket needs room for the retained copies (by default 7 daily, 4 weekly, 6 monthly).

`worker` always runs as one replica, so as data grows you raise its memory and `worker.threads`, not its replica count.

## Object storage on each cloud

DataxLand speaks the S3 protocol. How to connect each option, set under `objectStorage` in your values:

| Storage | `objectStorage.endpoint` | `urlStyle` | Credentials | Notes |
| --- | --- | --- | --- | --- |
| AWS S3 | `s3.<region>.amazonaws.com` | `vhost` | Static access key of an IAM user | IRSA and Pod Identity are not supported |
| Google Cloud Storage | `storage.googleapis.com` | `path` | HMAC key of a service account | Through the S3-compatible API; paths stay `s3://`, not `gs://` |
| Azure Blob Storage | leave empty, enable `s3proxy.enabled` | `path` | An S3 key pair you set for s3proxy | s3proxy runs in the cluster in front of the Storage Account. **Not tested on a real account** |
| On-prem | MinIO, Ceph RGW, or a local provider's S3-compatible object storage | usually `path` | Per system | MinIO allows per-workspace keys |

- On S3, GCS and s3proxy, set `objectStorage.tenantKeys: "off"`: one key for the whole bucket, each workspace's data under its
  own prefix.
- With MinIO, `objectStorage.tenantKeys: minio` gives each workspace a key that can only read and write its own area. The
  MinIO admin key then lives only in `worker`; the chart removes it from `api` and `mcp` automatically.
- An alternative on Azure if you would rather not use s3proxy: run your own MinIO on Azure Disk.
- Enable versioning on both buckets, and write-once locking (Object Lock, retention, immutability) on the backup bucket.

## Network

Pods must be able to reach Postgres, object storage, the AI model server, SMTP, the source databases you plan to connect, and
the cluster DNS. In air-gapped mode that is also the complete allow-list; see [Air-gapped mode](/en/docs/self-host/airgapped).

## Related

- [Install with Helm](/en/docs/self-host/helm)
- [Reference Terraform](/en/docs/self-host/terraform)
