ValkeyCluster CRD Definitions

This document provides detailed information about the ValkeyCluster Kubernetes Custom Resource Definition (CRD). The ValkeyCluster CRD allows you to specify Valkey cluster(s) configuration which will be used by Aonnis Valkey Panther within a Kubernetes environment. This documentation is intended to help new users understand how to configure and use the ValkeyCluster resource effectively.

Introduction

The ValkeyCluster CRD enables users to deploy and manage Valkey clusters on Kubernetes. It abstracts the complexity of setting up a distributed Valkey environment by providing a declarative interface. Users can specify the desired state of their Valkey cluster, and the operator will ensure that the cluster converges to that state.

ValkeyCluster Spec

The ValkeyClusterSpec defines the desired state of a Valkey cluster. Below are detailed explanations of each field available in the spec section of the ValkeyCluster resource.

numberOfPrimaries

Type: int32
Description: Specifies the number of primary nodes in the Valkey cluster.
Required: true
Example:
```
numberOfPrimaries: 3
```

replicationFactor

Type: int32
Description: Specifies the number of replica nodes per primary node in the Valkey cluster.
Default: 0.
Example:
```
replicationFactor: 2
```

serviceName

Type: string
Description: The name of the Kubernetes Service that fronts the ValkeyCluster nodes. If not provided, the ValkeyCluster name will be used.
Default: Uses the metadata.name of the ValkeyCluster resource with prfix kp-vc-. If metadata.name = valkeyCluster then servicename = kp-vc-valkeyCluster.
Example:
```
serviceName: valkey-cluster-service
```

podTemplate

Type: PodTemplateSpec (Same as kubernetees pod template)
Description: Defines the pod specification that will run the Valkey server process. You can customize the containers, volumes, and other pod-level configurations here. Valkey container name must be valkey-node. If changed then cluster will may run into issues.
Default: Uses a default pod template if not specified.

Example:

podTemplate:
	metadata:
	labels:
		app: valkey
	spec:
	containers:
		- name: valkey-server
		image: valkey/server:latest

rollingUpdate

Type: RollingUpdate (Dictionary or map)
Description: Configuration for key migration during rolling updates.
Fields:
- keyMigration:
  - Type: bool
  - Description: Whether to migrate keys during a rolling update. If false, all current data will be lost during migration.
- warmingDelayMillis:
  - Type: int32
  - Description: Time in milliseconds between each slot batch iteration.
- keyBatchSize:
  - Type: int32
  - Description: int32 Number of keys to get from a single slot during each migration iteration.
- slotBatchSize:
  - Type: int32
  - Description: int32 Number of slots to migrate on each iteration.
- idleTimeoutMillis:
  - Type: int32
  - Description: int32 Maximum idle time during key migration.

Example:

rollingUpdate:
	keyMigration: true
	warmingDelayMillis: 5000
	keyBatchSize: 100
	slotBatchSize: 10
	idleTimeoutMillis: 1000

scaling

Type: Migration (Dictionary or map)
Description: Configuration for key migration during scaling operations.
Fields:
- keyBatchSize:
  - Type: int32
  - Description: Number of keys to get from a single slot during each migration iteration.
- slotBatchSize:
  - Type: int32
  - Description: Number of slots to migrate on each iteration.
- idleTimeoutMillis:
  - Type: int32
  - Description: Maximum idle time during key migration.

Example:

scaling:
	keyBatchSize: 200
	slotBatchSize: 20
	idleTimeoutMillis: 2000

additionalLabels

Type: Dictionary or map
Description: Additional labels to add to the created ValkeyCluster resources (Deployment, ReplicaSet, Pod, etc.). Key and valie both will be user defined

Example:

additionalLabels:
	environment: production
	team: devops

automaticSnapshotConfig

Type: AutomaticSnapshotConfigType
Description: Configuration for automatic snapshots.
Fields:
- enabled:
  - Type: bool
  - Description: Whether automatic snapshots are enabled.
- storageConfig:
  - Type: StorageConfigType
  - Description: Storage configuration for snapshots.
  - Required: true
- snapshotRetentionDays:
  - Type: int32
  - Description: Number of days to retain snapshots (0 means retain forever).
  - Required: false
  - Default: 0
- cronExpression:
  - Type: string (cronExpression)
  - Description: Cron expression to schedule snapshot operations.
  - Required: true
- uploadRetryLimit:
  - Type: int32
  - Description: Number of retry attempts for snapshot uploads, in case of failure to connect to storageConfig.
  - Required: false
  - Default: 1

Example:

Automatic snapshot config
automaticSnapshotConfig:
	enabled: true
	snapshotRetentionDays: 7
	cronExpression: "0 0 * * *"
	uploadRetryLimit: 3
	storageConfig:
		protocol: S3
		accessConfig:
			endpoint: "https://s3.amazonaws.com"
			region: "us-east-1"
			bucket: "valkey-snapshots"
			folderPath: "daily-backups"
		accessKeysSecretRef:
			name: "s3-credentials"
			namespace: "default"
			accessKey: "access-key"
			secretKey: "secret-key"

manualSnapshotConfig

Type: ManualSnapshotConfigType
Description: Configuration for manual snapshots.
Fields:
- name: Name of the snapshot (must be unique).
- startTime:
  - Type: int64
  - Description: Unix epoch time to schedule the snapshot (immediate if in the past).
  - Required: true
- storageConfig:
  - Type: StorageConfigType
  - Description: Storage configuration for snapshots.
  - Required: true
- uploadRetryLimit:
  - Type: int32
  - Description: Number of retry attempts for snapshot uploads, in case of failure to connect to storageConfig.
  - Required: false
  - Default: 1

Example:

manualSnapshotConfig:
	name: "manual-backup-001"
	startTime: 1633072800
	uploadRetryLimit: 3
	storageConfig:
		protocol: S3
		accessConfig:
			endpoint: "https://s3.amazonaws.com"
			region: "us-east-1"
			bucket: "valkey-snapshots"
			folderPath: "manual-backups"
		accessKeysSecretRef:
			name: "s3-credentials"
			namespace: "default"
			accessKey: "access-key"
			secretKey: "secret-key"

initialRestoreConfig

Type: Dictionary or Map
Description: Configuration for automatic snapshots.
Fields:
- enabled:
  - Type: bool
  - Description: Whether initial restore config is enabled.
  - Default: false
- storageConfig:
  - Type: StorageConfigType
  - Description: Storage configuration for snapshots.
  - Required: true

Example:

initialRestoreConfig:
	enabled: true
	snapshotStoreConfig:
		protocol: S3
		accessConfig:
		endpoint: "https://s3.amazonaws.com"
		region: "us-east-1"
		bucket: "valkey-snapshots"
		folderPath: "initial-backups"
		accessKeysSecretRef:
			name: "s3-credentials"
			namespace: "default"
			accessKey: "asscessKey"
			secretKey: "secretKey"

tlsConfig

Type: Dictionary or Map
Description: TLS config for valkey cluster. TLS secret must to be prepopulated before creatinng a valkey clsuter.
Fields:
- enabled:
  - Type: bool
  - Description: Is TLS enabled on valkey cluster or not.
  - Required: false
  - Default: false
- insecure:
  - Type: bool
  - Description: Are TLS certificates signed by trusted CA or not
  - Required: false
  - Default: false
- tlsCertSecretRef:
  - Type: CertSecretRef
  - Description: Reference of secret where tls cert is stored
  - Required: true
- tlsKeySecretRef:
  - Type: CertSecretRef
  - Description: Reference of secret where tls key is stored
  - Required: true
- tlsCaSecretRef:
  - Type: CertSecretRef
  - Description: Reference of secret where tls ca cert is stored
  - Required: true

Example:

tlsConfig:
	enabled: true
	insecure: true
	tlsCertSecretRef:
		name: valkey-tls-secret
		key: tls.crt
	tlsKeySecretRef:
		name: valkey-tls-secret
		key: tls.key
	tlsCaSecretRef:
		name: valkey-tls-secret
		key: ca.crt

valkeyNodeResources

Type: Dictionary or Map
Description: CPU and memory request definition. This is same as k8s pod resources definition

Example:

valkeyNodeResources:
	limits:
		cpu: 200m
		memory: 512Mi
	requests:
		cpu: 200m
		memory: 512Mi

configMap

Type: Dictionary or Map
Description: Valkey config properties. There properties will be applied on all the valkey nodes.

Example:

lazyfree-lazy-expire: "yes"
maxmemory-policy: "allkeys-lru"

licenseAccessKeySecretRef

Type: Dictionary or Map
Description: Reference of secret where Aonnis license access keys are stored. These should be already populated before creating a valkey cluster.
Fields:
- name:
  - Type: string
  - Description: Name of secrete where license access keys are stored
  - Required: true
- namespace:
  - Type: string
  - Description: Namespace of secrete where license access keys are stored
  - Required: true
- accessKey:
  - Type: string
  - Description: Key in the secret where access key is stored
  - Required: true
- secretKey:
  - Type: string
  - Description: Key in the secret where access key is stored
  - Required: true

Example:

accessKeysSecretRef:
	name: "s3-credentials"
	namespace: "default"
	accessKey: "asscessKey"
	secretKey: "secretKey"

readinessProbeConfig

Type: ProbeConfigForValkeyContainer
Description: Readiness probe config for Valkey containers.
Required: false

Default Example:

readinessProbeConfig:
	initialDelaySeconds: 12
	timeoutSeconds: 5
	periodSeconds: 10
	successThreshold: 1
	failureThreshold: 5

livenessProbeConfig

Type: ProbeConfigForValkeyContainer
Description: Readiness probe config for Valkey containers.
Required: false

Example:

livenessProbeConfig:
	initialDelaySeconds: 10
	timeoutSeconds: 10
	periodSeconds: 60
	successThreshold: 1
	failureThreshold: 3

Common types

StorageConfigType

Type: Dictionary or Map
Description: Storage config for snapshots, custom module, etc.
Fields:
- Protocol:
  - Type: string
  - Description: Protocol of storage. Supported protocols: S3
  - Required: true
- storageConfig:
  - Type: Dictionary or Map for S3Config
  - Description: Storage configuration for snapshots.
  - Required: true
  - Fields:
    - endpoint:
      - Type: string (URL)
      - Description: Endpoint of storage.
      - Required: true
    - region:
      - Type: string
      - Description: Valid region for the Storage protocol and provider.
      - Required: false
      - Default: us-east-1
    - bucket:
      - Type: string
      - Description: Valid bucket which already exists at the storage provider end.
      - Required: true
    - FolderPath
      - Type: string
      - Description: Path in the bucket where RDB snapshot files are to be restored.
      - Required: true

Example:

snapshotStoreConfig:
	protocol: S3
	accessConfig:
	endpoint: "https://s3.amazonaws.com"
	region: "us-east-1"
	bucket: "valkey-snapshots"
	folderPath: "initial-backups"
	accessKeysSecretRef:
		name: "s3-credentials"
		namespace: "default"
		accessKey: "asscessKey"
		secretKey: "secretKey"

AccessKeysSecretRefType

Type: Dictionary or Map
Description: Reference of access key
Fields:
- name:
  - Type: String
  - Description: Name of the secret where access keys are stored.
  - Required: true
- namespace:
  - Type: String
  - Description: Namespace of the secret where access keys are stored.
  - Required: false
  - Default: default
- accessKey:
  - Type: String
  - Description: Key of the entry with access key as the value.
  - Required: false
  - Default: accessKey
- secretKey:
  - Type: String
  - Description: Key of the entry with secret key as the value.
  - Required: false
  - Default: secretKey

CertSecretRef

Type: Dictionary or Map
Description: CertSecretRef is a reference structure used to retrieve secret values stored in a secure location. It is specifically designed for secrets created by cert-manager to store certificates. These secrets store data in a dictionary format, requiring a specified entry to be accessed and utilized.
Fields:
- namespace:
  - Type: string
  - Description: Name of namespace of secret.
  - Required: true
- name:
  - Type: string
  - Description: Secret name
  - Required: true
- key:
  - Type: string
  - Description: Indicates the key name of the specific secret entry to be retrieved.
  - Required: true

Example:

tlsCertSecretRef:
	namespace: "testNamespace"
	name: valkey-tls-secret
	key: tls.crt

ProbeConfigForValkeyContainer

Description: Dictionary or Map
Fields:
- initialDelaySeconds:
  - Type: int32
  - Description: Number of seconds after the container has started before liveness probes are initiated.
  - Required: true
- timeoutSeconds:
  - Type: string
  - Description: Number of seconds after which the probe times out.
  - Required: true
- periodSeconds:
  - Type: string
  - Description: How often (in seconds) to perform the probe.
  - Required: true
- successThreshold:
  - Type: string
  - Description: Minimum consecutive successes for the probe to be considered successful after having failed.
  - Required: true
- failureThreshold:
  - Type: string
  - Description: Minimum consecutive failures for the probe to be considered failed after having succeeded.
  - Required: true

Default Example:

readinessProbeConfig:
	initialDelaySeconds: 12
	timeoutSeconds: 5
	periodSeconds: 10
	successThreshold: 1
	failureThreshold: 5

Introduction​

ValkeyCluster Spec​

numberOfPrimaries​

replicationFactor​

serviceName​

podTemplate​

rollingUpdate​

scaling​

additionalLabels​

automaticSnapshotConfig​

manualSnapshotConfig​

initialRestoreConfig​

tlsConfig​

valkeyNodeResources​

configMap​

licenseAccessKeySecretRef​

readinessProbeConfig​

livenessProbeConfig​

Common types​

StorageConfigType​

AccessKeysSecretRefType​

CertSecretRef​

ProbeConfigForValkeyContainer​

Introduction

ValkeyCluster Spec

numberOfPrimaries

replicationFactor

serviceName

podTemplate

rollingUpdate

scaling

additionalLabels

automaticSnapshotConfig

manualSnapshotConfig

initialRestoreConfig

tlsConfig

valkeyNodeResources

configMap

licenseAccessKeySecretRef

readinessProbeConfig

livenessProbeConfig

Common types

StorageConfigType

AccessKeysSecretRefType

CertSecretRef

ProbeConfigForValkeyContainer