Set Up and Configure Plural AI

Configure Plural AI and mix models to optimize cost and performance

The current state of GenerativeAI is a sprawl of vendors offering products with different specialties and price points, and its common to have an optimal AI setup involve usage of models across multiple different vendors or multiple models within the same vendor. Plural provides a number of knobs that are designed to make that degree of customization seamless, and compatible w/in a GitOps workflow.

Set Up Plural AI

Plural AI can be configured through Settings → AI settings in the Plural Console or with the DeploymentSettings CRD. If you installed Plural with plural up, the resource is already defined at bootstrap/settings.yaml.

For a basic OpenAI configuration, create a credential secret and reference it from DeploymentSettings:

yaml

apiVersion: v1
kind: Secret
metadata:
  name: ai-config
  namespace: plrl-deploy-operator
stringData:
  openai: <access-token>
---
apiVersion: deployments.plural.sh/v1alpha1
kind: DeploymentSettings
metadata:
  name: global
  namespace: plrl-deploy-operator
spec:
  ai:
    enabled: true
    provider: OPENAI
    openAI:
      tokenSecretRef:
        name: ai-config
        key: openai

Provider credentials are always read from secrets in the plrl-deploy-operator namespace. Use additional keys in the same secret when configuring other providers:

yaml

apiVersion: v1
kind: Secret
metadata:
  name: ai-config
  namespace: plrl-deploy-operator
stringData:
  openai: <access-token>
  anthropic: <access-token>
  azure: <access-token>
  vertex: <service-account-json>

Warning:

Never commit provider credentials to Git in plain text. Encrypt the secret with your organization's preferred secret-management workflow before storing it in a GitOps repository.

The full DeploymentSettings schema is available in the Management API reference.