Skip to content

Custom SecurityContext in Postgresql, OperatorConfiguration CRDs #2244

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions charts/postgres-operator/crds/operatorconfigurations.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -178,6 +178,10 @@ spec:
type: array
items:
type: string
dropped_pod_capabilities:
type: array
items:
type: string
cluster_domain:
type: string
default: "cluster.local"
Expand Down Expand Up @@ -324,8 +328,17 @@ spec:
type: integer
spilo_runasgroup:
type: integer
spilo_runasnonroot:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you need this if there is a spilo_runasuser?

Copy link

@tomsozolins tomsozolins Aug 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@szuecs such field is required to conform with pod security standards restricted profile
https://kubernetes.io/docs/concepts/security/pod-security-standards/#restricted

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok then let's have it!
Please resolve all the comments related with this one.
Thanks for your work!

type: boolean
spilo_fsgroup:
type: integer
spilo_seccompprofile:
type: object
properties:
localhostProfile:
type: string
type:
type: string
spilo_privileged:
type: boolean
default: false
Expand Down
9 changes: 9 additions & 0 deletions charts/postgres-operator/crds/postgresqls.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -466,8 +466,17 @@ spec:
type: integer
spiloRunAsGroup:
type: integer
spiloRunAsNonRoot:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

type: boolean
spiloFSGroup:
type: integer
spiloSeccompProfile:
type: object
properties:
localhostProfile:
type: string
type:
type: string
standby:
type: object
properties:
Expand Down
9 changes: 9 additions & 0 deletions charts/postgres-operator/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,10 @@ configKubernetes:
# additional_pod_capabilities:
# - "SYS_NICE"

# list of dropped capabilities for postgres container
# dropped_pod_capabilities:
# - "ALL"

# default DNS domain of K8s cluster where operator is running
cluster_domain: cluster.local
# additional labels assigned to the cluster objects
Expand Down Expand Up @@ -196,10 +200,15 @@ configKubernetes:
# set user and group for the spilo container (required to run Spilo as non-root process)
# spilo_runasuser: 101
# spilo_runasgroup: 103
# spilo_runasnonroot: true
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


# group ID with write-access to volumes (required to run Spilo as non-root process)
# spilo_fsgroup: 103

# spilo_seccompprofile:
# type: Localhost
# localhostProfile: profiles/audit.json

# whether the Spilo container should run in privileged mode
spilo_privileged: false
# whether the Spilo container should run with additional permissions other than parent.
Expand Down
89 changes: 9 additions & 80 deletions docs/administrator.md
Original file line number Diff line number Diff line change
Expand Up @@ -993,81 +993,7 @@ with `USE_WALG_BACKUP: "true"`.

### Google Cloud Platform setup

When using GCP, there are two authentication methods to allow the postgres
cluster to access buckets to write WAL-E logs: Workload Identity (recommended)
or using a GCP Service Account Key (legacy).

#### Workload Identity setup
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this all dropped?
I have no idea about GCP, but the PR seems to be about CRD change to allow to customize SecurityContext.
Please make sure the focus of the PR stays in line with the changes.


To configure the operator on GCP using Workload Identity these prerequisites are
needed.

* [Workload Identity](https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity) enabled on the GKE cluster where the operator will be deployed
* A GCP service account with the proper IAM setup to access the GCS bucket for the WAL-E logs
* An IAM policy granting the Kubernetes service account the
`roles/iam.workloadIdentityUser` role on the GCP service account, e.g.:
```bash
gcloud iam service-accounts add-iam-policy-binding <GCP_SERVICE_ACCOUNT_NAME>@<GCP_PROJECT_ID>.iam.gserviceaccount.com \
--role roles/iam.workloadIdentityUser \
--member "serviceAccount:PROJECT_ID.svc.id.goog[<POSTGRES_OPERATOR_NS>/postgres-pod-custom]"
```

The configuration parameters that we will be using are:

* `wal_gs_bucket`

1. Create a custom Kubernetes service account to be used by Patroni running on
the postgres cluster pods, this service account should include an annotation
with the email address of the Google IAM service account used to communicate
with the GCS bucket, e.g.

```yml
apiVersion: v1
kind: ServiceAccount
metadata:
name: postgres-pod-custom
namespace: <POSTGRES_OPERATOR_NS>
annotations:
iam.gke.io/gcp-service-account: <GCP_SERVICE_ACCOUNT_NAME>@<GCP_PROJECT_ID>.iam.gserviceaccount.com
```

2. Specify the new custom service account in your [operator paramaters](./reference/operator_parameters.md)

If using manual deployment or kustomize, this is done by setting
`pod_service_account_name` in your configuration file specified in the
[postgres-operator deployment](../manifests/postgres-operator.yaml#L37)

If deploying the operator [using Helm](./quickstart.md#helm-chart), this can
be specified in the chart's values file, e.g.:

```yml
...
podServiceAccount:
name: postgres-pod-custom
```

3. Setup your operator configuration values. Ensure that the operator's configuration
is set up like the following:
```yml
...
aws_or_gcp:
# additional_secret_mount: ""
# additional_secret_mount_path: ""
# aws_region: eu-central-1
# kube_iam_role: ""
# log_s3_bucket: ""
# wal_s3_bucket: ""
wal_gs_bucket: "postgres-backups-bucket-28302F2" # name of bucket on where to save the WAL-E logs
# gcp_credentials: ""
...
```

Continue to shared steps below.

#### GCP Service Account Key setup

To configure the operator on GCP using a GCP service account key these
prerequisites are needed.
To configure the operator on GCP these prerequisites that are needed:

* A service account with the proper IAM setup to access the GCS bucket for the WAL-E logs
* The credentials file for the service account.
Expand Down Expand Up @@ -1111,10 +1037,7 @@ aws_or_gcp:
...
```

Once you have set up authentication using one of the two methods above, continue
with the remaining shared steps:

1. Setup pod environment configmap that instructs the operator to use WAL-G,
3. Setup pod environment configmap that instructs the operator to use WAL-G,
instead of WAL-E, for backup and restore.
```yml
apiVersion: v1
Expand All @@ -1129,7 +1052,7 @@ data:
CLONE_USE_WALG_RESTORE: "true"
```

2. Then provide this configmap in postgres-operator settings:
4. Then provide this configmap in postgres-operator settings:
```yml
...
# namespaced name of the ConfigMap with environment variables to populate on every pod
Expand Down Expand Up @@ -1308,6 +1231,12 @@ configuration:
volumeMounts:
- mountPath: /custom-pgdata-mountpoint
name: pgdata
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
runAsNonRoot: true
- ...
```

Expand Down
19 changes: 18 additions & 1 deletion docs/reference/cluster_manifest.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,13 +78,26 @@ These parameters are grouped directly under the `spec` key in the manifest.
This must be set to run the container without root. By default the container
runs with root. This option only works for Spilo versions >= 1.6-p3.

* **spiloRunAsNonRoot**
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

determines whether the pod’s container should run as a non-root user. If set
to true, the image is validating at runtime to make sure that it does not run
as UID 0 (root) and won’t start the container if it does.

* **spiloFSGroup**
the Persistent Volumes for the Spilo pods in the StatefulSet will be owned and
writable by the group ID specified. This will override the **spilo_fsgroup**
operator parameter. This is required to run Spilo as a non-root process, but
requires a custom Spilo image. Note the FSGroup of a Pod cannot be changed
without recreating a new Pod. Optional.

* **spiloSeccompProfile**
Seccomp (Secure Computing) is a feature in the Linux kernel that allows a
userspace program to create syscall filters. In the context of containers,
these syscall filters are collated into seccomp profiles that can be used to
restrict which syscalls and arguments are permitted. Applying seccomp profiles
to containers reduces the chance that a Linux kernel vulnerability will be
exploited.

* **enableMasterLoadBalancer**
boolean flag to override the operator defaults (set by the
`enable_master_load_balancer` parameter) to define whether to enable the load
Expand Down Expand Up @@ -118,7 +131,7 @@ These parameters are grouped directly under the `spec` key in the manifest.
a map of usernames to user flags for the users that should be created in the
cluster by the operator. User flags are a list, allowed elements are
`SUPERUSER`, `REPLICATION`, `INHERIT`, `LOGIN`, `NOLOGIN`, `CREATEROLE`,
`CREATEDB`, `BYPASSRLS`. A login user is created by default unless NOLOGIN is
`CREATEDB`, `BYPASSURL`. A login user is created by default unless NOLOGIN is
specified, in which case the operator creates a role. One can specify empty
flags by providing a JSON empty array '*[]*'. If the config option
`enable_cross_namespace_secret` is enabled you can specify the namespace in
Expand Down Expand Up @@ -483,6 +496,10 @@ defined in the sidecar dictionary:
[CPU and memory requests and limits](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container)
for each sidecar container. Optional.

* **securityContext**
[A security context defining privilege and access control settings](https://kubernetes.io/docs/tasks/configure-pod-container/security-context)
for each sidecar container. Optional.

### Requests

CPU and memory requests for the sidecar container.
Expand Down
17 changes: 17 additions & 0 deletions docs/reference/operator_parameters.md
Original file line number Diff line number Diff line change
Expand Up @@ -453,12 +453,25 @@ configuration they are grouped under the `kubernetes` key.
This must be set to run the container without root. By default the container
runs with root. This option only works for Spilo versions >= 1.6-p3.

* **spilo_runasnonroot**
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

determines whether the spilo container should run as a non-root user. If set
to true, the image is validating at runtime to make sure that it does not run
as UID 0 (root) and won’t start the container if it does.

* **spilo_fsgroup**
the Persistent Volumes for the Spilo pods in the StatefulSet will be owned and
writable by the group ID specified. This is required to run Spilo as a
non-root process, but requires a custom Spilo image. Note the FSGroup of a Pod
cannot be changed without recreating a new Pod.

* **spiloSeccompProfile**
Seccomp (Secure Computing) is a feature in the Linux kernel that allows a
userspace program to create syscall filters. In the context of containers,
these syscall filters are collated into seccomp profiles that can be used to
restrict which syscalls and arguments are permitted. Applying seccomp profiles
to containers reduces the chance that a Linux kernel vulnerability will be
exploited.

* **spilo_privileged**
whether the Spilo container should run in privileged mode. Privileged mode is
used for AWS volume resizing and not required if you don't need that
Expand All @@ -475,6 +488,10 @@ configuration they are grouped under the `kubernetes` key.
PodSecruityPolicy allows the capabilities listed here. Otherwise, the
container will not start. The default is empty.

* **dropped_pod_capabilities**
list of capabilities to be dropped from the postgres container's
SecurityContext (e.g. ALL, SYS_NICE etc.).

* **master_pod_move_timeout**
The period of time to wait for the success of migration of master pods from
an unschedulable node. The migration includes Patroni switchovers to
Expand Down
6 changes: 6 additions & 0 deletions docs/user.md
Original file line number Diff line number Diff line change
Expand Up @@ -987,6 +987,12 @@ spec:
env:
- name: "ENV_VAR_NAME"
value: "any-k8s-env-things"
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
runAsNonRoot: true
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

```

In addition to any environment variables you specify, the following environment
Expand Down
10 changes: 10 additions & 0 deletions manifests/complete-postgres-manifest.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,11 @@ spec:
enableShmVolume: true
# spiloRunAsUser: 101
# spiloRunAsGroup: 103
# spiloRunAsNonRoot: true
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

# spiloFSGroup: 103
# spiloSeccompProfile:
# type: Localhost
# localhostProfile: profiles/audit.json
# podAnnotations:
# annotation.key: value
# serviceAnnotations:
Expand Down Expand Up @@ -186,6 +190,12 @@ spec:
# env:
# - name: "USEFUL_VAR"
# value: "perhaps-true"
# securityContext:
# allowPrivilegeEscalation: false
# capabilities:
# drop:
# - ALL
# runAsNonRoot: true
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


# Custom TLS certificate. Disabled unless tls.secretName has a value.
tls:
Expand Down
5 changes: 5 additions & 0 deletions manifests/configmap.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ metadata:
data:
# additional_owner_roles: "cron_admin"
# additional_pod_capabilities: "SYS_NICE"
# dropped_pod_capabilities: "ALL"
# additional_secret_mount: "some-secret-name"
# additional_secret_mount_path: "/some/dir"
api_port: "8080"
Expand Down Expand Up @@ -150,7 +151,11 @@ data:
spilo_allow_privilege_escalation: "true"
# spilo_runasuser: 101
# spilo_runasgroup: 103
# spilo_runasnonroot: true
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

# spilo_fsgroup: 103
# spilo_seccompprofile:
# type: Localhost
# localhostProfile: profiles/audit.json
spilo_privileged: "false"
storage_resize_mode: "pvc"
super_username: postgres
Expand Down
13 changes: 13 additions & 0 deletions manifests/operatorconfiguration.crd.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -176,6 +176,10 @@ spec:
type: array
items:
type: string
dropped_pod_capabilities:
type: array
items:
type: string
cluster_domain:
type: string
default: "cluster.local"
Expand Down Expand Up @@ -322,8 +326,17 @@ spec:
type: integer
spilo_runasgroup:
type: integer
spilo_runasnonroot:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

type: boolean
spilo_fsgroup:
type: integer
spilo_seccompprofile:
type: object
properties:
localhostProfile:
type: string
type:
type: string
spilo_privileged:
type: boolean
default: false
Expand Down
6 changes: 6 additions & 0 deletions manifests/postgresql-operator-default-configuration.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,8 @@ configuration:
kubernetes:
# additional_pod_capabilities:
# - "SYS_NICE"
# dropped_pod_capabilities:
# - "ALL"
cluster_domain: cluster.local
cluster_labels:
application: spilo
Expand Down Expand Up @@ -100,7 +102,11 @@ configuration:
spilo_allow_privilege_escalation: true
# spilo_runasuser: 101
# spilo_runasgroup: 103
# spilo_runasnonroot: true
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

# spilo_fsgroup: 103
# spilo_seccompprofile:
# type: Localhost
# localhostProfile: profiles/audit.json
spilo_privileged: false
storage_resize_mode: pvc
# toleration:
Expand Down
Loading