Open
Description
- Which image of the operator are you using?
registry.opensource.zalan.do/acid/postgres-operator:v1.10.1 - Where do you run it - cloud or metal? Kubernetes or OpenShift?
Bare Metal K8s - Are you running Postgres Operator in production?
yes - Type of issue?
Bug report
I've been trying to configure logical backups into DigitalOcean Spaces. Creating and uploading the backups work, but when I set the logical_backup_s3_retention_time config in Operator CRD, the Job that is created can fail with following logs:
+ aws_delete_outdated
+ [[ -z 7 days ]]
++ date -d '7 days ago' +%F
+ cutoff_date=2024-01-17
+ prefix=spilo/<REDACTED>/logical_backups/
+ args=("--no-paginate" "--output=text" "--prefix=$prefix" "--bucket=$LOGICAL_BACKUP_S3_BUCKET")
+ [[ ! -z <REDACTED> ]]
+ args+=("--endpoint-url=$LOGICAL_BACKUP_S3_ENDPOINT")
+ [[ ! -z <REDACTED> ]]
+ args+=("--region=$LOGICAL_BACKUP_S3_REGION")
+ aws s3api list-objects --no-paginate --output=text --prefix=spilo/<REDACTED>/logical_backups/ --buck
et=postgres-backups --endpoint-url=<REDACTED> --region=<REDACTED> '--query=Contents[?LastModified<='
\''2024-01-17'\''].[Key]'
/usr/local/lib/python3.6/dist-packages/OpenSSL/_util.py:6: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python c
ore team. Therefore, support for it is deprecated in cryptography. The next release of cryptography will remove support for Python 3.6.
from cryptography.hazmat.bindings.openssl.binding import Binding
argument of type 'NoneType' is not iterable
The Job is then in Error state, and a new one is automatically created (up to 6 times which seems to be the default of Job.backoffLimit), which creates several unnecessary backups.
I presume it's because aws can't find any files that satisfy the filter, and can't iterate through the result and return its [Key].
For now, I used a workaround and created a lifecycle policy on the bucket that should work, but just wanted to report a bug.