Storage capability

[cefothe]

To save and protect the user files (can be campaign documents or something else) we need to build/integrate a storage mechanism that needs to meet the following requirements.

1. Distributed
2. Auditable
3. Provide build-in encryption
4. High available (eventual consistency)

We are searching for something like AWS S3 but on premise.

Possible solutions:
https://geekflare.com/self-hosted-s3/

[DeyanAtanasov]

How do the current GDPR requirements fit into that storage system?

[cefothe]

Those files will contain PHI, so we need to restrict who can access those files.

[cefothe]

I don't think we need to take care, because that information will be used by doctors to verify the campaign.

[preslavgerchev]

How about Ceph? It offers pretty much everything on the list, has a S3 compliant API and is in active development as well.

Bonus: Working with Ceph becomes even easier if we run on k8s, we can then use Rook to manage it. 😄

[imilchev]

Wasn't there already a proposal for Ceph with S3. I know someone was already looking into backing up the Ceph cluster

[cefothe]

If there a PR or ticket, please share it here

[kachar]

Few S3 based storage candidates:

• Minio (https://min.io/)
• Zenko (https://github.com/scality/Zenko)
• Switt (https://www.openstack.org/software/releases/ocata/components/swift)
• Ceph (https://ceph.io/)

[imilchev]

I already saw there are multiple people familiar with Ceph. Personally, I think this is one of the best solutions for distributed storage. For our use-case it might look like an overkill as setting up Ceph (without Rook or some other automation tool) is quite a bit of work and we won't really push its boundaries. However, I do think that it's still worth setting up as on the long run this is a project that will only grow bigger.

I am not going to give any vague comparisons between these and will just post 2 links about the usage and scale of Ceph clusters at:

• CERN (https://www.youtube.com/watch?v=0i7ew3XXb7Q)
• China Mobile (https://www.youtube.com/watch?v=6Rb2edOWBX0)

The fact that Intel and RedHat are actively developing and improving Ceph is a bonus.

[igoychev]

Let's go with Ceph guys!