Anonymous credentials draft endpoints

[LDiazN]

This PR adds support for the anonymous credentials protocol to the Ooni API:

1. Adds ooniauth-py as a dependency
2. Adds new endpoints: /sign_credential, /manifest, /submit

Note that the new measurement /submit endpoint won't replace the old one, and for now it's meant to be used mostly during development

Also updates the fastpath so that it's able to work with the new fields

Migrations

This PR will require some database migrations

Postgres

Adds the following tables to postgres:

• ooniprobe_manifest: Describes the manifest that is reported to users when they register
• ooniprobe_server_state: Defines the key pair (secret_key, public_parameters) that is used for authentication

You can find the new models here:

backend/ooniapi/services/ooniprobe/src/ooniprobe/models.py

Line 58 in 3b3bfb2

class OONIProbeServerState(Base):

And the migrations:
https://github.com/ooni/backend/blob/userauth-dep/ooniapi/common/src/common/alembic/versions/7e28b5d17a7f_add_server_state_table_for_anonymous_.py

Clickhouse

In clickhouse we need to add the fields necessary for:

• Checking if a measurement is verified
• Running the verification offline in the fastpath machine at some point in the future when we try to implement offline verification

As an example of the changes, you can look at the clickhouse_init.sql script in fastpath:

backend/fastpath/clickhouse_init.sql

Lines 42 to 46 in 3b3bfb2

	`is_verified` Int8,
	`nym` Nullable(String),
	`zkp_request` Nullable(String),
	`age_range` Nullable(String),
	`msm_range` Nullable(String),

These are the alter table statements required to run the migration in production:

ALTER TABLE fastpath
ADD COLUMN is_verified Int8,
ADD COLUMN nym Nullable(String),
ADD COLUMN zkp_request Nullable(String),
ADD COLUMN age_range Nullable(String),
ADD COLUMN msm_range Nullable(String);

Feedback

This is still early work to define the API for the anonymous credentials protocol. Some things that could benefit from a bit of feedback are:

• Naming of the endpoints
  • /sign_credential used to be named /register but it clashes with the older /register one, so I chose a different name
• Parameters and usage
• Lifecycle of manifest and server state: Right now we always create a new manifest and server state if none is available to keep the API working, but we might want to be more deliberate with how we work with this

closes #1014 #1015

[codecov]

Codecov Report

❌ Patch coverage is 81.95266% with 61 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.19%. Comparing base (d943f03) to head (4328d27).
⚠️ Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
...niprobe/src/ooniprobe/routers/v1/probe_services.py	70.76%	32 Missing and 6 partials ⚠️
ooniapi/services/ooniprobe/src/ooniprobe/utils.py	57.14%	9 Missing ⚠️
ooniapi/services/ooniprobe/src/ooniprobe/models.py	91.07%	2 Missing and 3 partials ⚠️
ooniapi/services/ooniprobe/src/ooniprobe/main.py	73.33%	2 Missing and 2 partials ⚠️
ooniapi/services/ooniprobe/tests/test_anoncred.py	95.83%	3 Missing ⚠️
...ervices/ooniprobe/src/ooniprobe/routers/reports.py	83.33%	1 Missing ⚠️
ooniapi/services/ooniprobe/tests/conftest.py	88.88%	1 Missing ⚠️

❌ Your patch check has failed because the patch coverage (81.95%) is below the target coverage (95.00%). You can increase the patch coverage or adjust the target coverage.
❌ Your project check has failed because the head coverage (90.19%) is below the target coverage (95.00%). You can increase the head coverage or adjust the target coverage.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1016      +/-   ##
==========================================
- Coverage   92.81%   90.19%   -2.63%     
==========================================
  Files          17       19       +2     
  Lines        1281     1560     +279     
  Branches       65       78      +13     
==========================================
+ Hits         1189     1407     +218     
- Misses         78      127      +49     
- Partials       14       26      +12     

Flag	Coverage Δ
ooniprobe	90.19% <81.95%> (-2.63%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.