DM-42005: HTTP-based Worker Replication server

doc/dev/api/TODO.rst

@@ -0,0 +1,34 @@
+
+TODO
+----
+
+
+Finish in a scope of the current ticket DM-42005 before the X-Mas break:
+
+- [**x**] Think about the locking mechanism of the method WorkerHttpRequest::toJson(). The method
+  acquires a lock on the mutext while the request may too have a lock on the same mutex
+  while processing the request in WorkerHttpRequest::execute(). This may result in a deadlock.
+  Perhaps no locking is needed as all since the resulting data are not lock sencitive?
+- [**x**] Finish implementing a hierachy of the HTTP-based worker requests
+- [**x**] Finish implementing the request processor for these requests
+- [**x**] Add the new service to the Condfiguration and Registry to allow the Controller to send requests
+  to the worker via HTTP
+- [**x**] Display connection parameters of  the new service on the Web Dashboard
+- [ ] Document the REST services in the documentation tree.
+- [ ] Manually test the new implementation externally using ``curl`` or Python's ``requests`` module.
+  Think about the test cases to cover the new implementation.
+- [ ] Extend the integration tests to cover the new implementation. 
+
+Finish in a scope of a separate ticket during/after the X-Mas break:
+
+- [ ] Implement the MessengerHttp on the Controller side of the protocol. The class will
+  be providing the multiplexing API for the Controller to send requests to the worker.
+  The initial implementation will be based on the simple http::AsyncReq.
+- [ ] Create a parallel hierarchy of the HTTP-based request & job classes on the Controller
+  side of the protocol.
+- [ ] Test the new classes.
+- [ ] Implement the MessengerHttp to reuse the socket connections for sending multiple requests
+  to the same worker.
+- [ ] Test the new implementation to ensure it works the same way as the old one.
+- [ ] Remove the old implementation of the Controller - Worker protocol.
+

doc/dev/api/index.rst

@@ -0,0 +1,17 @@
+.. note::
+
+   Information in this guide corresponds to the version **40** of the Qserv REST API. Keep in mind
+   that each implementation of the API has a specific version. The version number will change
+   if any changes to the implementation or the API that might affect users will be made.
+   The current document will be kept updated to reflect the latest version of the API.
+
+##############################
+The internal REST API of Qserv
+##############################
+
+.. toctree::
+   :maxdepth: 4
+
+   introduction
+   repl-worker
+   TODO

doc/dev/api/introduction.rst

@@ -0,0 +1,22 @@
+.. _qserv-api-introduction:
+
+Introduction
+============
+
+The Qserv REST API is a collection of RESTful web services that provide access to various components of the Qserv system.
+The API enforces a specific interaction model between the client and the server. The following highlights are worth mentioning:
+
+- All ``POST``, ``PUT`` and ``DELETE`` requests must be accompanied by a JSON payload.
+- Responses of all but a few select services are in JSON format. Exceptions are documented in the API documentation.
+- Schemas of the JSON requests and payloads are defined in the API documentation.
+- The API is versioned. The version number is included in the URL path of the ``GET`` requests, and it's
+  included into the JSON payload of the ``POST``, ``PUT`` and ``DELETE`` requests.
+- Critical API services are protected by an authentication mechanism. The client must provide a valid
+  authentication token in the JSON payload of the ``POST``, ``PUT`` and ``DELETE`` requests.
+  No authentication is required for the ``GET`` requests.
+
+The general information on the structure of the API can be found in the following document:
+
+- :ref:`ingest-general`
+
+The rest of the current document provides detailed information on the individual services that are available in the Qserv API.