feat: Filebrowser in storage-proxy

[achimnol]

Moves file browsers to the storage proxy.
This will greatly reduce the burden of manager and webserver to mediate large file transfers.

• Implement filebrowser container mgmt (with a configurable maximum number of containers)
• Add new manager-facing storage-proxy APIs to create/destroy filebrowser sessions
• manager: Add new vfolder APIs to map the new manager-facing storage-proxy APIs
• client-py: Add new functional wrappers and vfolder subcommands to use the filebrowser session APIs
• webui: Update the current filebrowser UIs to use the new filebrowser session mgmt APIs

To think:

• Shall we allow explicit listing of filebrowser sessions in the client-side?
• How to expire unused filebrowser sessions automatically?

[leksikov]

• Implemented main feature such as the communication between Client, Manager and Storage Proxy to support FileBrowser session where user can attach multiple folders and copy/move data using UI.
• Implemented the CLI commands on client side to start filebrowser with attached folders.
• Implemented Manager API interface to redirect requests from Client to Storage Proxy and receive the response from Storage Proxy about the url, port number and container id.
• The user using CLI can destroy the filebrowser session using the container id information printed out in cli.

While implementing the filebrowser, I decoupled filebrowser from Vfolder as a separate app.
This will allow having filebrowser its own routing management for more complex commands.
Decoupling will help to improve maintenance and advanced feature implementation of Filebrowser without affecting Vfolder.

Next things to-do:

• Need to think how to and where to store the Object docker container in order to auto-destroy after a certain time.
• The FileBrowser has customizations options such as company title and logo for WebUI. Can be implemented by sending exec command into settings options
• Supporting multiple docker container sessions per user at the same time. Maybe by having different randomized access port as easy and quick solution?

[achimnol]

Just some initial reviews. (Since I'm the creator of this PR, I cannot "approve" or "request changes", so I'm leaving the review as a plain comment.)
We need to think more about automatically terminating "idle" filebrowser containers.

[achimnol]

We could use BinarySize of ai.backend.common.validators (usually imported as tx which means "extended trafarets") in .config.local_config_iv to avoid repetition like this calculation.

[achimnol]

Could we refactor this out as a reusable function?

[achimnol]

Please implement max-containers configuration.
This would require a local tracking of container states, like the agent, so that we could count the existing number of containers and get notified when the containers terminate by themselves.

Ideally, we could extend a ai.backend.common.docker module to provide Docker lifecycle events subscription and basic parsing into typed Python objects, and reuse it across the agent and storage-proxy.

[leksikov]

With recent git commit, proposed simple solution to prevent execution of multiple monitors and network monitoring for 1 filebrowser container. The executed monitor checks if there is a file lock exists. If file not exist then create asynchronously file lock and continue monitoring task.

Other monitors who get called are checking file lock existence. Since file lock already exists because of the first monitor, all following monitors will be terminated and not proceed with their monitoring task. Hence, only one monitor will be running who create the file lock first.

The file lock is automatically deleted when the storage proxy is shut down.

[achimnol]

With recent git commit, proposed simple solution to prevent execution of multiple monitors and network monitoring for 1 filebrowser container. The executed monitor checks if there is a file lock exists. If file not exist then create asynchronously file lock and continue monitoring task.

Other monitors who get called are checking file lock existence. Since file lock already exists because of the first monitor, all following monitors will be terminated and not proceed with their monitoring task. Hence, only one monitor will be running who create the file lock first.

The file lock is automatically deleted when the storage proxy is shut down.

Why not used ai.backend.common.lock.FileLock and/or ai.backend.common.distributed.GlobalTimer?
Simply opening files may have races!

[leksikov]

Previously, did the implementation on the server of preventing multiple monitors run with common.FileLock.

Implemented UID and GID settings in a startup.sh script after the containers are set. The input variables are defined in the .toml settings file.

Refactoring, in order to deal with multiple volumes and their path in the settings file.

Added a host:volume option in client-py. This is needed for the following things: to specify the volume on which folders are located. The same name folder can occur on multiple volumes. Then based on given argument options such as volume and name the vfid of vfolder is infered. Then filebrowser based on volume path and vfid is able to mount those directories into the container.

On client-py command execution example:
backend.ai filebrowser create -host local:volume1 -vf mydata1 -vf mydata2

[inureyes]

This PR is resolved at lablup/backend.ai#710