A web interface of gpustat ---
aggregate gpustat across multiple nodes.
NOTE: This project is in alpha stage. Errors and exceptions are not well handled, and it might use much network resources. Please use at your own risk!
pip install gpustat-web
Python 3.6+ is required.
Launch the application as follows. A SSH connection will be established to each of the specified hosts.
Make sure ssh <host> works under a proper authentication scheme such as SSH authentication.
gpustat-web --port 48109 HOST1 [... HOSTN]
You might get "Host key is not trusted for <host>" errors. You'll have to accept and trust SSH keys of the host for the first time (it's stored in ~/.ssh/known_hosts);
try ssh <host> in the command line, or ssh -oStrictHostKeyChecking=accept-new <host> to automatically accept the host key. You can also use an option gpustat-web --no-verify-host to bypass SSH Host key validation (although not recommended).
Note that asyncssh does NOT obey the ~/.ssh/config file
(e.g. alias, username, keyfile), so any config in ~/.ssh/config might not be picked up.
https://HOST:PORT/: A webpage that updates automatically through websocket.https://HOST:PORT/gpustat.html: Result as a static HTML page.https://HOST:PORT/gpustat.txt: Result as a static plain text.https://HOST:PORT/gpustat.ansi: Result as a static text with ANSI color codes. Trycurl https://.../gpustat.ansi
Query strings:
?nodes=gpu001,gpu002: Select a subset of nodes to query and display
By default the web server will run as a HTTP server.
If you want to run a secure SSL/TLS server over the HTTPS protocol, use --ssl-certfile and --ssl-keyfile option.
You can use letsencrypt (certbot) to create a pair of SSL certificate and keyfile.
Troubleshoothing: Verify SSL/TLS handshaking (if TLS connections cannot be established)
openssl s_client -showcerts -connect YOUR_HOST.com:PORT < /dev/null
To see CPU usage as well:
python -m gpustat_web --exec 'gpustat --color --gpuname-width 25 && echo -en "CPU : \033[0;31m" && cpu-usage | ascii-bar 27'
MIT License
Copyright (c) 2018-2023 Jongwook Choi