[monitoring] Switch to grafana cloud for monitoring

[maru-ava]

Why this should be merged

Our current monitoring stack was only ever intended to be a stop-gap until a managed alternative became available. Grafana Cloud is that alternative.

How this was tested

• CI jobs are able to verify collection of metrics and logs
• Local monitored process testing
• Local monitored kube testing

Need to be documented in RELEASES.md?

N/A

[Copilot]

Pull Request Overview

This PR migrates the monitoring infrastructure from a self-hosted solution to Grafana Cloud, replacing the previous "poc" endpoints with Grafana Cloud's managed services.

• Updates all hardcoded URLs from self-hosted monitoring instances to Grafana Cloud endpoints
• Adds URL configuration support for both Prometheus and Loki collectors
• Updates secret management to include URLs alongside credentials

Reviewed Changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
tests/reexecute/c/README.md	Updates Grafana URLs and dashboard IDs in documentation
tests/fixture/tmpnet/yaml/promtail-daemonset.yaml	Adds Loki URL configuration and updates secret name
tests/fixture/tmpnet/yaml/prometheus-agent.yaml	Adds Prometheus URL configuration and updates secret name
tests/fixture/tmpnet/network.go	Updates Grafana URI and dashboard ID constants
tests/fixture/tmpnet/monitor_processes.go	Refactors collector configuration to include URLs and removes hardcoded endpoints
tests/fixture/tmpnet/monitor_kube.go	Updates Kubernetes secret creation to include URL configuration
tests/fixture/tmpnet/check_monitoring.go	Removes URL validation and adds Prometheus path suffix for Grafana Cloud
tests/fixture/tmpnet/README.md	Updates documentation with new URL requirements and endpoints
tests/e2e/README.md	Updates documentation with new configuration requirements
scripts/configure-local-metrics-collection.sh	Updates Grafana link generation
.github/workflows/*.yml	Updates all workflow files to use new URL-based configuration
.github/actions/run-monitored-tmpnet-cmd/action.yml	Adds URL inputs and updates default dashboard ID

Comments suppressed due to low confidence (1)

tests/fixture/tmpnet/monitor_processes.go:464

• Error message is missing 'env' prefix for consistency with other error messages in the same function. Should be 'env var not set' to match the pattern used for other environment variables.

		return "", "", "", fmt.Errorf("%s env var not set", passwordEnvVar)

[joshua-kim]

LGTM pending a fix for the rate-limiting.

[aaronbuchwald]

LGTM - left one question on a comment I did not understand