dashboard: cpu/memory/virtual memory panels

[patapenka-alexey]

This patch adds CPU/memory/virtual memory utilization panels per instance and total.

Closes #TNTP-4365

[oleg-jukovec]

Please, squash commits.

[oleg-jukovec]

Please, revert the changes or fix the problem.

[patapenka-alexey]

It's temporarily debug stuff.

[DifferentialOrange]

So, do you plan to support it later for InfluxDB or not? If not, let's not add an empty panel to InfluxDB dashboard.

[patapenka-alexey]

InfluxDB panels have been added.

[DifferentialOrange]

To properly support non-default configurations, targets must consider the pre-configured filters (either static or dynamic) and metrics prefix.

The basic example is as follows. By default, Tarantool dashboard expects the user to have one Prometheus job per application, so all queries must get metrics only for a specific job. Otherwise we will display the data for all Tarantool application over Prometheus.

tnt_metric{job='myjob'}

grafana-dashboard library allows to provide various additional filters on build. The dashboard we publish to grafana.com has the following ones

grafana-dashboard/config.yml

Lines 49 to 52 in a225559

	filters: {
	job: ['=~', '$job'],
	alias: ['=~', '$alias'],
	}

It allows to choose a job for your application, as well as display data only for certain application instances instead of all of them, if one wishes to.

grafana-dashboard library has a couple of helpers to compute such stuff for you, but in case of non-trivial queries (with additional sums) we'll need to reimplement them a little bit.

[DifferentialOrange]

(I have used this Prometheus target as an example.)

So, let's rewrite this one. To add a prefix, we need to simply concat it

std.format(|||
    'rate(%(metrics_prefix)tnt_cpu_user_time[$__rate_interval]) + rate(%(metrics_prefix)tnt_cpu_system_time[$__rate_interval])'
|||, {
    metrics_prefix: cfg.metrics_prefix,
})

I use named std.format here since positional one will look more complicated here.

Next, we need to add common filters

std.format(|||
     'rate(%(metrics_prefix)tnt_cpu_user_time{%(filters)s}[$__rate_interval]) + rate(%(metrics_prefix)tnt_cpu_system_time{%(filters)s}[$__rate_interval])'
|||, {
    metrics_prefix: cfg.metrics_prefix,
    filters: common.prometheus_query_filters(cfg.filters),
})

[patapenka-alexey]

Thanks a lot for your tips!

[DifferentialOrange]

Next, to sum over all instances, we need to exclude alias filter

std.format(|||
     'sum(rate(%(metrics_prefix)tnt_cpu_user_time{%(filters)}[$__rate_interval])) + sum(rate(%(metrics_prefix)tnt_cpu_system_time{%(filters)}[$__rate_interval]))'
|||, {
    metrics_prefix: cfg.metrics_prefix,
    filters: common.prometheus_query_filters(common.remove_field(cfg.filters, 'alias')),
})

[patapenka-alexey]

Oops, forgot about that. Updated.

[Satbek]

Is it possible to sum system and user time for cpu graphs by thread?

[patapenka-alexey]

Is it possible to sum system and user time for cpu graphs by thread?

Yes, all required metrics are provided by metrics. But I think that will be done in another ticket.