diff --git a/Readme.ipynb b/Readme.ipynb index 4da047f..14cd2a1 100644 --- a/Readme.ipynb +++ b/Readme.ipynb @@ -118,7 +118,10 @@ "* [**Group Metrics**](tutorials/Stackdriver Monitoring/Group metrics.ipynb) - How to list Stackdriver groups, and query time series data for a given group.\n", "\n", "\n", - "* [**Time-shifted Data**](tutorials/Stackdriver Monitoring/Time-shifted data.ipynb) - How to compare today's metric data against the past week by time-shifting the time series data." + "* [**Time-shifted Data**](tutorials/Stackdriver Monitoring/Time-shifted data.ipynb) - How to compare today's metric data against the past week by time-shifting the time series data.", + "\n", + "\n", + "* [**Distribution Metrics**](tutorials/Stackdriver Monitoring/Distribution Metrics.ipynb) - How to visualize the timeseries for Distribution metrics." ] }, { diff --git a/tutorials/Stackdriver Monitoring/Distribution Metrics.ipynb b/tutorials/Stackdriver Monitoring/Distribution Metrics.ipynb new file mode 100644 index 0000000..f315ad1 --- /dev/null +++ b/tutorials/Stackdriver Monitoring/Distribution Metrics.ipynb @@ -0,0 +1,573 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Distribution metrics\n", + "\n", + "In this tutorial, we show how to extract the various components of a distribution metric and visualize them. \n", + "\n", + "**Note:** If your project has data in the last 1 hour for the pubsub metric for message sizes, then that will be used. Otherwise sample data will be loaded from a Google Cloud Storage bucket." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Load the monitoring module and set the default project\n", + "\n", + "If there is no default project set already, you must do so using 'set_datalab_project_id'." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "from datalab.stackdriver import monitoring as gcm\n", + "\n", + "# set_datalab_project_id('my-project-id')" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "METRIC_TYPE = 'pubsub.googleapis.com/topic/message_sizes'" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/html": [ + "
Metric type | Display name | Kind | Value | Unit | Labels |
---|---|---|---|---|---|
pubsub.googleapis.com/subscription/backlog_bytes | Backlog size | GAUGE | INT64 | By | |
pubsub.googleapis.com/subscription/byte_cost | Subscription cost | DELTA | INT64 | By | operation_type, response_code |
pubsub.googleapis.com/subscription/config_updates_count | Subscription updates | DELTA | INT64 | 1 | response_code, operation_type |
pubsub.googleapis.com/subscription/mod_ack_deadline_message_operation_count | ModifyAckDeadline operations | DELTA | INT64 | 1 | response_code |
pubsub.googleapis.com/subscription/mod_ack_deadline_request_count | ModifyAckDeadline requests | DELTA | INT64 | 1 | response_class, response_code |
pubsub.googleapis.com/subscription/num_outstanding_messages | Outstanding messages | GAUGE | INT64 | 1 | |
pubsub.googleapis.com/subscription/num_undelivered_messages | Undelivered messages | GAUGE | INT64 | 1 | |
pubsub.googleapis.com/subscription/oldest_unacked_message_age | Oldest message age | GAUGE | INT64 | s | |
pubsub.googleapis.com/subscription/pull_ack_message_operation_count | Acknowledge operations | DELTA | INT64 | 1 | response_code |
pubsub.googleapis.com/subscription/pull_ack_request_count | Acknowledge requests | DELTA | INT64 | 1 | response_class, response_code |
pubsub.googleapis.com/subscription/pull_message_operation_count | Pull operations | DELTA | INT64 | 1 | response_code |
pubsub.googleapis.com/subscription/pull_request_count | Pull requests | DELTA | INT64 | 1 | response_class, response_code |
pubsub.googleapis.com/subscription/push_request_count | Push requests | DELTA | INT64 | 1 | response_class, response_code, delivery_type |
pubsub.googleapis.com/subscription/push_request_latencies | Push latency | DELTA | DISTRIBUTION | us | response_code, delivery_type |
pubsub.googleapis.com/topic/byte_cost | Topic cost | DELTA | INT64 | By | operation_type, response_code |
pubsub.googleapis.com/topic/config_updates_count | Topic updates | DELTA | INT64 | 1 | response_code, operation_type |
pubsub.googleapis.com/topic/message_sizes | Publish message size | DELTA | DISTRIBUTION | By | |
pubsub.googleapis.com/topic/send_message_operation_count | Send operations | DELTA | INT64 | 1 | response_code |
pubsub.googleapis.com/topic/send_request_count | Send requests | DELTA | INT64 | 1 | response_class, response_code |
\n", + " | conductor_fast | \n", + "conductor_fast_aws | \n", + "conductor_initial | \n", + "conductor_initial_aws | \n", + "conductor_kicker | \n", + "conductor_realtime | \n", + "conductor_slow | \n", + "conductor_slow_aws | \n", + "conductor_status | \n", + "
---|---|---|---|---|---|---|---|---|---|
2016-12-08 18:02:52.409 | \n", + "{u'count': u'50850', u'bucketCounts': [u'0', u... | \n", + "{u'count': u'4509', u'bucketCounts': [u'0', u'... | \n", + "NaN | \n", + "{u'count': u'57', u'bucketCounts': [u'0', u'0'... | \n", + "{u'count': u'472', u'bucketCounts': [u'0', u'0... | \n", + "{u'count': u'501', u'bucketCounts': [u'0', u'0... | \n", + "{u'count': u'77', u'bucketCounts': [u'0', u'0'... | \n", + "{u'count': u'95', u'bucketCounts': [u'0', u'0'... | \n", + "{u'count': u'56534', u'bucketCounts': [u'0', u... | \n", + "
2016-12-08 18:03:52.409 | \n", + "{u'count': u'51121', u'bucketCounts': [u'0', u... | \n", + "{u'count': u'4549', u'bucketCounts': [u'0', u'... | \n", + "NaN | \n", + "{u'bucketOptions': {u'exponentialBuckets': {u'... | \n", + "{u'count': u'465', u'bucketCounts': [u'0', u'0... | \n", + "{u'count': u'423', u'bucketCounts': [u'0', u'0... | \n", + "{u'count': u'73', u'bucketCounts': [u'0', u'0'... | \n", + "{u'count': u'87', u'bucketCounts': [u'0', u'0'... | \n", + "{u'count': u'56265', u'bucketCounts': [u'0', u... | \n", + "
2016-12-08 18:04:52.409 | \n", + "{u'count': u'50852', u'bucketCounts': [u'0', u... | \n", + "{u'count': u'4546', u'bucketCounts': [u'0', u'... | \n", + "NaN | \n", + "{u'bucketOptions': {u'exponentialBuckets': {u'... | \n", + "{u'count': u'584', u'bucketCounts': [u'0', u'0... | \n", + "{u'count': u'540', u'bucketCounts': [u'0', u'0... | \n", + "{u'count': u'73', u'bucketCounts': [u'0', u'0'... | \n", + "{u'count': u'84', u'bucketCounts': [u'0', u'0'... | \n", + "{u'count': u'56064', u'bucketCounts': [u'0', u... | \n", + "
2016-12-08 18:05:52.409 | \n", + "{u'count': u'51105', u'bucketCounts': [u'0', u... | \n", + "{u'count': u'4421', u'bucketCounts': [u'0', u'... | \n", + "NaN | \n", + "{u'bucketOptions': {u'exponentialBuckets': {u'... | \n", + "{u'count': u'1191', u'bucketCounts': [u'0', u'... | \n", + "{u'count': u'1188', u'bucketCounts': [u'0', u'... | \n", + "{u'count': u'80', u'bucketCounts': [u'0', u'0'... | \n", + "{u'count': u'81', u'bucketCounts': [u'0', u'0'... | \n", + "{u'count': u'56705', u'bucketCounts': [u'0', u... | \n", + "
2016-12-08 18:06:52.409 | \n", + "{u'count': u'51318', u'bucketCounts': [u'0', u... | \n", + "{u'count': u'4488', u'bucketCounts': [u'0', u'... | \n", + "NaN | \n", + "{u'bucketOptions': {u'exponentialBuckets': {u'... | \n", + "{u'count': u'1307', u'bucketCounts': [u'0', u'... | \n", + "{u'count': u'1205', u'bucketCounts': [u'0', u'... | \n", + "{u'count': u'63', u'bucketCounts': [u'0', u'0'... | \n", + "{u'count': u'66', u'bucketCounts': [u'0', u'0'... | \n", + "{u'count': u'56686', u'bucketCounts': [u'0', u... | \n", + "