Using Telemetry to Monitor Performance

Telemetry provides information on the structure, status, and progress of data flowing in s-Server, such as the net input and output bytes for a particular stream, when a stream started processing data, and when a stream stopped processing data. Telemetry allows you to see the overall structure of your data pipelines, how your data is flowing and where it is actually transformed, and the progress and throughput of data moved through pipelines, so that, for example, you can identify performance-slowing bottlenecks. The management views SESSIONS_VIEW and STATEMENTS_VIEW may both be useful in identifying graphs and nodes related to your streaming application.

This topic contains the following information:

Background Concepts: Stream-graphs, Nodes, and s-Server

To understand telemetry and related concepts, it helps to have a clear picture of how s-Server computes the results of a query.

s-Server consist of two layers:

  1. A query engine that maintains a collection of SQL definitions and translates a SQL statement into a specified data-flow.
  2. A data engine that executes and coordinates these data-flows.

The specification of a data-flow is called a physical query plan. The actual data flow is called a stream graph, which is a chain of stream nodes, each of which is a small software machine that applies some relational operation to its input data stream(s) to produce is output data stream(s). Stream graphs are also referred to as stream operators. Nexus nodes are defined by a CREATE STREAM statement. They are freestanding nodes to which other stream graphs can connect and disconnect, in order to receive or deliver data.

Nodes Stream Graph

In the most common case, a node has 1 input and 1 output, and so the graph is a linear chain of operations. But data flows can merge and split, and the graph of a new query can connect to the graphs of existing queries as they run.

Reading Telemetry Data

Telemetry data is available through system functions that you can query using SQL. Functions are available as both tables and streams. Tables provide a single snapshot of the server, while streams return continually-updating information.

Category Table Function Stream Function
Stream Graph Virtual Table/Stream. Contains a row to describe each stream-graph. Stream-graphs represent one SQL statement running in s-Server, usually either a SELECT, INSERT, or a running PUMP. getStreamGraphInfo(session integer, recent float) getStreamGraphInfoForever(session integer, period float)
Stream Operator (Node) Virtual Table/Stream. Contains a row to describe each stream-node. Stream-nodes represent one step in the running implementation of a SQL statement. getStreamOperatorInfo(session integer, recent float) getStreamOperatorInfoForever(session integer, period float)
Server Info Virtual Table/Stream. Contains a single row, which describes s-Server as a whole. Each column is some attribute (status or statistic) of the overall server. getServerInfo(recent float) getServerInfoForever(period float)

These functions take the following arguments:

Argument Definition
SESSION getStreamGraphInfo and getStreamOperatorInfo only. Sessions refer to a single connection to s-Server. You can designate a session ID, or use 0 to mean all sessions.To determine session, you can query sys_boot.mgmt.sessions_view: select session_name, id from sys_boot.mgmt.sessions_view; Version 6.0 Feature As of version 6.0, you no longer need to use the preface SYS_BOOT.MGMT. This query will return results along the following lines:

Session_Name: LOCALDB.StreamLab_Output_Buses.pipeline_2_step_3-to-pipeline_2_out_sink_1-Pump ID: 7338* You should be able to identify your desired session by session_name. Use the corresponding ID to get information on this session. See the topic s-Server Features for Managing and Monitoring for more details on sys_boot.mgmt.sessions_view.
RECENT In seconds, how recent a snapshot is required. 0 will force a new snapshot, 10 means use a extant snapshot no more than 10 secs old. You can use RECENT to get a pair of results that match the same snapshot. For more detail, see How RECENT Works below.Note: It’s a good idea to add 0.5 sec to RECENT for the second query. This allows room for the 2nd query to be processed slightly later than the 1st.
PERIOD In seconds, how often to fetch and emit new data.

You query these using a statement along the following lines:

SELECT NAME_IN_QUERY_PLAN, NODE_ID, NET_INPUT_ROWS, NET_INPUT_BYTES, NET_OUTPUT_ROWS, NET_OUTPUT_BYTES FROM TABLE (sys_boot.mgmt.getStreamOperatorInfo(0, 10));

Note: Columns do not require quotation marks.

Version 6.0 Feature As of version 6.0, you no longer need to use the preface SYS_BOOT.MGMT.

Note that the above query uses ‘0’ for SESSION. This tells s-Server “give me information on all sessions.” This query returns a result along the following lines:

NAME_IN_QUERY_PLAN NODE_ID NET_INPUT_ROWS NET_INPUT_BYTES NET_OUTPUT_ROWS NET_OUTPUT_BYTES
[LOCALDB, StreamLab_Output_Meetup, dashboard_map_step_2] 325006 36684 733688 25216 504312
[LOCALDB, StreamLab_Output_Meetup, dashboard_map_step_2] 325006 40779 815584 29310 586208
[LOCALDB, StreamLab_Output_Meetup, us_rsvps] 325008 41870 837408 113269 2265384
[LOCALDB, StreamLab_Output_Meetup, dashboard_bar_CHARt_step_2] 325010 42614 852288 0 0
[LOCALDB, StreamLab_Output_Meetup, us_rsvps] 325008 46490 929792 127127 2542536
[LOCALDB, StreamLab_Output_Meetup, dashboard_bar_CHARt_step_2] 325010 47316 946328 0 0

To see all the info about all the statements that belong to session 7338, run the query:

SELECT * FROM TABLE(SYS_BOOT.MGMT.getStreamGraphInfo(7338, 0));

Version 6.0 Feature As of version 6.0, you no longer need to use the preface SYS_BOOT.MGMT.

Filtering Data

You can restrict your results using a SELECT list or WHERE clauses, and you can compute expressions based on the telemetry data. That is, you can use SQL to limit or filter telemetry data. The management views SESSIONS_VIEW and STATEMENTS_VIEW may both be useful in identifying graphs and nodes related to your streaming application.

All streams created by users have a NAME_IN_QUERY_PLAN that corresponds to the fully qualified sql name of the stream, which has the format [LOCALDB, , ].

Therefore, you could filter for only named streams by running the following query:

SELECT STREAM * FROM STREAM(sys_boot.mgmt.getStreamOperatorInfoForever(0, 10)) WHERE NAME_IN_QUERY_PLAN LIKE '%LOCALDB%';

To filter only streams from a particular schema, you could run a query along the following lines:

SELECT SELECT STREAM * FROM STREAM(sys_boot.mgmt.getStreamOperatorInfoForever(0, 10)) WHERE NAME_IN_QUERY_PLAN LIKE '%LOCALDB, my-schema%';

You can also query individual columns, as in

SELECT STREAM NAME_IN_QUERY_PLAN, NODE_ID, NET_INPUT_ROWS, NET_INPUT_BYTES,  NET_OUTPUT_ROWS, NET_OUTPUT_BYTES FROM STREAM(sys_boot.mgmt.getStreamOperatorInfoForever(0, 10));

A full list of columns for each virtual table/stream appears at the end of this page.

Version 6.0 Feature As of version 6.0, you no longer need to use the preface SYS_BOOT.MGMT.

Table/Stream Descriptions

Stream Graph Virtual Table/Stream

(getStreamGraphInfo/getStreamGraphInfoForever)

This table/stream returns rows for each stream graph. A stream graph generally corresponds to one SQL statement (DML or query). Stream graph data offers you an overview of how a particular node in a stream graph is processing data, including information on how much data it has processed since inception (in bytes and rows), how quickly it is processing data, and how much data it has output since inception.

We recommend looking at StreamGraphInfo first. Once you have identified a stream graph with issues, you can then use StreamOperatorInfo to see how the operation is doing in more detail, likely by filtering on graph_id, which serves as a foreign key between the two tables/streams.

The set of columns fall into four groups: identification, description, status, and statistics. Identification and description are stable identifiers and CHARacteristics for the node, including numerical identifiers for the node and stream_graph, and so on. Status and statistics offer information on what the stream node is doing now, how long it has been doing it, how much data it has processed, how quickly it is doing so, and so on. The former are useful for determining information about a node, and the latter are useful for understanding how the stream node is performing.

Identification and Descriptive Columns

  • node_id is a VARCHAR. It consists of a dotted pair of integers, such as 1.0 or 2.3. The first number represents the stream graph, and the second the stream graph node. 1.0 means node #0 in stream graph #1, and 4.3 means node #3 in stream graph #4.
  • graph_id is the same as the first (graph) part of node_id. It is provided as a column to serve as a foreign key.
  • name_in_query_plan is for advanced debugging. It is a generated name that matches the logical and physical query plans as reported in the s-Server trace log. The overall structure of the data flow graph is indicated by listing the node_id of the input neighbor and the output neighbor.

Note: The s-Server query makes a unique identifier for each stream node.

Status Fields

These fields are helpful in answering questions like “why is my pipeline stuck?” or “why won’t the scheduler do what I want it to do next?” Important columns include the following:

  • sched_state. Whether node is blocked (needs more input / needs room to output), suspended, finished, or ready to run. The values of sched_state are:
    • R* running
    • R ready to run
    • N not runnable, because a neighbor is running
    • B blocked for data (underflow or overflow)
    • T suspended externally,
    • E finished (has reached end of input stream)
    • C stream graph is closed (but can be re-opened and re-executed)
    • Z stream graph was removed (SQL statement is closed).
  • last_exec_result. Indicates why the node is blocked. UND means underflow, and OVR means overflow. EOS means the node has seen the end-of-stream marker in its input; it is finished.
  • num_busy_neighbors. When the data engine puts a DOUBLE-buffer between two adjacent data nodes, they both can run at the same time. But when a single-buffer lies between two adjacent data nodes, only one can run at a time. A value num_busy_neighbors > 0 means the node is blocked because of a busy buffer.
  • input_rowtime_clock. The rowtime field of the next available input row, or else the next input rowtime bound.
  • output_rowtime_clock. The rowtime field of the latest row output by this node. The clock rowtimes show how far the data has progressed down the pipeline. This shows latency, overall and between nodes. This is important for diagnosing when a pipeline is stuck because a node is waiting for an earlier row from an empty input buffer.

Input and output are measured in bytes and bytes/sec. Input and output are also measured in rows and rows/sec when possible. Most stream operators process row by row, and so can count rows. But a few process a whole buffer at a time, and since row-length can vary, we don’t pay the cost of converting, and sometimes report null for the stats in terms of rows.

A full list of columns and descriptions appears at the end of this page.

Stream Operator (Node) Virtual Table/Stream

(getStreamOperatorInfo/getStreamOperatorInfoForever)

This table/stream offers you an overview of how a node (also known as a stream operator) is processing data.

Columns for these rows fall into four groups: identification, description, status, and statistics. Identification and description are stable identifiers and CHARacteristics for the stream, including numerical identifiers for the stream graph, its session number, and so on. Status and statistics offer information on what the stream graph is doing now, how long it has been doing it, how much data it has processed, how quickly it is doing so, and so on. The former are useful for determining information about a stream graph, and the latter are useful for understanding how the stream graph is performing.

Identification columns consist of graph_id and statement_id. These are both unique numerical identifiers.

Description columns include session_id, source_sql, is_global_nexus, is_auto_close, num_nodes, and num_data_buffers. These tell you what the stream looks like before any processing occurs.

Status columns include sched_state, num_live_nodes, and num_data_buffers. These columns tell you how the scheduler is handling the stream, as well as how many active nodes the stream is engaging.

Statistics columns include net_execution_time, net_schedule_time, net_input_bytes, net_input_rows, net_input_rate, net_input_row_rate, net_output_bytes, net_output_rows, net_output_rate, and net_output_row_rate.

A full list of columns and descriptions appears at the end of this page.

Server Info Virtual Table/Stream

(getServerInfo/getServerInfoForever)

The Server Info Virtual Table provides a summary of the entire server. This includes information on the number of currently active sessions, the number of currently active SQL statements, the number of currently executing threads and the number of stream graphs currently open.

A full list of columns and descriptions appears at the end of this page.

Common Use Cases

Using Telemetry to Track Memory Use

Telemetry tracks two kinds of memory:

  • Some stream operators/nodes require a significant amount of auxiliary memory. For example, windowed aggregation requires remembering data from past rows, and this is kept in a window structure that can be large. The amount of auxiliary memory can vary over time. The Stream Node Table/Stream (getStreamOperatorInfo) keeps track of the current memory used by a stream operator/node in the net_memory_bytes column. It also tracks the maximum value since the operator started in the max_memory_bytes column. Each node, represented by row in getStreamOperatorInfo has its own pair of values. You can query these columns with a statement along the following lines:

    SELECT STREAM net_memory_bytes, max_memory_bytes from stream(sys_boot.mgmt.getStreamOperatorInfoForever(0, 1));
    
  • The other kind of memory that can be significant is that used by the data buffers that s-Server uses to pass rows from node to node. You can track this memory using the Stream Graph Table/Stream. In the Stream Graph Table, keeps track of the total of all the buffers in the graph plus all the auxiliary memory required by all the nodes in the graph in the net_memory_bytes column. It also tracks the maximum over the lifetime of the stream graph in the max_memory_bytes column. You can query these columns with a statement along the following lines:

    SELECT STREAM net_memory_bytes, max_memory_bytes from stream(getStreamGraphInfoForever(0, 1));
    

Counting Total Records or Bytes Read in From External Sources

You can get information on bytes or rows read by a stream graph by calling getStreamGraphInfo() for a session in your application. To identify your application’s session, we recommend looking at sys_boot.mgmt.sessions_view. You may also be able to identify your application’s stream graph by looking at the column NAME_IN_QUERY_PLAN. For named streams, these correspond to the fully qualified sql name of the stream, which has the format [LOCALDB, , ].

To get information on bytes or rows read into a stream graph, query the following columns:

Column Type Description
net_input_bytes BIGINT Overall input read by the stream graph since it started, in bytes.
net_input_rows BIGINT Overall input read by the stream graph since it started, in rows.
net_input_rate DOUBLE Overall input rate, averaged since the stream graph started, as bytes/sec.
net_input_row_rate DOUBLE Overall input rate, averaged since the stream graph started, as rows/sec.
net_output_bytes BIGINT Overall output written by the stream graph since it started, in bytes.
net_output_rows BIGINT Overall output written by the stream graph since it started, in rows
net_output_rate DOUBLE Overall output rate, averaged since the stream graph started, as bytes/sec.
net_output_row_rate DOUBLE Overall output rate, averaged since the stream graph started, as rows/sec.


Overall input means the total input that enters the stream graph from the outside.

Measuring Throughput In and Out of s-Server

To determine total throughput, you can aggregate the throughput columns emitted by getStreamGraphInfo(). Because getStreamGraphInfo() lists all stream graphs, including those used internally by s-Server, you will likely want to filter on the columns session_id, graph_id, or statement_id in order to limit your aggregation to only those stream graphs used by your application.

Moving Telemetry Data into a Native Stream

You can use a pump to move telemetry data into a stream that you’ve created. You first need to set up a stream that will accept columns, in order, from the telemetry table/stream that you want to use. The example below uses columns from getStreamOperatorInfoForever().


CREATE OR REPLACE SCHEMA "telemetry-results";
SET schema '"telemetry-results"';

CREATE OR REPLACE STREAM "telemetry-results-stream"

(NODE_ID VARCHAR(32),
LAST_EXEC_RESULT VARCHAR(32),
NET_INPUT_ROWS BIGINT,
INPUT_ROWTIME_CLOCK TIMESTAMP,
NET_INPUT_BYTES BIGINT,
NAME_IN_QUERY_PLAN VARCHAR(32),
NET_OUTPUT_ROWS BIGINT,
OUTPUT_ROWTIME_CLOCK TIMESTAMP,
NET_OUTPUT_BYTES BIGINT
)
;

CREATE OR REPLACE PUMP "telemetry-results"."my-Pump" STOPPED AS
INSERT INTO "telemetry-results"."telemetry-results-stream"
(NODE_ID, LAST_EXEC_RESULT, NET_INPUT_ROWS, INPUT_ROWTIME_CLOCK, NET_INPUT_BYTES, NAME_IN_QUERY_PLAN, NET_OUTPUT_ROWS, OUTPUT_ROWTIME_CLOCK, NET_OUTPUT_BYTES)
SELECT STREAM NODE_ID, LAST_EXEC_RESULT, NET_INPUT_ROWS, INPUT_ROWTIME_CLOCK, NET_INPUT_BYTES, NAME_IN_QUERY_PLAN, NET_OUTPUT_ROWS, OUTPUT_ROWTIME_CLOCK, NET_OUTPUT_BYTES FROM STREAM (sys_boot.mgmt.getStreamOperatorInfoForever(0, 1));
--selects columns from telemetry Stream Operator stream

By default, pumps are created as stopped. That means when you initially create them, they don’t move any data.

To start the pump, enter the following line:

ALTER PUMP "telemetry-results"."my-Pump" START;

To query the new stream, enter the following line:

SELECT STREAM * from "telemetry-results"."telemetry-results-stream";

How RECENT Works

Here the 2 requests session 2, and the 0 require a snapshot no older than 0 secs. That is, it forces a new snapshot.

If you asked instead:

SELECT * FROM TABLE(SYS_BOOT.MGMT.getStreamGraphInfo(2, 5));

the system would use a snapshot it already had if this snapshot was no older than 5 seconds. (If that data is good enough, it saves a little work to use the existing snapshot.)

Or consider:

SELECT * FROM TABLE(SYS_BOOT.MGMT.getStreamGraphInfo(0, 0));

Session = 0 means report on all sessions (since the lowest session ID is 1).

You can use RECENT to match queries:

SELECT * FROM TABLE(SYS_BOOT.MGMT.getServerInfo(0, 0));
SELECT * FROM TABLE(SYS_BOOT.MGMT.getStreamGraphInfo(0, 2));
SELECT * FROM TABLE(SYS_BOOT.MGMT.getStreamOperatorInfo(0, 2));

We want all 3 parts to come from the same snapshot, so that all the data is consistent. Note that the 1st query forces a new snapshot, and the following 2 queries accept stale data (up to 2 seconds state). They will thus re-use the snapshot from the 1st query.

Version 6.0 Feature As of version 6.0, you no longer need to use the preface SYS_BOOT.MGMT.

Full List of Column Descriptions

The following tables list all columns, types, and definitions for the telemetry virtual tables.

Stream Graph Virtual Table Column Types and Definitions

Column Name Type Definition
measured_at TIMESTAMP NOT NULL Time of query snapshot.
graph_id INT NOT NULL Numerical identifier for stream graph. Numbered from 1 as graphs are added to scheduler. The same column appears in the Stream Node Virtual Table, serving as a foreign key. You can use graph_id in conjunction with the Stream Node table to learn more about individual nodes in a graph.
name_in_query_plan VARCHAR A generated name that matches the logical and physical query plans as reported in the s-Server trace log. All streams created by users have a NAME_IN_QUERY_PLAN that corresponds to the fully qualified name of the stream, which has the format [LOCALDB, , ].
statement_id INT 0 if not from a SQL statement. These are listed in the monitoring view called STATEMENTS_VIEW. See s-Server Features for Managing and Monitoring for more details.
session_id INT Numerical identifier for the session, which is equates to JDBC Connection to s-Server. These are listed in the monitoring view called SESSIONS_VIEW. See s-Server Features for Managing and Monitoring for more details.
source_sql VARCHAR(2048) SQL for statement.
sched_state CHAR(1) State of the stream graph in the scheduler. Either B (blocked).O(pen), C(losed), R(unnable), or E(nd of Stream)
close_mode CHAR(4) When the stream graph was closed, and how it was closed.
is_global_nexus BOOLEAN Indicates a named stream. These are defined by a CREATE STREAM statement. They are freestanding nodes to which other stream graphs can connect and disconnect, in order to receive or deliver data.
is_auto_close BOOLEAN Whether or not the stream will close automatically at state E
num_nodes INT NOT NULL Number of nodes in the stream graph. These may be shared with other stream graphs, as in the case of named streams.
num_live_nodes INT NOT NULL Number of open nodes not at state E.
num_data_buffers INT NOT NULL Number of data buffers between nodes.Note: Two adjacent nodes share a data buffer; the upstream node writes rows to the buffer, and the downstream node reads rows from the buffer.
total_execution_time DOUBLE Time spent executing this stream graph in seconds.
total_opening_time DOUBLE Time spent setting up this stream graph in seconds.Note: When a user sends a SQL query to the server, the query engine sends a query plan to the data engine layer. The data engine then “sets up” the query, by performing a number of steps, including allocating and initializing software objects that do the data operations, allocating data buffers to stand between the operators, adding to internal control & bookkeeping data structures, and connecting the graph to one or more global nexuses.
total_closing_time DOUBLE Time spent tearing down this graph in seconds.Note: When the user ends the query, the data engine stops executing the stream graph and does the set-up in reverse.
net_input_bytes BIGINT Overall input read by the stream graph since it started, in bytes.
net_input_rows BIGINT Overall input read by the stream graph since it started, in rows.
net_input_rate DOUBLE Overall input rate, averaged since the stream graph started, as bytes/sec.
net_input_row_rate DOUBLE Overall input rate, averaged since the stream graph started, as rows/sec.
net_output_bytes BIGINT Overall output written by the stream graph since it started, in bytes.
net_output_rows BIGINT Overall output written by the stream graph since it started, in rows
net_output_rate DOUBLE Overall output rate, averaged since the stream graph started, as bytes/sec.
net_output_row_rate DOUBLE Overall output rate, averaged since the stream graph started, as rows/sec.
net_memory_bytes BIGINT Bytes of working memory now for whole stream graph.
max_memory_bytes BIGINT Maximum bytes of working memory since stream graph opened.
when_opened timestamp System time when stream graph was opened.
when_started timestamp System time of first execution of stream graph.
when_finished timestamp System time of latest execution of stream graph.
when_closed timestamp System time when stream graph was closed.

Stream Nodes Virtual Table Column Types and Definitions (getStreamOperatorInfo)

Column name Type Definition
measured_at TIMESTAMP NOT NULL Time of query snapshot.
node_id VARCHAR(8) NOT NULL Node identifier, such as 4.6 for the 6th node in the 4th stream graph
graph_id INT NOT NULL Graph identifier. This column also appears in the Stream Graphs table/stream.
source_sql VARCHAR(2048) SQL for stream graph.
num_inputs INT NOT NULL Count of input neighbors.
input_nodes VARCHAR(64) NOT NULL node_id numerical identifier of input neighbors.
num_outputs INT NOT NULL Count of output neighbors.
output_nodes VARCHAR(64) NOT NULL node_id numerical identifier of output neighbors.
sched_state CHAR(2) State of node in scheduler. Either C(losed), E(ended), R* (running), R(unnable), or B (blocked).
last_exec_result CHAR(3) Latest code returned by ExecStream::execute() eg UND(erflow), OVR(overflow)
num_busy_neighbors INT When positive, indicates that this node is blocked because some neighboring nodes are busy.
input_rowtime_clock timestamp Rowtime of the latest input row read by this node.
output_rowtime_clock timestamp Rowtime of the latest output row written by this node.
execution_count BIGINT Number of times this node has been executed by the scheduler.
started_at timestamp System time of first execution.
latest_at timestamp System time of latest execution.
net_execution_time DOUBLE Total time node has executed in seconds.
net_schedule_time DOUBLE Total overhead scheduling for operator in seconds.
net_input_bytes BIGINT Overall input read by the stream node since it started, in bytes.
net_input_rows BIGINT Overall input read by the stream node since it started, in rows.
net_input_rate DOUBLE Overall input rate, averaged since the stream node started, as bytes/sec.
net_input_row_rate DOUBLE Overall input rate, averaged since the stream node started, as rows/sec.
net_output_bytes BIGINT Overall output written by the stream node since it started, in bytes.
net_output_rows BIGINT Overall output written by the stream node since it started, in rows
net_output_rate DOUBLE Overall output rate, averaged since the stream node started, as bytes/sec.
net_output_row_rate DOUBLE Overall output rate, averaged since the stream node started, as rows/sec.

Server Info Virtual Table

The Server Info Virtual Table provides a snapshot of the entire server instance. Because it summarizes the entire server, this table has only one row.

Column Type Meaning
measured_at TIMESTAMP NOT NULL Time of snapshot.
is_running BOOLEAN NOT NULL Whether or not server is running.
is_licensed BOOLEAN NOT NULL Whether or not server has some kind of license.
license_kind VARCHAR(32) Type of license.
license_version VARCHAR(32) Version number for license.
is_throttled BOOLEAN NOT NULL Whether or not the server is currently throttled.
num_sessions INT Number of currently active sessions.
num_statements INT Number of currently active SQL statements.
started_at timestamp Time server started.
throttled_at timestamp Time throttling started. If server is not throttled, value is null.
throttle_level DOUBLE Throughput limit enforced by throttling. If server is not throttled, value is null.
num_exec_threads INT Number of currently executing threads.
num_stream_graphs_open INT Number of stream graphs currently open.
num_stream_graphs_closed INT Number of stream graphs currently closed.
num_stream_operators INT Number of stream operated.
num_stream_graphs_open_ever INT Number of stream graphs opened since server started.
num_stream_graphs_closed_ever INT Number of stream graphs closed since server started.
net_memory_bytes BIGINT Total bytes of working memory.
max_memory_bytes BIGINT Max bytes of working memory since server started.
usage_at timestamp Timestamp of last usage snapshot.
usage_since timestamp Usage data is from usage_since to usage_at.
usage_reported_at timestamp Timestamp of last successful upload of usage data.
net_input_bytes_today BIGINT Overall input read by the server since midnight, in bytes.
net_input_bytes BIGINT Overall input read by the server since it started, in bytes.
net_output_bytes BIGINT Overall output written by the server since it started, in bytes.