Analyzing Data in s-Server

Purpose of this document

This document describes the basics of analyzing streaming data in s-Server, including writing streaming queries, using windowed aggregation, and moving data around in s-Server.

Audience

This document is intended for architects and developers who will analyze data in s-Server.

Sections

In basic cases, querying streaming data is not different from querying static data. To filter for all rows that meet a specific criterion, for example, you can use a WHERE clause, just as you would for static data. The topic Writing Simple Streaming SQL Queries provides a few simple examples of streaming queries.

In most cases, though, you will need to consider the time dimension of streams. (By definition, streams have timestamps attached to each rows, known as rowtimes.)

These considerations fall into three general categories “how far into the past do I want this data?” or “over what limited time window do I want this data?” or “what number of rows do I want to analyze” (There are some exceptions to these general categories.) These involve using windows, and the topic Windowed Aggregation on Streams describes how to use windows to aggregate data. You can also use windows to correlate streams in order to join them, as described in the topic

Moving Data Around in s-Server

As you develop your s-Server streaming application, you’ll find that it is important to be able to move data around. In s-Server, you use pumps to get data from one stream to another. A pump a SQLstream schema object that provides a continuously running INSERT INTO stream SELECT … FROM query functionality. The topic Using Pumps to Create Processing Pipelines describes how to use pumps to structure your applications.

You will also often want to join two streams together, as described in the topic Creating Streaming JOINs.

Bringing it All Together in Query Patterns

The topic Query Patterns describes some common use cases for streaming SQL queries.

Using UDXs and UDFs to Analyze Data

User-Defined Transforms and User-Defined Functions allow you to perform more complex analysis on data. The topic Using UDXs and UDFs for Analysis describes how to use these.

Viewing Demonstration Applications

To help you understand the concepts above, s-Server ships with demonstration applications. The topic Running Demo Applications describes how to use these.

Using UDXs and UDFs for More Complicated Analysis

Some business problems will require you to create customized User Defined Transforms or User Defined Functions. The topic Using UDXs and UDFs for Analysis offers an overview of doing so.