Using and Managing Trace Files

SQLstream uses the standard Java logging interface (defined in the package java.util.logging) to trace execution for debugging purposes. Trace files can let you troubleshoot stream functioning during development. Errors logged to the trace log are also available in the global error stream. See the topic Streaming SQL Reference Guide for more details.

As of s-Server 6.0, trace files track Periodic Parser Statistics

Configuration Files for Trace Log

To control the depth of the logging and the formatting of the output, Java logging uses external configuration files in the standard Java properties format. You determine the location of these files during installation. By default, SQLstream uses the following logging configuration files:

Component Configuration File Default Trace Log File(s)
SQLstream s-Server /var/log/sqlstream/Trace.properties /var/log/sqlstream//Trace.log/var/log/sqlstream/ApplicationTrace.log
SQLstream s-Studio andother client tools /var/log/sqlstream/ClientTrace.properties /var/log/sqlstream/ClientTrace.log

The SQLstream s-Server configuration controls tracing to two different log files: Trace.log and ApplicationTrace.log.

Trace.log This is the “master” trace, which includes all trace information for the entire server. ApplicationTrace.log This contains only trace information relevant to installed application components (such as plugins, user-defined transforms, and SQL statements). ClientTrace.log This records trace messages generated by client interaction with Streaming Data Protocol or JDBC; that is, all client interactions with s-Server. You can modify the properties files to change the default logging. Before doing so, you should save a copy of the original, unmodified file. Note: The installed files are made read-only by default so as to avoid accidental modification.

Changing tracing levels

Because trace files are intended for debugging, by default, very little is logged to these files. You can change the default tracing level by editing the Trace.properties or ClientTrace.properties file. To view more logging, open one of the properties files in a text editor and uncomment (remove the “#”) lines such as the following (both from ClientTrace.properties):

# com.sqlstream.aspen.jdbc.level=FINE
# com.sqlstream.aspen.sdp.level=FINE

You can also change the tracing level for a particular package or class. For example, suppose you created a set of plugins that you want to debug at a high level of tracing. If the plugins are all part of a package called com.example.plugin, you can add the following to your Trace.properties:

com.example.plugin.level=FINEST

See also the topic Error Logging * in the Streaming SQL Reference Guide.

Limiting trace log size

The default tracing configurations are intentionally set for development mode, with no limit is placed on the maximum size of a trace file. Once you move to production, it is important to change these settings to prevent trace files from eventually using up all available disk space. There are two ways to limit trace files:

  • By placing a limit on the maximum size of any single trace file
  • By specifying that trace files should “roll over”. (This option is enabled by default.) “Roll over” means that the first time a trace file is full, it will be saved with a “1” appended to the filename, and a new trace file will be started. Subsequent rollovers will be given a “.2” extension instead, and so on up to the maximum trace file count defined in the configuration file (set to 20 by default). Once that limit has been reached, the oldest trace file is deleted each time a new one is created.

To control the maximum file size and rollover count, change the following properties in the file /var/log/sqlstream/Trace.properties:

race.log Property Name Default
File size java.util.logging.FileHandler.limit 10000000 bytes
Rollover count java.util.logging.FileHandler.count 20
ApplicationTrace.log Property Name Default
File size com.sqlstream.aspen.trace.ApplicationLogConfigurator.limit 10000000 bytes
Rollover count jcom.sqlstream.aspen.trace.ApplicationLogConfigurator.count 20

For SQLstream s-Studio, use the following properties in /var/log/sqlstream/ClientTrace.properties:

ClientTrace.log Property Name Default
File size java.util.logging.FileHandler.limit 10000000 bytes
Rollover count java.util.logging.FileHandler.count 20

If you set file size limit to zero, the file will never roll over, even if a rollover count is set.

Using different configuration files

You can override which logging configuration file SQLstream uses by overriding the value for the property java.util.logging.config.file. To do so, set the property in the file

$SQLSTREAM_HOME/aspen.custom.properties.

To change the user interface logging configuration file, set the property in

$HOME/.SQLstreamrc.

For example, to use the file /usr/local/MyTrace.properties for user interface logging configuration, add the following line to the .SQLstreamrc file in your home directory:

java.util.logging.config.file=/usr/local/MyTrace.properties

Periodic Parser Statistics Logging

The trace log contains information on how parsers are reading data from external sources such as log files, Kafka, Amazon Kinesis, and AMQP. This is known as Periodic Parser Statistics Logging. These sources are read through the Extensible Common Data Adapter. For more information on parsing data from external sources, see the topic Reading Data from Other Sources in the Integrating Guavus SQLstream with Other Systems guide.

Version 6.0 Feature This feature is new in s-Server 6.0.

You can control how often s-Server logs parser statistics with the following entry in Trace.properties:

com.sqlstream.aspen.namespace.common.input.logPeriod=10

This value determines how often in minutes that s-Server logs parser statistics. By default it is set to every 10 minutes.

For each Extensible Common Data reader open, the trace log lists information on the following:

Statistic Description
Throughput Bytes per second of input processed over the previous period.
Runnable Percent of time that the parser was running (or runnable but waiting for an available core to run in).
Blocked waiting for input Percentage of time that the parser was not runnable because it was waiting for more input to process.
Blocked waiting for downstream writes Percentage of time that the parser was not runnable because of backpressure from downstream processing.

Statisticcs on runnable and blocked parsers are approximated based on sampling log parser statistics. You can control sampling and reporting by changing the following parameters in Trace.properties:

Parameter Definition
com.sqlstream.aspen.namespace.common.input.logPeriod How often in minutes to log parser statistics. Defaults to 10. Set to 0 to disable logging.
com.sqlstream.aspen.namespace.common.input.probeFrequency How may times per logPeriod should parser statistics be sampled. Defaults to 100.

Example from Trace Log

```INFO: Parser for LOCALDB.TEST_READER.P_Pump: throughput=12598532bps, runnable 20%, blocked waiting for input 80%, blocked waiting to write downstream 0%

```