Installing s-Server on Linux

This page contains the following topics:

Installation Overview

s-Server needs to be installed on a Linux machine. See requirements for details on the specifications for this machine. The s-Server installation package includes s-Studio, s-Dashboard, client tools, and WebAgent.

Both s-Studio and client tools can be installed separately using their own installers. See Installing SQLstream s-Studio as a Standalone Package and Installing SQLstream Client Tools as a Standalone Package in this guide for more details.

StreamLab must be installed separately. See Installing SQLstream StreamLab in this guide for more details.

In a development environment, you may wish to install s-Server, s-Dashboard, WebAgent, and s-Studio on this same Linux machine. However, in production environments, you will often want to distribute these components.

Installation Requirements

SQLstream s-Studio and s-Server requirements (hardware and software components) are as follows:

Component Specifics (size, version)
Linux version s-Server requires a 64-bit x86 architecture version 2.6 or later Linux kernel distribution with Libc version 2.12 or later. Other Linux kernel versions are not supported.

Tested Linux kernel distributions include: CentOS 7.x, 8.x; Redhat Enterprise Linux 7.x, 8.x, Ubuntu 18.04 LTS (Bionic Beaver) and 20.04 LTS (Focal Fossa).

s-Server is designed to run on any LSB compliant Linux, but specific distributions may or may not work
CPU A minimum of 2 CPU cores (64 bit, x86 architecture) and 4GB RAM are required to run s-Server. A server class processor such as an Intel Xeon is preferred. Since SQLstream s-Server includes a multithreaded scheduler, multiple cores can improve data processing performance.

Larger caches, faster CPUs, and faster memory interconnect all improve data processing performance.
Disk space Installed software requires 2GB. Log files, repository, and so on require an additional 1 GB.

The amount actually used will depend on installation options selected, how you configure logging, how many objects you create, and so on.

2GB is probably a safe minimum for an initial install.

See SQLstream s-Server Installation Directories in this guide's appendix for more information on installation directories.
Memory Minimum 3GB, Production 4GB, Production with more streams or longer data windows 5GB.

Memory requirements will be application dependent: more streams or streams that hold longer data windows will require more RAM. When running under virtual machine technology, 3GB physical RAM is required on the guest system.
Java JDK 8 is required. The installer will check for an appropriate version of an installed JDK. If it finds an installed JDK, it asks you to confirm its location. If it does not find an appropriate version, it will try to use the native package installer (apt-get on Debian/Ubuntu and yum on Redhat/CentOS) to download and install OpenJDK 8.
s-Studio When installed alongside the server, has the same Linux and JDK requirements.

2G RAM. When running under virtual machine technology, 2G physical RAM is required on the guest system.
s-Dashboard Browser: Microsoft Edge, Internet Explorer 11 (or later), Google Chrome 31 (or later), Mozilla Firefox 17 (or later), or Apple Safari 7 (or later).

1G RAM. When running under VMWare, 1G physical RAM is required on the guest system.

1G available disk space.

Installed Components

Component Description Disk Space
SQLstream s-Server Core component, installed by default N/A

Plugin Adapters

Plugin Description Disk Space
Core plugins Plugins that are essential for s-Server's running. (always installed) N/A
Cassandra NoSQL Database Allows s-Server to join streams against the Cassandra distributed database. 12 MB
Hadoop Distributed File System Adapter Allows s-Server to write streams to Hadoop and other systems using HDFS, including, as of version 6.0.1, Hive tables. Supports Kerberos authentication using keytabs. (Installed by default.) 40 MB
Amazon Kinesis Data Streams Adapter Allows s-Server to read and write AWS Kinesis streams. 13 MB
Snowflake Cloud Data Warehouse Adapter Allows s-Server to write streams to the Snowflake data warehouse. 19 MB

Other Components

Component Description Disk Space
SQLstream s-Studio Integrated Development Environment for building, debugging, and monitoring streaming applications. 137 MB
SQLstream s-Dashboard Browser-based data visualization too.l 105 MB
Demonstration Apps and Examples Sample data sets, for transportation, Internet of Things, and social media. 3 MB

SQLstream Client Tools

Client Tool Description Disk Space
SQLline SQLLine is a command-line tool for communicating with the SQLstream s-Server. It is a generic & open-source database client that works with any jdbc database. More information about SQLLine can be found at http://sqlline.sourceforge.net/ 11 KB
datagen Datagen generates test data for SQLstream applications, and pushes the data into SQLstream using the SQLstream JDBC Driver. 1 MB
jdbc JDBC (Java DataBase Connectivity) is a standard Java API for connecting to relational databases and other data sources that can produce relational data. 2 MB
ECD Agent The ECD Agent is a standalone Java application that mediates data between - SQLstream s-Server and external data sources. It can be installed on a remote server in order to process or parse data at the site of collection. For example, if you have a server which generates log files, you may find it advantageous to have the ECDA Agent parse those files on the server itself, and pass rows to s-Server, as opposed to passing the entire file. 60 MB
WebAgent WebAgent functions as a simple, miniature Web server. It accepts HTTP requests, sends these to s-Server, and returns JSON objects. See json.org for more details on JSON objects. 11 MB
Google Big Query Connector SQLstream Google Big Query Connector is a bulk load script that lets you load s-Server data into Google Big Query. 17 MB

Preparing to Install s-Server on Linux

To prepare for installation, follow these steps:

  1. Uninstall any previous versions of SQLstream s-Server (see Uninstalling s-Server below).

    Note: In some cases, you may have modified aspen.custom.properties in your old installation of s-Server. You will need to move this file from the old installation directory to the new installation directory before you uninstall the old version. Some users may have also modified bin/defineAspenRuntime.sh and need to move this file as well.

  2. Identify the directories where s-Server will be installed. By default, if you are installing as sudo (see below) s-Server installs into the /opt/ directory. Most users can accept the default options. See the topic Appendix: SQLstream s-Server Installation Directories in this guide for more detail.

Installing as Sudo/Root vs. Installing as a Regular User

For most production environments, you will want to install s-Server as sudo, as doing so lets you run s-Server run as a service called s-serverd. With this option, the system logs will also be written under /var/log.

In some cases, you might wish to install as a regular user, particularly if

  • You are a developer, and you know that only you are going to use the server. Usually, you would do so as a trial. In this case, you would install s-Server in your own user directory and have access to all the files yourself.
  • You need to install multiple servers on the same computer and you want to manage them under separate users. (This is not a common use case, and it would be best to consult SQLstream technical support if you are doing so.)

Installing and Running Client Tools and s-Studio

You can install and run client tools and s-Studio as a regular user. These connect to s-Server through JDBC.

Using a Linux Server with a Windows Machine

s-Server requires Linux to run. s-Studio can be located on either a Windows or Linux machine as long as it can connect to s-Server via JDBC.

StreamLab and s-Dashboard are client-side technologies which communicate with s-Server via WebAgent. StreamLab and s-Dashboard can all be run on the same machine as your Web Server.

WebAgent streams data from s-Server to s-Dashboard or other browser-based clients. Because WebAgent consumes resources, you may want to install it on one or more separate servers for load balancing. Because it communicates with s-Server via JDBC and s-Dashboard via HTTP, you can locate WebAgent anywhere as long as it can connect to s-Server via JDBC. When installing as sudo, you can install and run WebAgent as a service, webagentd. When installed as sudo, webagentd runs as the sqlstream user, but you can also run WebAgent as a regular user process (see running WebAgent.

StreamLab lets you view and manipulate streams through a browser.

Installing

The easiest way to launch the installer is by initiating the installation file from the command line. You can also install in text mode if you do not have or do not wish to use X-windows.

Proceed with the installation by performing the following steps:

  • Give the installation file permission to run by entering chmod +x sServer--x64.run.
  • From a terminal prompt enter sudo ./SQLstream-sServer-X.Y.Z-x64.run, where X.Y.Z represent the release and version numbers, as in SQLstream-, and x64 or x32 represents a 64-bit or 32-bit operating system.

Note: You can run the installer as a normal user on most Linux systems for demonstration purposes, but if you are running a production environment, you will likely want to install as sudo. Installing as sudo lets you run the sserver.d service and install into the recommended /opt location. See - Managing s-Server as a Service for more details.

If the installer does not launch, see the topic - Troubleshooting Installation for suggestions. A Welcome page appears, introducing the Setup Wizard.

  1. Click Forward. The SQLstream license text appears in a new dialog box.
  2. Click the radio button next to I Accept the Agreement and then click Forward. The Select Components page appears. If desired, select or deselect components for installation. See Installed Components below.
  3. Click Forward. The Registration Information Page opens. SQLstream requires you to enter a valid email address in order to verify your registration.
  4. Enter a valid email address and click Forward. The Installation Directories page appears. This page lets you select directories for the installation itself, the s-Server object database, and the s-Server's trace log files.

    Note: the installation user must have permission for these directories.To change directories, click the browse button to the right of the directory field. The installer defaults to /opt/ if you are installing as root (RedHat/CentOS) or using sudo (Ubuntu/Debian).

    Note: If the installation directory contains a previous installation of s-Server, the installer will ask you if you want to overwrite the directory. If you choose to do so, all customizations and catalog object definitions for the previous installation will be lost.

  5. Click Forward. The Server Port Number page appears. By default, port is set to 5570. We recommend leaving it set to this port unless you have a reason to not do so.

  6. Click Forward. The Java Development Kit (JDK) Home Directory page appears. JDK Home Directory should be filled in with your current JDK directory.

  7. Click Forward. If you are installing as root or using sudo, the Service/Daemon page appears. This page lets you specify whether s-Server should start when this computer starts. Choose Yes to have the installer create a service so that s-Server starts automatically when the system boots. This service can automatically start when the installer finishes.

  8. Deselect as desired, and click Forward. The WebAgent Service/Daemon page appears. The SQLstream WebAgent allows s-Server to communicate with a browser. If you are going to run any browser-based applications such as s-Dashboard, StreamLab, or the Mochi demonstration application, you need to run WebAgent.

  9. Change options if desired and click Forward. The Create Dashboard Service page opens. The Dashboard service lets you create and view visualizations with s-Dashboard. For more information, see The s-Dashboard Guide.

  10. Change options if desired and click Forward. The Server User page appears. This page requires that you specify a user for the installation. This user must have permissions for the installation directories specified earlier.

  11. Enter a user with such permissions and click forward.

  12. Select I accept the agreement and click Forward. The Oracle JDBC Driver License page opens. Here, you need to accept the license for this driver. Installation will not proceed if you do not accept this license.

  13. Select I accept the agreement and click Forward. The Teradata JDBC Driver License page opens. Here, you need to accept the license for this driver. Installation will not proceed if you do not accept this license.

  14. Select I accept the agreement and click Forward. The Ready to Install page opens.

  15. Click Forward to begin the installation process, which then displays a progress bar to show you how far along installation has progressed.

Installation is complete when the progress bar reaches 100%. You will see a confirmation dialog box.

Click Finish to exit the installer and/or view the Readme file.

The first time you run the server, it creates a checkpoint. Before you run the server for the first time, you cannot restore the catalog because no checkpoint exists.

Third-party JDBC Drivers

The SQL/MED plugin lets SQLstream s-Server interface with third-party databases such as Oracle, Microsoft SQL Server, PostgreSQL, MySQL, and Teradata. The s-Server installer includes JDBC drivers for Oracle, Microsoft SQL Server, Teradata, and PostgreSQL SQL. Those drivers require no additional steps to use. In order to use other JDBC drivers with SQLstream s-Server, you need to copy all required jar files to the $SQLSTREAM_HOME/corelib directory and restart s-Server.

  • Microsoft SQL Server JDBC Driver
  • Oracle JDBC Driver
  • Teradata JDBC Driver
  • PostgreSQL JDBC Driver

Installing the MySQL JDBC Driver

To use the MySQL JDBC driver, download the driver package from http://dev.mysql.com/downloads/connector/j/ Next, copy the file *mysql-connector-java.jar *to $SQLSTREAM_HOME/corelib and restart s-Server.

Note: s-Server is currently defined to use mysql-connector-java-3.1.14-bin.jar. If you are using a version of mysql-connector other than this one, you will need to edit a script in $SQLSTREAM_HOME/bin/ called defineAspenRuntime.sh. To do so, open this script in a text editor and search for "mysql". The line you need to change is as follows:

MYSQL_DRIVER=$EIGEN_HOME/thirdparty/mysql-connector/mysql-connector-java-3.1.14-bin.jar

Note: Some Linux distributions include the MySQL JDBC driver as part of their installation package. If this is the case for your Linux distribution, then copy the file /usr/share/java/mysql-connector-java.jar to $SQLSTREAM_HOME/corelib and restart s-Server.##

Confirming s-Server Is Running

To confirm that SQLstream s-Server is running, do one of the following:

  • Check if s-Server appears in SQLstream s-Studio.
  • Use jps -l to confirm that "AspenVJdbcServer" is running. (This is the actual name of s-Server.)

Start SQLstream s-Studio

Once the SQLstream s-Server is started, you can open SQLstream s-Studio by double-clicking its desktop icon or by running the SQLstream script from any directory. When you launch s-Studio, you will need to enter connection information for your installation of s-Server. If the server is running, s-Studio should connect to it and its icon will appear in the left pane of s-Studio:

Use jps -l

SQLstream s-Server runs as a Java Virtual Machine called  AspenVJdbcServer.

To confirm that SQLstream s-Server is running, you can use the Java jps -l command. (See https://docs.oracle.com/javase/8/docs/technotes/tools/unix/jps.html for more detail on this command.) Since s-Server runs as the sqlstream user, you will need to run jps-l it as sudo in order to see s-Server running. This command will produce something like the following if s-Server is running:

6301 sun.tools.jps.Jps
2560 com.sqlstream.aspen.server.AspenVJdbcServer

Installation Logs

While install is running, the install log is in tmp/bitrock_installer.log. After install completes, the install log is in the installation directory as install.log.

System Logs

Provided the server being installed on uses rsyslog (most Linux installations do) and if the server provides a directory for add-on syslog configurations the installer will place a sqlstream.conf file in /etc/rsyslog.d, and reload the rsyslog service as part of the install.

This will produce a timestamped console.log in /var/log/sqlstream/console.log in addition to the normal syslog entries in the messages file and any daemon trace logs defined on the system. You can edit /etc/rsyslog.d/sqlstream.conf to point the console output to a different file if desired.

Uninstalling SQLstream s-Server on Linux

When to Uninstall

You should always uninstall SQLstream before installing a newer version or patch.

How to Uninstall s-Server

To uninstall s-Server, either log in as root (CentOS/Redhat) or use sudo (Debian/Ubuntu), navigate to the directory in which you installed s-Server and enter the following command:

./uninstall

Note: You can also uninstall in text mode by appending --mode text to the above command. s-Server asks for your verification.

When you click Yes, all s-Server components are uninstalled.

It is then possible to install your new or upgraded version of SQLstream s-Server.

The processes for uninstalling s-Studio or client tools in a standalone installation are different. See the topics Uninstalling s-Studio in a standalone installation and Uninstalling clienttools in a standalone installation in this guide for more details.

Installing s-Server from the Command Line

You can also install s-Server from the command line, which does not display the graphical installation wizard. To do so, enter

sudo ./SQLstream-sServer-X.Y.Z-x64.run --mode text

where X.Y.Z represent the release and version numbers, as in SQLstream-<version>, and x64 or x32 represents running on a 64-bit or 32-bit operating system.

Note: substitute sStudio or clienttools to install either as a standalone in text mode.

if you are logged in as a super user, you do not need to include "sudo".

The text-mode installer performs the same steps as the graphical one, but displays its prompts and messages in the console window.

By default, the installer will run in graphical mode, GTK. If this is not possible, it will run in xwindow mode.

If there is no graphical environment available it will fall back to text mode.

Viewing Options for Command-line Installation

There are command-line options to perform nearly all of the functions available in the GUI version. For a list of the command-line options and their meanings, type

./<installername> --help

If the installation directory contains a previous installation of s-Server, the installer will ask you if you want to overwrite the directory. If you choose to do so, all customizations and catalog object definitions for the previous installation will be lost. You can choose to override this warning by appending --overwrite to the installer when you launch it.

Installing in Unattended Mode

You can also install in --mode unattended. This mode installs with all default values. There are two required parameters: --email and --companyName. To run in unattended mode, enter a command along the following lines:

sudo ./SQLstream-sServer-<version>
.0-x64.run --mode unattended --email user@example.com --company Example

If the company name contains a space, you need to put it in quotation marks:

--company "SQLstream Inc."

You can also combine unattended mode with other options to provide a customized install. When you install in unattended mode, you are implicitly agreeing to the license agreement. The most likely options for one to use for an unattended server install would be the last ones listed by --help:

Option Description
‑‑email Registration email.
‑‑company <company name> Company Name. Place in quotes if it contains spaces
‑‑mode <mode> Installation mode.

Allowed: gtk xwindow text unattended

Use unattended for command line install where no further questions will be asked.
‑‑prefix <prefix> Installation Directory. Defaults to /opt/sqlstream/<VERSION>
‑‑catalogdir <dir> Catalog Directory. Defaults to /opt/sqlstream/<VERSION>/catalog
‑‑createService <0 or 1> Whether or not a service, s- serverd, is created.
‑‑overwrite Asks installer to overwrite any existing installation.
‑‑enable‑components <components> Lets you specify components to be installed by entering a comma-separated list of components.

Defaults to: Server, Plugins, corePlugins, Studio, Demo, clienttools, JDBC, SQLLine, datagen, ecdaagent, webagent, thirdparty, MSJDBC, OJDBC, TeradataJDBC, PostgresJDBC.

Allowed Options:
Server, cassandra, hdfs, kinesis, snowflake, Studio, Demo, clienttools, SQLLine, datagen, ecdaagent, webagent, gbqconnector, thirdparty, MSJDBC, OJDBC, TeradataJDBC, PostgresJDBC
‑‑disable‑components <components> Lets you specify components to not install by entering a comma-separated list of components.

Defaults to: cassandra, hdfs, kinesis, snowflake, gbqconnector.

Allowed Options: Server, cassandra, hdfs, kinesis, snowflake, Studio, Demo, clienttools, SQLLine, datagen, ecdaagent, webagent, gbqconnector, thirdparty, MSJDBC, OJDBC, TeradataJDBC, PostgresJDBC

(These options correspond to the options in the GUI.)

Options for s-Studio Command-line Installation

Option Description
‑‑optionfile <optionfile> Installation option file
‑‑mode <mode> Installation mode. Defaults to gtk. Allowed: gtk xwindow text unattended
‑‑installer‑language <language> Language selection. Defaults to en. Allowed: sq ar es_AR pt_BR bg ca hr cs da nl en et fi fr de el he hu it ja ko lv lt no pl pt ro ru sr zh_CN sk sl es sv th zh_TW tr va cy
‑‑prefix <prefix> Installation Directory. Defaults to /opt/sqlstream/<VERSION> (depends on whether installer user is super user or not).
‑‑defaultServer <server> Name of the server on which s-Server is installed

Options for Client Tools Command-line Installation

Option Description
‑‑optionfile <optionfile> Installation option file
‑‑mode <mode> Installation mode. Allowed: gtk xwindow text unattended
‑‑enable‑components <components> Comma-separated list of components. Defaults to All. Allowed: SQLLine datagen ecdaagent logfileagent filewriteragent socketagent webagent gbqconnector
‑‑disable‑components <components> Comma-separated list of components. Defaults to none. Allowed: SQLLine datagen ecdaagent logfileagent filewriteragent socketagent webagent gbqconnector
‑‑installer‑language <language> Language selection. Defaults to en. Allowed: sq ar es_AR pt_BR bg ca hr cs da nl en et fi fr de el he hu it ja ko lv lt no pl pt ro ru sr zh_CN sk sl es sv th zh_TW tr va cy
‑‑prefix <prefix> Installation Directory. Defaults to /opt/sqlstream/<VERSION> (depends on whether installer user is super user or not).
‑‑defaultServer <server> Name of the server on which s-Server is installed. Required.