AMI Realtime Database

Goals

The 3Forge Realtime database builds on all of the features available in the AMI SQL language. The real-time datamodeling

• High Speed - 100k+ inserts per second, with 1 billion+ rows
• Persistent - Tables can be configured for long-term persistence
• Data Virtualization - Access external sources seamlessly
• Realtime Streaming - Chaining streams of updates into aggregations and other projections
• Source Synchronizing - Synchronizing in-memory tables based on external datasources
• Extendable - Create custom triggers, procedures, etc.

Broadly speaking, the logic steps involved in data-visualization are:

1. Data is "streamed" into the AMI real-time database through a combination of methods
   • High Speed Messaging
   • JDBC
   • Synchronizing from External Datasources
   • User updates, from the front end visualization layer
2. Data is fed through logical components which can aggregate, etc.
3. Visualizations are built on the real-time tables and streams.

About Realtime (PUBLIC) tables

For the AMI Center, by default tables that are created are only visible to the current session and will be cleaned up when the connection is terminated.  When a table is created with the PUBLIC directive (ex CREATE PUBLIC TABLE ...) the table will now be accessible to all sessions (external connections, triggers, etc) and front ends for real-time viewing. Public tables will continue to exist until they are explicitly dropped.  Please see the reserved columns on public tables section for details on specialized columns that can be added.

Realtime Tables

create_public_table_clause

Overview

This command is used to create a new, empty (no rows) public table with a specified set of columns (names, types, no-null restriction).  If another public table already exists with the tblname, the create table will fail.

At least one column must be specified, and each column must have a unique name. Columns with the nonull constraint cannot hold a null value. Updates and inserts in violation of the rule will fail.

The Following Use Options can be applied to a table:

• PersistEngine -
  • "FAST": The table will be persisted to disk using a fast, binary protocol. Additional options include:
    • "persist_dir" = "/path/to/directory/for/storing/file" - this is the directory where the .fast binary file will be saved/loaded from
    • "persist_compact_percent" = ".95" - a number between 0 and 1 which says how much of the file should contain updates & deletes before it's compacted. Because the FAST table uses a journaling strategy, updates and deletes will get appended and can create large files. Setting a value of .95 would indicate that if the file becomes 95% updates and deletes then the file should be rebuilt.