Difference between revisions of "AMI Historical Database"
(Created blank page) |
|||
| Line 1: | Line 1: | ||
| + | = Introduction to AMI Historical Database = | ||
| + | The AMI Historical Table is a columnar-based on-disk table engine with support for partitioning. | ||
| + | The introduction section will briefly describe the core concepts of the AMI Historical database: | ||
| + | Partitioning, Data storage and optimizations. | ||
| + | == Partitioning == | ||
| + | Partitioning is used to organize data for performance. To understand why partitioning is critical | ||
| + | for large datasets, let’s first consider a non-partitioned standard table: as the number or rows | ||
| + | increases, the performance (time to insert, update, query and alter) will decline. Partitioning | ||
| + | provides an elegant solution by breaking up rows into separate isolated “virtual tables” called | ||
| + | partitions. Once data is broken into partitions, the row count of one partition will not affect the | ||
| + | performance of other partitions. In order to dictate how rows are organized into partitions, the | ||
| + | AMI Historical Table introduces a special '''PARTITION''' column type. The value in the PARTITION | ||
| + | column(s) determines which partition a given row will be assigned to. In other words, all rows for | ||
| + | a given partition will have the same value in the PARTITION column(s).<br> | ||
| + | By way of example, data is often partitioned by a date: Imagine we have the following historical | ||
| + | table, partitioned by the column “date”<br> | ||
Revision as of 16:34, 26 May 2023
Introduction to AMI Historical Database
The AMI Historical Table is a columnar-based on-disk table engine with support for partitioning. The introduction section will briefly describe the core concepts of the AMI Historical database: Partitioning, Data storage and optimizations.
Partitioning
Partitioning is used to organize data for performance. To understand why partitioning is critical
for large datasets, let’s first consider a non-partitioned standard table: as the number or rows
increases, the performance (time to insert, update, query and alter) will decline. Partitioning
provides an elegant solution by breaking up rows into separate isolated “virtual tables” called
partitions. Once data is broken into partitions, the row count of one partition will not affect the
performance of other partitions. In order to dictate how rows are organized into partitions, the
AMI Historical Table introduces a special PARTITION column type. The value in the PARTITION
column(s) determines which partition a given row will be assigned to. In other words, all rows for
a given partition will have the same value in the PARTITION column(s).
By way of example, data is often partitioned by a date: Imagine we have the following historical
table, partitioned by the column “date”