DBToaster Adaptors Reference
DBToaster adaptors transform events from external sources into an appropriate format and deliver them to the query engine. The current release supports two file adaptors in both the C++ and Scala backends:
- CSV - for reading in string-delimited input files
- Order Book - for reading in stock market historical data
1. CSVA simple string-delimited adaptor. Fields are separated using the delimiter passed in the delimiter parameter. If not provided, comma (",") will be used as a default delimiter.
The optional deletions parameter can be used to generate a single stream of both insertions and deletions. When set to "true", the input source is assumed to have an extra, leading column. When the value in this column is 0, the record is treated as a deletion. When the value is 1, the record is treated as an insertion.
Fields are parsed based on the type of the corresponding column in the relation. Ints, floats, and strings are parsed as-is. Dates are expected to be formatted in the SQL-standard [yyyy]-[mm]-[dd] format.
CREATE STREAM R(A int, B int) FROM FILE 'r.dat' LINE DELIMITED CSV (delimiter := '|');
2. Order BookAn adaptor that allows reading in stock trade historical data. It assumes that all the input records obey the following predefined schema: <timestamp : float, message_id : int, action_id : char, volume : float, price : float>. Insertions and deletions are triggered for each record as follows:
- If action_id is 'b', and the orderbook adaptor was instantiated with the parameter book := 'bids', an insertion will be generated for the record.
- If action_id is 'a', and the orderbook adaptor was instantiated with the parameter book := 'asks', an insertion will be generated for the record.
- If action_id is 'd', and the orderbook had previously inserted a record with the same message_id, a deletion will be generated for the record.
CREATE STREAM bids(T float, ID int, BROKER_ID int, VOLUME float, PRICE float) FROM FILE 'history.csv' LINE DELIMITED orderbook (book := 'bids', brokers := '10', deterministic := 'yes');
|yes||A string delimiter used to extract fields from a record. If not specified, the default value is ','.|
|yes||If set to "true", use the first field of the input file to distinguish between rows for insertion and rows for deletion. A 0 in the first column triggers a deletion event. A 1 in the first column triggers an inertion event. The first column is stripped off of the record before further parsing is performed.|
|no||The value of this parameter may be 'bids' or 'asks', and determines for which orderbook events will be generated.|
|yes||The number of brokers to simulate. By default, 10 brokers will be used.|
|yes||If the value of this parameter is 'yes', broker ids will be generated deterministically based on the message id. By default, broker ids will be generated randomly using the rand() system call or equivalent.|