DBToaster generates lightweight, specialized, embeddable query engines for applications that require real-time, low-latency data processing and monitoring capabilities. The DBToaster compiler accepts queries written in SQL and generates code that can be easily incorporated into any C++ or JVM-based (Java, Scala, ...) project.
DBToaster is not a classical database system
DBToaster-generated engines are optimized for long-lived queries, where query results must be kept up-to-date with rapidly changing input data. Using database terminology, DBToaster engines maintain in-memory materialized views.
Applications include:
- Algorithmic Trading
- Real-time Data Warehousing
- Network/Cluster Monitoring
- Clickstream Analysis
- and more...
Check out DBToaster if you need to
- maintain materialized views of complex SQL queries,
- care about very high view refresh rates / low refresh latencies, and
- work with standing (aka continuous) rather than ad-hoc queries, i.e. you want to monitor the changing result of a given query over time, as the data changes.
DBToaster may be also right for you even if you do not care about low view refresh latencies: DBToaster turns a set of queries into efficient specialized code for processing just these queries. DBToaster generates code that you can link into your applications. No further software (such as a separate database server or CEP engine) is required. Thus DBToaster is a very lightweight way of embedding (parameterized) SQL queries into your applications.
Here is a white paper that may help you decide whether to try out DBToaster.
The DBToaster at EcoCloud 2013 presentation gives a high-level overview of the DBToaster project.
DBToaster code is fast
Traditional relational databases are slow because they are designed to support arbitrary hand-written queries. Nowadays though, few people execute queries directly. Most queries are generated automatically based on templates. DBToaster custom-tailors each engine it creates to the needs of a specific application. This engine supports only query processing functionality that the application requires, avoiding the overhead of supporting unnecessary features.
DBToaster employs incrementality to efficiently maintain query results in real-time as data changes. As a consequence, DBToaster-generated engines provide extremely low-latency access to query results, and efficiently support monitoring of result values.
DBToaster-generated code is typically by 3-6 orders of magnitude faster than existing state of the art data-management systems when we measure the time it takes to refresh a view given an update to the base data.