Introducing Trafodion

Architecture.jpg

 

On Tuesday, June 10, 2014 at HP Discover, Hewlett-Packard announced that the Trafodion project (http://www.trafodion.org) is now available as open source. Trafodion is an enterprise-class SQL-on-HBase solution targeting big data transactional or operational workloads. There have been many SQL engines or SQL subsets developed for the Apache Hadoop space, but most of those engines are focused on analytics. Trafodion is meeting a different need: The need for transactional and operational management of data. Using Trafodion with Apache HBase, it is now possible to support OLTP applications on Hadoop. As one of the architects of Trafodion, I’d like to give you a high-level overview of Trafodion’s features.

 

At a glance, Trafodion is a transactional ANSI SQL engine on top of Apache HBase. It has performance improvements for OLTP workloads. It also supports large data sets through parallelism. It offers JDBC and ODBC connectivity for Linux and Windows clients. We’ll go through some of these features in more detail.

 

 Transactions: Trafodion features ACID distributed transactions. This is a key feature for supporting OLTP. A transaction might involve multiple rows, multiple tables and/or multiple statements. Trafodion uses the Multi-Version Concurrency Control model. Instead of locking, Trafodion checks for conflicts between transactions at commit time. This is a good model when transaction conflicts are few. 

 

ANSI SQL: Trafodion supports the core feature set of ANSI SQL-99, with many extensions. Some noteworthy extensions include support for secondary indexes and cost-based optimization. You can create Trafodion tables within HBase. These tables are salted across regions on a set of columns you specify. You can also access native HBase tables directly. There is syntax for accessing individual HBase values. You can access multiple versions of values in one SQL statement.

 

Performance/parallelism: The Trafodion engine supports many optimizations, both compile-time and run-time. For OLTP, there are fast paths for single-row queries. For large data sets, the optimizer can choose a parallel plan. Trafodion’s optimizer searches a space of possible plans for one that is low cost. Determining cost depends in part on statistics about table data gathered with the UPDATE STATISTICS utility. The Trafodion optimizer also caches query plans, so frequently recurring queries need not be repeatedly optimized. Trafodion leverages the natural parallelism inherent in the multiple region servers in the HBase architecture. But it can go beyond that. Imagine, for example, that you are joining two tables, and the join is on a non-key column. Trafodion can generate a query plan using multiple execution servers, each of which will receive a dynamically generated partition of each table. The data are partitioned on a hash of the join column. Join methods supported include hash, nested loop and merge joins. Hash joins do not require all data to be in memory; the algorithm can spill intermediate results to disk files.

 

JDBC/ODBC: JDBC and ODBC drivers for Trafodion are available now for Linux and Windows clients. Connectivity via JDBC and ODBC is provided by Database Connectivity Services (DCS), which provides fault-tolerant access to the Trafodion cluster.

 

We welcome users and contributors to the Trafodion community. If you’d like to download the software, learn more about its architecture or contribute to its future development, visit our Trafodion web site.

Leave a Comment

We encourage you to share your comments on this post. Comments are moderated and will be reviewed
and posted as promptly as possible during regular business hours

To ensure your comment is published, be sure to follow the Community Guidelines.

Be sure to enter a unique name. You can't reuse a name that's already in use.
Be sure to enter a unique email address. You can't reuse an email address that's already in use.
Type the characters you see in the picture above.Type the words you hear.
Search
About the Author
For years I've been doing video and music production back and forth between Boston MA and New Orleans LA. Starting in 2010, I've began worki...


Follow Us
The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation