Today the Transaction Processing Performance Council (TPC) today announced the immediate availability of the TPCx-BB benchmark. The benchmark is designed to measure the performance of Hadoop based systems including MapReduce, Apache Hive, and Apache Spark Machine Learning Library (MLlib). The benchmark suite is publicly available for download as a self-contained kit via the TPC web site.
TPCx-BB executes frequently performed analytical queries in the context of retailers with physical and online store presence. Historically, online retailers recorded only completed transactions, whereas today they demand much deeper insight into their online consumer behavior. Simple shopping basket analysis techniques have been replaced by detailed behavior modeling; new forms of analysis have resulted in an explosion of Big Data Analytics Systems. Yet there have been no mechanisms to compare disparate solutions in a real-world scenario. TPCx-BB was specifically designed to meet this need, and provides concrete insight in terms of performance versus cost tradeoff.
“TPCx-BB marks a significant milestone for the TPC,” said Bhaskar Gowda, chairman of the TPCx-BB committee. “With the advent of so many Big Data and Analytics Systems – from an array of hardware and software vendors – there is immediate demand for apples-to-apples, cross-platform comparison. TPCx-BB was carefully crafted to meet this need.”
First Results
Hewlett Packard Enterprise published the following results soon after the benchmark became available:
- 9x HPE ProLiant DL 380 Gen 9 and 3x HPE ProLiant DL360 Gen9, 337.26 BBQpm@3000, $1,102.94 USD/BBQpm@3000, Availability Date: 03/31/2016
- 9x HPE ProLiant DL 380 Gen 9 and 3x HPE ProLiant DL360 Gen9, 265.93 BBQpm@3000, $1,222.72 USD/BBQpm@3000, Availability Date: 03/23/2016
The speediness of how fast the TPC stepped in to address the vacuum of an end user application benchmark – usable right from launch – really is remarkable,” continued Mr. Gowda. “Immediately after benchmark release, Hewlett Packard Enterprise published two results. This is a striking proof point as to industry demand for such a benchmark. It is also a testament to the talent of the members of the development committee.”
TPCx-BB is currently the TPC’s third Big Data benchmark. TPC-DS 2.0 was recently launched in December 2015 and measures the performance of SQL-based Big Data systems by expanding upon the original TPC-DS benchmark. And, in August 2014, the TPC introduced TPCx-HS, the industry’s first Express-class benchmark for benchmarking Big Data systems, designed to assess a broad range of system topologies and implementation methodologies.
