Dell & Terascala Turnkey Lustre Performance

Print Friendly, PDF & Email

dellFor those of you following the saga surrounding the Lustre file system and its respective constituents, you might want to check out a recent blog post on the Dell HPC site.  The post is written by Scott Collier of Dell’s HPC lab crew on recent tests he performed on the new Dell/Terascala turnkey Lustre solution.  I really enjoy it with vendors and users post performance numbers, especially when they back it up with very specific details on the configuration.  Kudos to Collier for laying it out nice and clean for the audience.

Collier outlines the technical characteristics of the setup as follows:

Compute Node Hardware & Software:

  • Platform OFED 1.4.2
  • RHEL 5.3: 2.6.18-128.7.1.el5
  • Lustre 1.8.2
  • Mellanox ConnectX-{1,2} QDR HCA’s
  • 64 Dell R410’s for compute nodes w/ 24GB of memory
  • 1 Dell R710 as frontend node
  • 2 Dell PC6248 Ethernet leaf switches
  • 1 Dell PC8024F 10GB/s Ethernet core switch
  • 1 Qlogic 12800 QDR IB switch
  • 1 Qlogic 9024 DDR IB switch (uplinked to QDR switch via 5 QDR-DDR cables)

OSS/MDS Hardware & Software

  • Lustre
  • Mellanox Infinihost III DDR
  • Full remote management
  • 2 Terascala MDS servers (connected to 1 MD3000); 8GB of memory per server, connected via DDR IB fabric
  • 2 Terascala OSS servers (connected to 2 MD3000); 8GB of memory per server, connected via DDR IB fabric

Scott goes on to cover the first series of IOzone benchmarks they ran in distributed mode in order to test “N-to-N” sequential reads and writes.  The configuration hit its peak sequential write performance with 8 nodes at ~1300 MB/s.  Block scaling was reasonably flat for write operations.

On sequential reads, the peak performance hit ~1400 MB/s with 4 and 12 nodes active.  Performance variations of up to 200 MB/s occurred between block sizes and as the number of nodes increased.  This is potentially due to head seeks, induced due to the randomness of requests and variance of block sizes [although Scott does not comment on why this occurs].

All in all, this looks to be a solid test from the Dell HPC labs.  Scott plans on releasing more test results in future blog posts.  For now, check out his first set of tests here.


  1. […] folks at Dell were kind enough to point us to their second installment of the Dell & Terascala technical performance review.  In part two of the series, Scott Collier outlines their recent tests of the Lustre metadata […]


  1. Part 2 of this blog series from Scott Collier about the Dell | Terascala Storage Solution has now been posted covering:
    1. GUI usage / functionality
    2. Metadata performance
    3. Links to additional resources

    Link to the blog: