Intel Bolsters Hadoop Distribution with Lustre Support

Print Friendly, PDF & Email

Over at The Register, Simon Sharwood writes that Intel has added Lustre file system support to its new Hadoop 2.5 distribution.

Many customers do not want to deploy an entirely separate physical cluster just because we couldn’t figure out how to run Hadoop on their file system,” said Girish Juneja, Intel’s general manager for big data and software services. We abstracted out an HDFS layer but underneath that it is actually talking to Lustre. So if you look at Los Alamos lab and these research labs with humongous clusters that run HPC jobs 90 per cent of the time, but for ten per cent of the time they want to run a Hadoop job they can run it in exactly the same environment without moving data.

Read the Full Story.