The book benefits software engineers, scientific researchers, and high performance and supercomputing developers in need of high-performance computing resources, by:

• Providing a guide to exploiting the parallel power of the Intel Xeon Phi coprocessor for high-performance computing
• Presenting best practices for portable, high-performance computing and a familiar and proven threaded, scalar-vector programming model
• Including simple but informative code examples that explain the unique aspects of this new highly parallel and high performance computational product
• Covering wide vectors, many cores, many threads and high bandwidth cache/memory architecture

I got my hands on a preliminary copy of the book back in November at SC12, and I can tell you that Jim and James did a great job.

The book release coincides with the formal dedication of the Stampede supercomputer at the Texas Advanced Computing Center in Austin. Stampede is currently ranked number seven on TOP500, with over 6400 Intel Xeon Phi coprocessors. Jeffers and Reinders have today committed several hundred books to support TACC’s training efforts for Stampede.

Read the Full Story.

Related posts:

• Colfax International Offers Developer Training for Intel Xeon Phi Coprocessor
• James Reinders on Exploiting Parallelism in the new Intel Xeon Phi
• Video: Intel Xeon Phi Powers #7 TACC Stampede Super

The post New Book Offers Insight into Coding for Intel Xeon Phi appeared first on insideHPC.

Big data is more than hadoop, map reduce, SAS or other programmatic and analytical focused tool, solution or platform, yet those all have been and will be significant focus areas in the future. This also means big data is more than data warehouse, data mart, data mining, social media and event or activity log processing which also are main parts have continued roles going forward. Just as there are large MByte, GByte or TByte sized files or objects, there are also millions and billions of smaller files, objects or pieces of information that are part of the big data universe.

Read the Full Story or check out this interview with author Rick Smolan.

Related posts:

• Book Review Provides Sneak Peek at SC10 Keynote
• Book review: Principles of Parallel Programming
• New Book on Data-Intensive Computing

The post Book Review: The Human Face of Big Data appeared first on insideHPC.

I think Eijkhout does a great job of covering the bases for computational scientists. I have a copy at home and I keep it handy as a reference.

The book also features tutorials on LaTeX, Unix, Compilers and libraries, Managing code with Make, Source control, and Programming practices. Course slides are also available for download.

Related posts:

• Free eBook: Introduction to High-Performance Scientific Computing
• Download: Introduction to High Performance Scientific Computing
• Free eBook: The Future of Computing Performance: Game Over or Next Level?

The post Fresh Ink – Introduction to High Performance Scientific Computing appeared first on insideHPC.

The book by Rob Faber on CUDA Application Design and Development is required reading for anyone who wants to understand and efficiently program CUDA for scientific and visual programming. It provides a hands-on exposure to the details in a readable and easy to understand form. — Jack Dongarra, Innovative Computing Laboratory, EECS Department, University of Tennessee

http://www.youtube.com/watch?v=mjC8mU9t-qQ

This is a “home-movie” demonstration of the chapter 12 real-time video example from the book, “CUDA Application Design and Development”. Full source code is included in the book, and you can use this framework to create your own “kinect” like video controller, start with augmented reality, and add video feedback to your games.

Farber demonstrated this code running on a battery powered GPU at Supercomputing 2011 in the NVIDIA booth. Download the slides (PDF).

Related posts:

• Rob Farber on Intel Xeon Phi Programming for CUDA Developers
• Interview: Author Rob Farber on the Secret Sauce for Programmers in the Kepler GPU
• Rob Farber on the Far-reaching HPC Implications from GTC 2013

The post Farber Book on Cuda Serves up Easy Teraflops appeared first on insideHPC.

Since February, 2011, Platform Computing has been blogging on the Applications of HPC from A to Z. As an avid reader, I have to offer them a Tip of the Hat for sticking with it for 26 posts.

I remember trying a similar kind of experiment when I first started blogging some years ago. As a way to motivate myself to write every day, I reviewed Wayne Dyer’s 101 Ways to Change Your Life one day at a time. It was great fun, and you can follow the full thread here.

Related posts:

• Intel series on developing multithreaded applications
• Video: IBM Platform HPC Speeds Applications with Intel Xeon Phi
• White House blog cites CCC as model for other communities, what about HPC?

The post Platform Concludes 8-Month Blog Series on HPC Applications A-Z appeared first on insideHPC.

Chapter 4 is entitled ‘The end of programming as we know it’. It notes: “The developers of the applications and programming systems have made much progress in providing appropriate abstractions, but not enough in that most developers and programming systems currently use the sequential model.” The chapter goes on to discuss the software implications of parallelism; the importance of managing memory latency; and the need for simple software abstractions and hardware mechanisms for parallel programming. It then goes into the state of the art in parallel programming, covering thread programming for shared memory, message passing interface, MapReduce, and distributed computing (with examples including the Seti@Home project where users donate spare computing cycles to search for alien life, and the less noble example of botnets).

Download the PDF or check out the 4000 other textbooks that National Academies Press has made freely available online.

Related posts:

• Free eBook: Introduction to High-Performance Scientific Computing
• New eBook: Getting Started in HPC Development
• Free Ebook: NAS Optimization for Dummies

The post Free eBook: The Future of Computing Performance: Game Over or Next Level? appeared first on insideHPC.

In this video, Dr. David Padua discusses his soon-to-be-published Encyclopedia of Parallel Programming. The book can be pre-ordered prior to its August 16, 2011 release.

Related posts:

• Book-in-Progress on Parallel Programming Available for Download
• Win a free copy of Programming Massively Parallel Processors [CONTEST CLOSED]
• Parallel Programming Summer School at CSCS, 17-19 August 2011

The post Video: Previewing the Encyclopedia of Parallel Programming appeared first on insideHPC.

In this video, Adaptive Computing COO Michael Jackson describes the HPC Evolution towards Gen4, a future capability where entire datacenters can be managed as a single resource. Recorded at the ISC’11 conference in Hamburg, June 22, 2011.

Related posts:

• Video: SGI – The Future of Technical Computing
• Video: CERN HLT Analysis of Intel Knights Ferry on High Energy Physics
• Video: Current and Future Lustre Developments at CEA

The post Video: Adaptive Computing’s HPC Evolution appeared first on insideHPC.

Foreword,
by Rich Brueckner

“Through our eyes, the universe is perceiving itself. Through our ears, the universe is listening to its harmonies. We are the witnesses through which the universe becomes conscious of its glory, of its magnificence.”
— Alan Watts

We all know of the Big Bang, how our universe came to be in a massive explosion, seemingly starting from nothingness. And for those who study cosmology, further understanding requires us to define the dark energies that somehow endowed our world with order.

Now, we haven’t observed dark energy, dark matter, and the secrets of dark gravity directly, but we do see their effects. As we learn in this book, without them, the universe would not have formed in a way that could have spawned intelligent life.

As a writer, I am intrigued by these dark energies because they imply a backstory–phenomena that happened first that led to this outcome. So in this way, dark energies seem to me to be metaphors of science. Like the stories of Genesis and Adam and Eve, what they really represent is a deeper truth.

In this book, Dr. Perrenod does a wonderful job of explaining the origins of the universe in way that is accessible to the layman. When you want to understand how the universe came to be, you ask an astrophysicist. But when you really want to know why, I think you have to start by asking yourself some questions. Try a thought experiment.

Put yourself in the place of a Universal Mind before the Big Bang. If you really wanted to understand yourself, you would need to have something intelligent outside of yourself that could experience that which is you. Not to get metaphysical here, but if we were at the scene of a crime, what I’d be suggesting here is motive.

Thanks to modern physics and cosmology, we no longer live in a universe where dark forces lurk far beyond our capacity for comprehension. I believe that, through the works of Stephen Perrenod and others, we will come to that knowing. But even as we look out to the stars, I think it begins with understanding that not only are we within the universe, but the universe is within us.

Dark Matter, Dark Energy, Dark Gravity is now available as an ebook on a number of different platforms including the Kindle and Nook, with more on the way. It’s well worth the read, and Dr. Perrenod has now started a blog on related topics in cosmology.

Related posts:

• Simulating the Multiverse With the Grid
• COSMOS – Fundamental Cosmology, Dark Energy, and the Cosmic Microwave Sky
• Perrenod: Why the Higgs Boson is not Dark Matter

The post New Book Examines how Dark Energy Enabled a Universe that Supports Intelligent Life appeared first on insideHPC.

Intel^® Parallel Studio XE 2011 offers the best optimizing compilers in the industry along with highly tuned performance libraries for math, crypto, imaging, and compression. Get latest processor performance improvements by a simple re-link of your code. Innovations being introduced include new parallel programming paradigms for C++ like the Intel^® Parallel Building Blocks and enhancements in the Fortran compiler with Fortran 2008 standards support, including Co-Array Fortran. Try a free 30-day trial.

Related posts:

• Intel Rolls Out Parallel Studio 2011
• Video: Intel Parallel Studio XE Demo at SC12
• Video: Intel Parallel Studio XE Array Building Blocks Demo

The post Sponsored Post: Get extreme performance and reliability: Intel® Parallel Studio XE appeared first on insideHPC.

The thesis of the book is that incumbents in markets – especially large and well entrenched markets – seldom survive fundamental technology changes in their industries. In a world where we’ve seen newspapers crumbling, record labels struggling and Blockbuster imploding and making way for the rise of Netflix it seems kinda intuitive to most of us but we can’t quite place why this happens.

Suster uses profanity in his review, which is a shame, but I think he has some good insights into this topic. The past year or two has seen some great HPC companies fall into acquisition (SGI and Sun Microsystems for example) and this book is now on my must read list.

Related posts:

• Book review: Principles of Parallel Programming
• Book Review: The Human Face of Big Data
• Book Review: Programming Massively Parallel Processors by Kirk and Hwu

The post Book Review Provides Sneak Peek at SC10 Keynote appeared first on insideHPC.

ISBN 1420072137

I just recently finished reading Introduction to Concurrency in Programming Languages, one of the entries in CRC’s incredibly active Computational Science Series (“Incredibly active?” Yes: the series homepage lists 7 titles applicable to HPC coming in 2011, and a similar number published in 2010.)

I picked this book out of my large-ish stack of books waiting, mostly patiently, to be reviewed because I’m working on a research project these days that has to do with new models of parallel programming. I figured I’d get a decent grounding in what’s already been done, and why, but I was also concerned I’d get lost in computer science formalism. I was right, and wrong.

The authors are from a nice mix of the theoretical and the applied: Sottile is from the U of Oregon, Mattson is from Intel, and Rasmussen is from Los Alamos National Lab. All are active in supercomputing; you may recognize Mattson from his work on OpenMP or his book adapting the patterns concept to parallel programming. They set out for themselves as a goal “the motivation and definition of language level constructs for concurrency that have been designed, studied, and found their way into languages that we use today.”

Don’t let that turn you off. The authors don’t assume you are fresh out of a computer science languages course. They go to some pain to create a gentle slope back into the languages pool, with many explanations by way of analogy. “Language level constructs,” they explain, are things like loops that let the programmer express the concept of a loop without forcing her to manage the program counter explicitly. By extension, a language-level construct for parallelism might be a for loop that executes in parallel. Of course, in theory, the benefit of this approach is that the compiler is free to pick the implementation that works best and the programmer gets to worry about higher-level tasks, like whether the science is right.

In practice, however, we’ve never really gotten much further than first base with this approach. The literature is littered with attempts to separate specification from implementation in HPC that worked fine for some subset of special cases, but never really panned out in the general case (for example, HPF). But there is also quite a large body of advances that have made it into general use, and the authors cover those in this text in the hope that the way forward is in understanding why these worked, and building upon them.

What won’t you find?

This is not a book about OpenMP, MPI, and the other libraries and language tools that extend or augment traditional sequential programming languages like C and Fortran so that programmers can develop code that executes concurrently. These approaches are discussed, but only to set them apart from the true focus of the book: languages that include concurrency in fundamental operators as part of the language.

I’ve already mentioned one alluring, if difficult to attain, advantage of including concurrent constructs in the language: the programmer can raise his focus from the implementation details and focus on correctness. This can be, as I’ve already discussed, a somewhat suspect motivation for further work given the dismal history of the practice. Happily there are more convincing reasons to care. Whenever the compiler encounters a library call, say to MPI_Send, it has to assume that no optimizations are possible across that call. No code reordering, no optimizations on the calling side of the function to help the function execute more effectively, no elimination of variables that are never used again, and no optimization across processors to create more efficient code (for example, by coalescing many small messages on the programmer’s behalf). Promotion of concurrent constructs to be a member of the language itself, as the authors explain, puts all of this back into play, and puts the compiler back to work on behalf of the programmer.

This seems like a fairly small step that could have large payoffs in programmer productivity, and to my mind makes a convincing case for pursuing this work, no matter how jaded you are by previous grand plans to give over implementation to all-knowing compilers.

The lay of the text

As I’ve already mentioned, a clear focus in this book is on keeping the material accessible. The authors succeed at this brilliantly. I came to this book with the equivalent of a minor in computer science finished almost two decades ago, and almost no memory of language theory (other than the word “automata”, which I always liked as a word). That was enough grounding to enable me to easily keep up with the authors and still come away from the book with a deeper understanding of the concurrent world around me.

The text opens with a few chapters of introductory material on the core concepts in parallelism and concurrency. Then they move on in chapter 3 to concurrency control mechanisms, discussing the merits and demerits of techniques such as synchronization, locks, monitors, and so on. In chapter 4, “The State of the Art,” the authors cover libraries (and their limitations), along with message passing, explicitly controlled threads, and more advanced techniques such as transactional memory. Chapter 5 lays the groundwork for discussing and assessing the effectively of high-level language constructs for concurrency, including a nice subsection on cognitive dimensions which I found quite helpful.

Chapter 6 introduces the historical context (dataflow or ALGOL, anyone?), and chapter 7 pushes forward to modern day approaches with work on array notation, co-arrays, functional languages, and more. Chapter 8 addresses performance considerations.

Chapters 9 through 13 look at parallel algorithms and how they present themselves for effective concurrent implementation. After an introduction to parallel algorithms in general, each of the remaining chapters looks at a specific pattern of parallel processing (remember, Mattson is one of the authors of this book) and studies its implementation from several different perspectives. These chapters are quite helpful, and one can imagine them serving as focal points for practitioners expanding their knowledge or for students finishing out a semester with a big project.

The book has three appendices that provide additional material on three very different approaches to programming in parallel today: OpenMP, Erlang, and Cilk.

The last word

Ok, so you (probably) won’t be cracking this book open in front of a crackling fire at a ski lodge this winter. At least not if you want to go home with someone you didn’t know at the start of the trip. But if you are just jumping into the world of concurrent programming, or taking a more theoretical look at the approaches we’ve all been taking for granted for the past 20 years in an attempt to make things better, then this book is a great start.

The authors present a clear motivation for the relevance of continuing this work, and provide both the historical context and knowledge of present day practice that you’ll need to get off on the right foot. That they manage to do this while keeping the language clear and the text accessible is a tribute to the effort Sottile, Mattson, and Rasmussen put into the creation of the text.

Related posts:

• Book Review: Programming Massively Parallel Processors by Kirk and Hwu
• John Shalf Talks Programming Languages
• Do we need new languages for parallel processing?

The post Review: Introduction to Concurrency in Programming Languages appeared first on insideHPC.

edited by Michael A. Heroux, Padma Raghavan, and Horst D. Simon
SIAM (2006)

ISBN 0898716195

I just finished reading Parallel Processing for Scientific Computing, one of the most recent volumes to join SIAM’s Software, Environments, and Tools series of scientific computing books (Jack Dongarra is the editor in chief of the series). Parallel is organized around the themes and problems presented at the Eleventh SIAM Conference on Parallel Processing for Scientific Computing, held in San Francisco in 2004 (as fate would have it, I’m writing this review in a hotel in San Francisco); even though 2004 seems like a long time ago, the editors and contributors took care in the creation of this book, and it remains timely today.

The book includes 20 articles from 91 contributors organized into 4 sections. The authors are a computational who’s who — you are sure to recognize names like Simon, Gropp, Lumsdaine, Snavely, Stevens, Bader, Foster, Bailey, and more — and each section includes a mix of both practical and pragmatic articles.

For example the first section, Performance Modeling, Analysis, and Optimization, opens with an article by Jesús Labarta and Judit Gimenez on the changes in structure and implementation that are needed to move performance analysis from an art to a first class science. This article takes a step back and looks at the big picture, but still manages to stay grounded via the authors’ references to their attempts to implement some of their ideas in real software. This is followed up by a survey article that covers much of the state-of-the-practice in architecture-aware scientific computation, written as a collection of mini-articles on specific projects. The section is rounded out by a chapter on specific experiences getting to high performance on an early IBM Blue Gene, and then looks forward with a chapter on application performance modeling for ultra-scale systems.

The entire book follows this structure, with each section featuring a mix of the pragmatic and the theoretical, the strategic and the practical.

Section 2, Parallel Algorithms and Enabling Technologies, covers partitioning and load balancing (with a great section on partitioning in parallel contact/impact applications), non-PDE based computations, adaptive mesh refinement, multigrid, solvers, and fault tolerance. The fault tolerance chapter was of particular interest to me in this section. It is well-written, and a great place to start if you are just beginning to think about one of the main problems facing the practical use of exascale systems in the near future.

Section 3, Tools and Frameworks for Parallel Applications, is a well-written survey by William Gropp and Andrew Lumsdaine that would serve as an excellent primer for a scientist wanting to stand up a cluster and get busy using it to run codes. Other articles in this section include a survey of parallel linear algebra software by Eijkhout, Langou, and Dongarra, as well as two chapters that point to a possible future for HPC software development in component-based software systems and frameworks for scientific computing.

Finally the last section, Applications of Parallel Computing, walks through challenging broad categories of HPC applications. The chapters here include a treatment of PDE-constrained optimization, parallel mixed-integer programming, multicomponent simulations, and computational biology, all with an emphasis on parallel aspects.

The text closes with a capstone article by the editors that looks at the challenges and opportunities for computational science.

The last word

I think the editors have done an excellent job of herding a collective view of scientific computing from what would otherwise have been just another collection of articles. Even though the book is four years old now, and even though the conference that inspired it was held six years ago, Parallel remains quite up to date in some aspects of its outline of the start-of-the-art in computing. Even in those areas where it is beginning to show its age (for example, the Blue Gene performance tuning chapter), the book remains an excellent starting point for more research.

The clear, jargon-free writing style makes for an easy read, and the references alone make exploring this text well worth your time. They are often quite complete: I found several chapters with 100 or more citations that readers can explore to develop a fuller understanding of a topic of particular interest. If you are just starting graduate studies in HPC and want to get a broad overview of the many facets of research in our field, then this book is an outstanding starting place. And if you are a seasoned practitioner, I think you’ll find the text provides a valuable point of view on a broad range of topics, with references that should keep you busy well into many sleepless summer nights.

Be sure to check out the other book reviews we’ve done here at insideHPC.

Related posts:

• Book Review: Programming Massively Parallel Processors by Kirk and Hwu
• Review of Petascale Computing
• Book review: Principles of Parallel Programming

The post Review: Parallel Processing for Scientific Computing appeared first on insideHPC.

I just finished reading the new book by David Kirk and Wen-mei Hwu called Programming Massively Parallel Processors. The generic title notwithstanding, readers should not come to this book expecting one of the highly theoretical and general parallel programming texts that most of us had at least some experience with in grad school. This book is very focused on one thing: teaching readers how to develop parallel applications that perform well on NVIDIA’s GPUs using NVIDIA’s CUDA language.

People learn in different ways, some responding well to a theory-based approach that only eventually gets down to implementation, and others responding well to generalization from the specific. I’m a specific kind of guy as, apparently, are the authors of this book. Kirk and Hwu wrote the book on the premise that learning the specifics of writing high performance code for GPUs with CUDA is a useful way to learn about parallel programming in general. Some suspicion of this point of view is warranted, given that the authors are both affiliated with NVIDIA (Kirk is an NVIDIA Fellow and was until 2009 the company’s chief scientist, and Hwu is principle investigator for the first NVIDIA CUDA Center of Excellence at the University of Illinois at Urbana-Champaign).

However, this book does at least as good a job at teaching general parallel principles through implementation as other, more platform-agnostic, MPI and OpenMP books I’ve read; and being tied to specific hardware gives Programming… at least one advantage those other books haven’t had. Namely, parallel programming on any HPC system is complex and targeted at specific hardware in direct proportion to the degree you care about performance, and it is precisely because it is tied to specific hardware that this book does a good job teaching that lesson alongside the more generally useful patterns for parallel programming.

Outline of the book

The book starts out introducing GPUs and parallelism, the history of GPU computing, and CUDA in general in the first three chapters. Chapter 4 presents an overview of CUDA threads, including thread scheduling and basic latency hiding techniques. Chapter 5 begins to look at the device in more detail, focusing on the CUDA memories and various techniques for organizing a computation to make the most use of the highest performing memory. Chapter 6 goes into more detail on coding for performance on the GPU, with helpful discussions and examples of techniques to hide latency in memory accesses and increase FLOPS. Chapter 7 provides a general discussion of floating point arithmetic. Chapters 8 and 9 are application case studies that walk the reader through applying some of the techniques already discussed in the context of two specific applications: MRI reconstruction, and molecular visualization. Chapter 10 attempts to generalize the heretofore GPU-centric discussion to other devices and paradigms, while Chapter 11 introduces OpenCL. The final chapter offers some comments about what the future may hold for GPUs.

Good introduction to parallelism, CUDA, and GPU programming

As an overall impression, the text is well-written, well-organized, and gives a lucid explanation of some difficult concepts. As a caveat, readers should note that this is not my first parallel programming book nor my first exposure to the concepts, but I still feel like the explanations and accompanying example code should be accessible even to beginning parallel programmers.

The book proceeds from a first easy-to-understand parallel implementation through more difficult specialized optimizations for several real-world application examples, an approach that lends relevance to the lessons. Each of the optimizations is described in prose and via accompanying pseudo-code, which keeps the pace lively but still offers the benefit of concretizing the discussion with something close to working examples. The techniques introduced along the way — memory coalescing, block decomposition, tiling, loop fusion, and so on — are techniques that are part of any parallel programmer’s tool chest, and are vital to learn. Naturally the implementation details in CUDA for a GPU will not bear much resemblance to what a programmer will find on an IBM Blue Gene or Cray XT5, but the authors are careful to concentrate not just on writing the CUDA code to make a particular optimization work (though that is covered), but also on developing the “computational thinking” that readers need to understand why the techniques work, and to spot when they’ll be useful in other settings.

A hallmark of application performance optimization is understanding how features of the hardware will influence the decision to select a particular algorithmic approach over others (however much we might wish for a robust infrastructure to optimize code without binding it to specific hardware). Programming… does a good job of describing the Tesla hardware architecture, and of demonstrating how this knowledge should be integrated into the programming process during application design. Experienced readers will recognize the thought process the authors describe, and beginning parallel programmers will have a solid example to serve as a go-by as they migrate to other platforms throughout the course of their career.

The book includes a chapter (chapter 10) on “computational thinking” in which the authors attempt to take a step back from their heads-down CUDA focus and provide a more generalized context for the information already presented. A chapter on OpenCL is also included. Still, in approaching this book it is important to remember that the text is primarily an introduction to parallel programming using CUDA, and a beginner will need exposure to other texts to round out his or her understanding. Fortunately the patterns and applications that Programming… relies upon to teach its lessons are fairly common, and as readers explore other material they will find ample opportunity to learn even more by directly comparing implementations in other programming approaches and for other parallel architectures to those presented in this book.

The last word

As a beginning text this book has a significant advantage that beginning texts written for MPI, OpenMP, and so on don’t have: there are 200 million CUDA-capable GPUs already deployed, and the odds are pretty good that most readers either have, or can readily get access to, a computer on which they can meaningfully learn parallel programming. If you are new to parallel programming and have access to a Tesla GPU, this book is a fine place to start your education. Readers already comfortable with parallel programming will find clear explanations of the Tesla GPU architecture and the performance implications of its hardware features, as well as a solid introduction to the principles of programming in CUDA, though they’ll probably do a lot of skimming over the already-familiar basics.

Be sure to check out the other book reviews we’ve done here at insideHPC.

Related posts:

• Win a free copy of Programming Massively Parallel Processors [CONTEST CLOSED]
• Book review: Principles of Parallel Programming
• Update, Kirk and Hwu available in the Kindle store

The post Book Review: Programming Massively Parallel Processors by Kirk and Hwu appeared first on insideHPC.