Your last HPC book? The best HPC-related book ever?

Print Friendly, PDF & Email

What’s the last HPC-related book you bought, and when did you buy it? What’s the best HPC-related book you’ve ever bought (you define “best” however you’d like: most useful, most inspiring, whatever)?


  1. Hmmm…. I haven’t bought a good HPC book … didn’t know there were any (good ones). Could we invert the discussion and start with a list of HPC books (or have people nominate books)? I have a bunch of old ones talking about vector machines, and Cray T3D as the next new thing …

    I have some favorite numerical analysis books, and a few programming/methodology books.

    Maybe we need an HPC book? A modern HPC book. But one that is a living book. Technology changes too quickly for the dead wood version.

  2. I would have to second Joe’s note. I’ve read some good MPI and numerical analysis-specific books.

  3. Well, I really meant the last technical book you bought to help you accomplish your HPC-related job better (you define better), but excluding Perl and CSH books. Numerical Recipes would count on my list, for example, and so would Synthesis of Parallel Algorithms and Workflows for e-Science (for example).

    There are at least a couple hundred that have been printed over the past 20 years that are directly tied to HPC and parallel computing, and this doesn’t include the “cross purpose” books like GPU books and numerical methods books or most of the “encyclopedias” (MPI library references, for example).

    Just published in 2008 I’ve found:

    Petascale Computing: Algorithms and Applications (ISBN: 1584889098)
    The Art of Multiprocessor Programming (0123705916)
    Principles of Parallel Programming (0321487907)
    High Performance Computing and Grids in Action (1586038397)
    Parallel Computing: Architectures, Algorithms and Applications (781586037963)
    Parallel Algorithms (1584889454)
    Supercomputing Research Advances (1604561866)
    Grid Resource Management: Towards Virtual and Services Compliant Grid Computing (1420074040)

    One of the promising titles set to come out early this year is High Performance Computing: History, Concepts, and Implementation (3540451218).

    I’m actually compiling a list of all the HPC-specific books I can find, and if there is enough interest I’ll publish it.

  4. I’ll confess to not having read any of the above books, but that’s because skimming through some similar books in the past has done precious little to better my understanding of the field. To rephrase, for me, my work focuses on helping people develop and run parallel scientific applications, so, for example, low-level books about how best to implement a ‘gather’ operation isn’t terribly important to me. Knowing where to use it, though, obviously is.

    I think that books on algorithms -not numerical methods, mind you, but algorithms- would be most important for people new to HPC, followed closely by books on data structures. I see people who simply use Numerical Recipes as a cut-and-paste for doing something but without thinking about how that thing should be done. An example would be an N-body code (without any long-range forces) that someone I know is running which uses an O(N^2) algorithm when an O(N) one exists. Having a sense of why algorithms are important, rather than just ‘how do I get a number that I can publish?’ would be immensely useful for them.

    Data structures are also important for two reasons – first, for ease of use, since having everything as global can be messy and makes it difficult for people new to an application to parse what a given routine needs and does. And second, though this is more involved, for properly dealing with things like caches. A lot of people are now more concerned about memory movement than flops, since that’s becoming the bottleneck, and proper design of data structures with regards to the memory hierarchy is incredibly important.

    So, book recommendations? Algorithms in C by Sedgewick is still on my bookshelf and was pretty good. I find that algorithms tend to get more focus in C than in Fortran, too, so even if you’re a Fortran programmer, a C book is fine. In fact, maybe even better than fine, since you’ll need to translate the ideas, and thus think. For data structures, any book will deal with the basics, but I don’t know of one that also tries to tie in data structure design to modern x86 (ideally) memory system architecture.

    All in all, for me, HPC is about the science, so I find myself reading far more scientific books than computing ones these days. That is, if you’re an HPC specialist and find yourself working with people in, say, astrophysics, pick up a book on astrophysics. If you find yourself working on biostatistics, pick up a book on that. Each of those will probably take you farther, hour for hour, than yet another book on computational ideas that you’re largely familiar with.

    That’s my (long winded) two cents. 🙂

    – B

    (PS. If you’re on the hardware design side, obviously little of the above applies!)

  5. Doug Enright says

    Hi John,

    I have used extensively the Patterson & Hennessy books:


    “Computer Organization & Design: The Hardware/Software Interface” – Patterson & Hennessy (0123744938)

    “Computer Architecture: A Quantitative Approach” – Hennessy & Patterson (0123704901)

    Parallel architecture is important too. I have used:

    “Parallel Computer Architecture: A Hardware/Software Approach” – Culler, Singh with Gupta (1558603433)

    The Patterson & Hennessy books should make ANY HPC book list.

    Compilers & Programming Languages:

    Not being a compiler and/or programming language expert, but having to delve into compiler and language issues at times one should have an understanding of how a compiler works and how languages impact the ability of the compiler to generate efficient assembly. This is even more critical given the discussion of the languages (UPC/CAF/Sisel/Haskell/etc) required for efficient parallel code generation for large multicore architectures. Scott’s book is a really good introductory and mid-level book for these topics:

    “Programming Language Pragmatics” M.L. Scott (1558604421)

    Numerical algorithms are important too, but I’ll have to think about which of the multiple books I have qualify as HPC-related numerical books.

  6. Hamming’s “Numerical Methods for Scientists and Engineeers” text is my favorite. And quite dog eared.

    I have Sedgewick around somewhere. As well as some “Scientific Programming in C/C++” or similar. Never had a Fortran book, learned by doing (on an IBM mainframe at that … eek!)

    I (usually) pick up the SCxx proceedings, though I forgot to this year.

    Might be worth putting a page or two for discussion of texts/books.

  7. Brian – thanks for the not-at-all long-winded reply. I have Sedgewick (in C) on my own shelf, and Golub and van Loan along with a few of the Numerical Recipes books.

    Given that we have those resources available, to what extent to you (and everyone else) feel that these books aren’t sufficient to hand to an apprentice HPC practitioner? IE, does our community need its own set of algorithms and data structures books written in an HPC context (whether from the particular perspective of our applications or our hardware or the scale of our problems)?

  8. Kevin Buterbaugh says

    Happy New Year John,

    While I know it has been maligned in some circles, I feel that the O’Reilly book “High Performance Linux Clusters” by Sloan is very good for those just starting out in HPC. I also like another O’Reilly book, “High Performance Computing (2nd edition)” by Dowd and Severance.

    But the main reason I’m posting a comment is that I find myself reading technical books less and less as the years go by. More and more of the information I need can be found online, often for free…


  9. Kevin – (I haven’t forgotten about that package I owe you – too much reveling over the holidays!).

    I have Dowd, first ed., but not the clusters book.

    I agree with you about the wealth of knowledge available online for free, but I wonder if you feel that, especially for new entrants to the field, a curated approach like you’d find in a book (for example, a general introduction to HPC architectures, or GPUs for HPC, or whatever) might serve to get readers on a solid footing from which they could do their own spelunking on the internet? Is there a need/use for that type of book in our profession? Especially if it was less than the typical $50 for many HPC books, pithy, not stodgy, and possibly electronic?