An on-line text on parallel computation is Designing and Building Parallel Programs, by Ian Foster. The author is well respected, though he complicates the subject a bit by starting from C++.

An up-to-date collection of tutorials is at NERSC NERSC Tutorials Topics particularly relevant To NC State HPC machines include MPI, OpenMP, IBM C/C++, XL Fortran, Selecting Math Libraries, IMSL Fortran 90 Library, ACTS Tools, Intro to make, and Using the Totalview Debugger.

One standard text is "Parallel Programming: Techniques and Applications Using Networked Workstations and Parallel Computers", by B. Wilkinosn and M. Allen, Prentice Hall, 1999.

A list of on-line texts can be found at Web-Texts compiled by Hanson.

A standard text is "Foundations of Multithreaded, Parallel, and Distributed Programming", Gregory R. Andrews, Addison-Wesley, 2000.

A more entertaining account is "In Search of Clusters--The Ongoing Battle in Lowly Parallel Computing", 2nd Ed. Gregory F. Pfister. Prentice-Hall, 1998.

A classic and since it is algorithmically oriented, still useful account is "Parallel and Distributed Computation -- Numeric Methods" by Bertsekas and Tsitsiklis. Prentice-Hall, 1989.

Another well-written introduction to parallel scientic computation is "Scientific Computating -- An Introduction with Parallel Computing", by Gene Golub and James Ortega. Academic Press, 1993.

It is often possible to get tremendous speed-ups just by using a serial processor at nearer peak capacity. A good account is "Performance Optimization of Numerically Intensive Codes", by Stefan Goedecker and Adolfy Hoisie. SIAM 2001.

One online MPI Tutorial is NERSC MPI Tutorial

Another online MPI Tutorial is LLNL MPI Tutorial

An MPI text is "Parallel Programming with MPI" by Peter S. Pacheco, Morgan Kaufmann, 1997.

A more complete reference is "MPI--The Complete Reference, Volumes I and II", by Gropp, Huss-Lederman, Lumsdine, Lusk, Nitsberg, Saphir, Snir, Walker, Otto, and Dongarra. MIT Press, 1998.

Another online OpenMP tutorial is NERSC OpenMP Tutorial

An online OpenMP tutorial is LLNL OpenMP Tutorial .

An OpenMP text is "Parallel Programming in OpenMP", by Chandra, Dagum, Kohr, Maydan, McDonald, and Menon. Morgan Kaufmann, 2001.

An online Totalview (debugger) tutorial is An LLNL Totalview Tutorial .

SCALAPACK is a standard library for parallel dense matrix computations. SCALAPACK User's Guide

The "ARPACK User's Guide -- Solution of Large-Scale Eigenvalue Problems with Implicitly Restarted Arnoldi Methods" by Lehoucq, Sorenson, and Yang. SIAM 1998.

A basic UNIX tutorial is Unix Tutorial UCal Tech via U of Washington .

A Fortran 77 tutorial is Stanford University Fortran 77 tutorial .

"An Introduction to Fortran 90 for Scientific Computing", James Ortega. Saunders Publishing, 1994 is a straightforward introduction.

"Fortran 90 Explained", Michael Metcalf and John Reid. Oxford Science Publications, 1990, is a standard reference.

A tutorial in the C language is Drexel University C Language Tutorial .