The IBM p690 is a 32-processor shared memory system. A single instance of AIX is the operating system for all 32 processors. Currently AIX version 5.1 is running on the p690.

The p690 may be accessed as mcrae.hpc.ncsu.edu.

Only SSH access is permitted for login sessions. FTP is enabled from the p690, but file transfers originating from other systems to the p690 must use SFTP (or SCP) rather than FTP.

Free SSH clients are available from various sources. Links to some commonly used versions are included here:
• Windows
• Unix, Linux

Use of some applications running on the p690 from a Windows desktop may require an X-Window server to be installed on the desktop system. X-Win32 is licensed for NC State users. See the ITECS remote access page for additional information and to download X-Win32 (Unity ID is required).

Use of the IBM compilers is strongly recommended. The GNU compilers are installed, however, these compilers produce significantly less efficient code than the IBM compilers.

• Fortran
  The IBM Fortran compiler is invoked with the xlf command. There are also xlf90 and xlf95 commands available. All the commands invoke the same compiler, just with different parameters. The file /etc/xlf.cfg is used to determine the options used by each of the commands.

  The following command line is recommended as a starting point for compiling serial executables on the p690:
  

      xlf -03 -qstrict -qarch=pwr4 [your_source_file] 

  Note that default size for reals is 32 bits. Also, the default data segment size is small, less than one gigabyte. For programs that require more memory for data use the -bmaxdata option to specify the amount of memory required in bytes. The following would request 2 gigabytes (which is the maximum data size available without going to 64-bit addresses):

      xlf -O3 -qstrict -qarch=pwr4 -bmaxdata:0x80000000 [your_source_file] 

  Additional details about AIX user process memory models.

• C/C++
  The command to invoke the IBM C compiler is xlc and to invoke the IBM C++ compiler is xlC. The following command line is recommended as a starting point for compiling serial executables on the p690:

     xlc -O3 -qstrict -qarch=pwr4 [your_source_file] 

• MPI
  MPI codes are compiled using the mpxlf, mpcc, and mpCC commands for Fortran, C and C++ programs respectively. It is not necessary to specify include paths or mpi libraries when using the mp* commands.

All parallel jobs and long serial jobs (more than about 15 minutes) should be submitted through the batch system, LSF.

An initial set of queues have been enabled. These queue will be adjusted in response to usage patterns and user needs. The primary resources controled by the queues are number of processors and CPU time for the job.

• Single Processor LSF Jobs
  • Create a script file containing the commands to be executed for your job
  • Use the bsub command to subit the script to the batch system. In the following example two hours of run time are requested:
    

    bsub -W 2:00 < script.csh

  • The bjobs command can be used to monitor the progress of a job
  • When the job completes any standard output or standard error generated by the job will be placed in the directory that was the current working directory when the bsub command was issued.
  • The bpeek command can be used to view standard output and standard error for a running job.
  • The bkill command can be used to remove a job from LSF (regardless of current job status).
• Multiprocessor LSF Jobs
  • Follow the same proceedure as for single processor jobs. However, on the bsub command one additional argument is required, -n number_of_tasks:

     bsub -W 2:00 -n 8 < script.csh 

  • In this example two hours of run time and 8 tasks are requested.
  • Note that MPI executables built with the mpxl* compilers are not started with an mpirun command. The poe command can be used, but is not required. The number of MPI tasks will be the number of tasks requested from LSF.