Workshop -- Monday afternoon 

Logging in.  

putty to unity.  

To get into the head node for the Xeon cluster  henry2, 

ssh -l username henry2.hpc.ncsu.edu

(ssh -l username mcrae.hpc.ncsu.edu 
to get to the p690) 

There is a HowTo at
http://www.ncsu.edu/itd/hpc/Documents/HowTo.php

To get the intel environment, type 

>add intel 
(or add pgi for the Portland group compilers) 

To compile a parallel code codename.f (having set one of the environments) 
you can type 

mpif77 codename.f -o myprog

thereby producing the executable program  myprog 

(mpicc codename.c -o myprog) 


The henry2 cluster (as well as mcrae) schedule parallel jobs
with the LSF scheduler.  

SOME STUFF

To get some sample codes you can type
ftp cs.fit.edu
login as anonymous
cd pub/howell
get pachec.tar
exit

tar xvf pachec.tar

cd mpi-pacheco/chap03 

EDITING IN UNIX 

You can look at files by
view ring.f

:q 
to get out of the editor   

If you actually want to edit
vi ring.f 

i 
to insert
[esc] to quit inserting
x
to cut a character.  
dd
to cut a line
p to put the character or line you just cut.  
:wq to get out and save
:q! to abandon changes .  

emacs (over putty? ) 

pico   also works.  



BASIC COMMANDS IN LSF 

To submit a parallel job, use the bsub
command.  The easiest way is with a bsub script.  
>man bsub
will give you some information.  

Here's a sample bsub file.  

#! /bin/csh
#BSUB -q single_chassis
source /usr/local/lsf/conf/cshrc.lsf
source /usr/local/intel/compiler70/ia32/bin/ifcvars.csh
#BSUB -W 10
#BSUB -n 4
mpiexec  ./ring
#BSUB -o /share/gwhowell/chap03/ring.%J

If the file's name is bring, the job ring can be submitted by

bsub < bring 

Below is a commented version.  

#! /bin/csh
#BSUB -q single_chassis
# submitted to the single chassis queue (will run locally, good for 
#                                          a few processors) 
source /usr/local/lsf/conf/cshrc.lsf
source /usr/local/intel/compiler70/ia32/bin/ifcvars.csh
# these lines source the intel environment, not actually necessary
#          if you've just typed add intel 
#BSUB -W 10
#  Job will time out after ten minutes
#BSUB -n 4
#  Request 4 processors 
mpiexec  ./ring
# best to use mpiexec, as opposed to mpirun, which
#    would actually end up running on the head node. 
#BSUB -o /share/gwhowell/chap03/ring.%J
#    standard output goes to  this file, where the %J
#      opens a file ring.LSFIDnumber 

Some other common lines would be 
#BSUB -i input.dat

to specify a standard input file

#BSUB -e ring.%J

to specify an error message file (else error messages will
go to the file specified by #BSUB -o ) 

You can play with the queues to see what's appropriate
The standard queue is good for jobs that take quite a few
processors.  The debug queue is good for short jobs with a 
few processors.  To see a list of queues, try

>bqueues

>bqueues -l 
will show you more information.  

After you've submitted a job (called bsub) you can check on 
its status by 

>bjobs 

or 

>bhist 

and find out how long the job has been queued or running.

>bpeek

will let you look at the output file (which isn't actually copied
to where you think it is till your jobs exits LSF).  
      call MPI_Comm_rank(MPI_COMM_WORLD, my_rank, ierr)
      call MPI_Comm_size(MPI_COMM_WORLD, p, ierr)

SIMPLE MPI 

      program simple 
      include 'mpif.h' 
      integer ierr, p, my_rank
      call MPI_Init(ierr) ! first MPI call 
      call MPI_Comm_rank(MPI_COMM_WORLD, my_rank, ierr)
      call MPI_Comm_size(MPI_COMM_WORLD, p, ierr)
c
      if ( my_rank.eq.0 ) then
c            some stuff
      else      ! rank not zero
c           whatever all the other procs should do
      endif 
c
      call MPI_Finalize(ierr) ! may be no parallel procs after this
      stop 
      end 

If you're in the intel environment there are MPI man pages.  Of course, 
if you don't know what the MPI commands are? 

>cd /usr/local/intel/mpich/man/man3

>ls 
will give you a list.  


sends, receives

ring code, paired sends and receives