Using Elmer on the Blade Center
- About Elmer
- Serial Usage
- Serious Serial Usage
- Parallel Runs
- About Elmer
Software licenses for commercial finite element codes are
more expensive than the hardware. Moreover, some
potential sofware use (e.g. for consulting) may be restricted
under license terms. Elmer is "open source" and versatile, so
if someone learns to use it, they can use it anywhere.
Elmer is general purpose finite element software, developed since 1995
by the Finnish government corporation CSC (not the same as
the U.S. corporation of the same name). Some application areas are analysis of
stresses and strains in solid objects, temperature, fluid flow, and
radation problems. Elmer uses standard numeric libraries such as
the metis partitioner and numerical linear algebra tools such
as lapack, arpack, the umf package for solution of sparse systems
of equations, and also allows sparse iterative solution.
Elmer can be run in parallel. Ansys input files can
be used.
- Serial Usage
From one of the login64 nodes on the blade center, (login04 or login05), type
source /home/gwhowell/scripts/elmer.csh
to add the needed paths. Example codes are available in
/usr/local/apps/elmer/tests
with some more serious examples in
/usr/local/apps/elmer/examples
Each test tries out a particular input file. So for example,
to try out the elasticity test, do
cp -r /usr/local/apps/elmer/tests/elasticity .
cd elasticity
cat Makefile
Makefile consists of the lines
# Elsticity solver test in 2D
#
run:
$(ELMER_GRID) 1 2 elasticity
$(ELMER_SOLVER)
clean:
/bin/rm test.log temp.log mon.out
/bin/rm -r elasticity
Type
ElmerGrid 1 2 elasticity
ElmerSolver
to run elmer. "ElmerGrid 1 2 elasticity" uses the file elasticity.grd
to set up the grid. "ElmerSolver" uses ELMERSOLVER_STARTINFO consisting
of
elasticity.sif
1
to pick up the solver type and desired convergence criteria from
the file elasticity.sif
Here the elasticity.grd file consists of
Header
Mesh DB "." "elasticity"
End
Constants
End
Simulation
Max Output Level = 3
Coordinate System = Cartesian 2D
Simulation Type = Steady State
Steady State Max Iterations = 1
Steady State Min Iterations = 1
Output Intervals = 1
! Output File = "elasticity.result"
! Post File = "elasticity.ep"
End
Body 1
Equation = 1
Material = 1
End
and the grid file elasticity.grd is
##### ElmerGrid input file for structured grid generation ######
Version = 210903
Coordinate System = Cartesian 2D
Subcell Divisions in 2D = 1 1
Subcell Sizes 1 = 1
Subcell Sizes 2 = 0.1
Material Structure in 2D
1
End
Materials Interval = 1 1
Boundary Definitions
# type out int
1 -4 1 0
2 -3 1 0
3 -1 1 0
4 -2 1 0
End
Numbering = Vertical
Element Degree = 2
Element Innernodes = True
Triangles = False
Surface Elements = 100
Element Ratios 1 = 1
Element Ratios 2 = 1
Element Densities 1 = 1
Element Densities 2 = 1
and uses the content elasticity.sif from this directory.
- Serious Serial Usage
Only short jobs should be run from the login nodes.
Longer jobs should be submitted to the LSF scheduler so that
they can run on any of the thousand or so computational cores.
As a first try, it's convenient to run the same simple job
to make sure it goes without a hitch.
Assume that in the elasticity directory,
you have the files elasticity.grd, elasticity.sif and
ELMERSOLVER_STARTINFO and the following file.
#! /bin/csh
#BSUB -q debug
source /home/gwhowell/scripts/elmer.csh
#BSUB -W 5
#BSUB -n 1
#BSUB -R em64t
ElmerGrid 1 2 elasticity
ElmerSolver
#BSUB -o /share/myname/out.%J
#BUSB -e /share/myname/err.%J
If the above file is belmer, then the job can be submitted by
bsub < belmer
The #BSUB -W 5 line asks for 5 minutes.
Once you know the job runs, edit the file to ask for more time. Also,
make sure the -o and -e output and error files are writing to a directory
for which you have write privileges.
- Parallel Runs
For the parallel case, environments from the login node
are distributed only to the first node of a parallel job,
so you may need to include the line
source /home/gwhowell/scripts/elmer.csh
in your .tcshrc file (necessary since elmer uses some .so
libraries linked at run-time. These .so files must be located
by all instances of the elmer parallel code). This time we take belmer as
#! /bin/csh
#BSUB -q debug
source /home/gwhowell/scripts/elmer.csh
#BSUB -W 5
#BSUB -n 4
#BSUB -R em64t
mpiexec /usr/local/apps/elmer/64gnu/5.4.1/bin/ElmerSolver_mpi TempDist.sif
#BSUB -o /share/myname/out.%J
#BUSB -e /share/myname/err.%J
and put it in the directory heateq-par, which we've again copied
from a subdirectory of the /usr/local/apps/elmer/tests directory.
This small sample job appears to run correctly and use 4 processors.
For more information, please see Elmer
NOTE. The current port of elmer (Feb. 2009) passes all the supplied tests,
but would likely give
better parallel performance if recompiled to use the HYPRE package. If a
user finds a need to scale elmer to many processors, please contact gary_howell@ncsu.edu.
