Using Elmer on the Blade Center
- About Elmer
- Serial Usage
- Serious Serial Usage
- Parallel Runs
- About Elmer
Software licenses for commercial finite element codes are
more expensive than the hardware. Moreover, some
potential sofware use (e.g. for consulting) may be restricted
under license terms. Elmer is "open source" and versatile, so
if someone learns to use it, they can use it anywhere.
Elmer is general purpose finite element software, developed since 1995
by the Finnish government corporation CSC (not the same as
the U.S. corporation of the same name).
Some application areas are analysis of
stresses and strains in solid objects, temperature, fluid flow, and
radation problems. Elmer uses standard numeric libraries such as
the metis partitioner and numerical linear algebra tools such
as lapack, arpack, the umf package for solution of sparse systems
of equations, and also allows sparse iterative solution.
Elmer can be run in parallel. Ansys input files can
be used.
- Serial Usage
Elmer is a 64-bit application, so to use it connect to one
of the 64-bit login nodes with
ssh login64.hpc.ncsu.edu
and them use the command
add elmer
to configure your environment. Example codes are available in
/usr/local/apps/elmer/tests
with some more serious examples in
/usr/local/apps/elmer/examples
Each test tries out a particular input file. So for example,
to try out the elasticity test, do
cp -r /usr/local/apps/elmer/tests/elasticity .
cd elasticity
cat Makefile
Makefile consists of the lines
# Elsticity solver test in 2D
#
run:
$(ELMER_GRID) 1 2 elasticity
$(ELMER_SOLVER)
clean:
/bin/rm test.log temp.log mon.out
/bin/rm -r elasticity
Type
ElmerGrid 1 2 elasticity
ElmerSolver
to run elmer. "ElmerGrid 1 2 elasticity" uses the file elasticity.grd
to set up the grid. "ElmerSolver" uses ELMERSOLVER_STARTINFO consisting
of
elasticity.sif
1
to pick up the solver type and desired convergence criteria from
the file elasticity.sif
Here the elasticity.grd file consists of
Header
Mesh DB "." "elasticity"
End
Constants
End
Simulation
Max Output Level = 3
Coordinate System = Cartesian 2D
Simulation Type = Steady State
Steady State Max Iterations = 1
Steady State Min Iterations = 1
Output Intervals = 1
! Output File = "elasticity.result"
! Post File = "elasticity.ep"
End
Body 1
Equation = 1
Material = 1
End
and the grid file elasticity.grd is
##### ElmerGrid input file for structured grid generation ######
Version = 210903
Coordinate System = Cartesian 2D
Subcell Divisions in 2D = 1 1
Subcell Sizes 1 = 1
Subcell Sizes 2 = 0.1
Material Structure in 2D
1
End
Materials Interval = 1 1
Boundary Definitions
# type out int
1 -4 1 0
2 -3 1 0
3 -1 1 0
4 -2 1 0
End
Numbering = Vertical
Element Degree = 2
Element Innernodes = True
Triangles = False
Surface Elements = 100
Element Ratios 1 = 1
Element Ratios 2 = 1
Element Densities 1 = 1
Element Densities 2 = 1
and uses the content elasticity.sif from this directory.
- Serious Serial Usage
Only short jobs should be run from the login nodes.
Longer jobs should be submitted to the LSF scheduler so that
they can run on any of the thousand or so computational cores.
As a first try, it's convenient to run the same simple job
to make sure it goes without a hitch.
Assume that in the elasticity directory,
you have the files elasticity.grd, elasticity.sif and
ELMERSOLVER_STARTINFO and the following file.
#! /bin/tcsh
#BSUB -o out.%J
#BUSB -e err.%J
#BSUB -W 5
#BSUB -n 1
#BSUB -R em64t
source /usr/local/apps/env/elmer.csh
ElmerGrid 1 2 elasticity
ElmerSolver
If the above file is run_elmer, then the job can be submitted by
bsub < run_elmer
The #BSUB -W 5 line asks for 5 minutes.
Once you know the job runs, edit the file to ask for more time.
- Parallel Runs
For the parallel case, environments from the login node
are distributed only to the first node of a parallel job,
so you may need to include the line
source /usr/local/apps/env/elmer.csh
in your .tcshrc file (necessary since elmer uses some .so
libraries linked at run-time. These .so files must be located
by all instances of the elmer parallel code).
A batch script to run Elmer on multiple processors would look
similar to the following.
#! /bin/tcsh
#BSUB -o /share/myname/out.%J
#BUSB -e /share/myname/err.%J
#BSUB -W 5
#BSUB -n 4
#BSUB -R em64t
#
source /usr/local/apps/env/elmer.csh
mpiexec /usr/local/apps/elmer/64gnu/5.4.1/bin/ElmerSolver_mpi TempDist.sif
and put it in the directory heateq-par, which we've again copied
from a subdirectory of the /usr/local/apps/elmer/tests directory.
This small sample job appears to run correctly and use 4 processors.
For more information, please see Elmer
NOTE. The current port of elmer (Feb. 2009) passes all the supplied tests,
but would likely give
better parallel performance if recompiled to use the HYPRE package. If a
user finds a need to scale elmer to many processors, please contact oit_hpc@help.ncsu.edu.
