Fdi 2008 Lecture7
Fdi 2008 Lecture7
Fdi 2008 Lecture7
John Burkardt1
1 Virginia Tech
qsub myprog.sh
When your job is completed, you will be able to access the output
file as well as a log file describing how the job ran, on any one of
the compute nodes.
# include <stdlib.h>
# include <stdio.h>
# include "mpi.h"
if ( 0 < id )
MPI_Send ( h[1] => id-1 )
if ( id < p-1 )
MPI_Recv ( h[n+1] <= id+1 )
if ( id < p-1 )
MPI_Send ( h[n] => id+1 )
if ( 0 < id )
MPI_Recv ( h[0] <= id-1 )
All processes can use the four node stencil now to compute the
updated value of h.
Actually, hnew[1] in the first process, and hnew[n] in the last
one, need to be computed by boundary conditions.
But it’s easier to treat them all the same way, and then correct the
two special cases afterwards.
if ( 0 == id ) hnew[1] = bc ( x[1], t );
if ( id == p-1 ) hnew[n] = bc ( x[n], t );
Here is almost all the source code for a working version of the heat
equation solver.
I’ve chopped it up a bit and compressed it, but I wanted you to see
how things really look.
This example is also available in a FORTRAN77 version. We will
be able to send copies of these examples to an MPI machine for
processing later.
i n t main ( i n t a r g c , c h a r ∗ a r g v [ ] )
{
i n t id , p ;
d o u b l e wtime ;
M P I I n i t ( &a r g c , &a r g v ) ;
MPI Comm rank ( MPI COMM WORLD, &i d ) ;
MPI Comm size ( MPI COMM WORLD, &p ) ;
update ( id , p ) ;
MPI Finalize ( );
return 0;
}
/∗ RHS r e t u r n s r i g h t hand s i d e f u n c t i o n f ( x , t ) . ∗/
{
return 0.0;
}
/∗ S e t t h e X c o o r d i n a t e s o f t h e N n o d e s . ∗/
f o r ( i = 0 ; i <= n + 1 ; i++ )
{
x [ i ] = ( ( double ) ( i d ∗ n + i − 1 ) ∗ x max
+ ( double ) ( p ∗ n − id ∗ n − i ) ∗ x min )
/ ( double ) ( p ∗ n − 1 );
}
/∗ S e t t h e v a l u e s o f H at the initial t i m e . ∗/
f o r ( j = 1 ; j <= j m a x ; j++ ) {
time new = j ∗ t i m e d e l t a ;
/∗ Send H [ 1 ] t o ID −1. ∗/
i f ( 0 < id ) {
tag = 1;
MPI Send ( &h [ 1 ] , 1 , MPI DOUBLE , i d −1, tag , MPI COMM WORLD ) ;
}
/∗ R e c e i v e H [ N+1] from ID +1. ∗/
i f ( i d < p−1 ) {
tag = 1;
MPI Recv ( &h [ n +1] , 1 , MPI DOUBLE , i d +1 , tag , MPI COMM WORLD, &s t a t u s ) ;
}
/∗ Send H [ N ] t o ID +1. ∗/
i f ( i d < p−1 ) {
tag = 2;
MPI Send ( &h [ n ] , 1 , MPI DOUBLE , i d +1 , tag , MPI COMM WORLD ) ;
}
/∗ R e c e i v e H [ 0 ] from ID −1. ∗/
i f ( 0 < id ) {
tag = 2;
MPI Recv ( &h [ 0 ] , 1 , MPI DOUBLE , i d −1, tag , MPI COMM WORLD, &s t a t u s ) ;
}
/∗ Update t h e t e m p e r a t u r e b a s e d on t h e f o u r p o i n t s t e n c i l . ∗/
f o r ( i = 1 ; i <= n ; i++ )
{
h new [ i ] = h [ i ]
+ ( time delta ∗ k / x d e l t a / x d e l t a ) ∗ ( h [ i −1] − 2 . 0 ∗ h [ i ] + h [ i +1] )
+ time delta ∗ rhs ( x [ i ] , time ) ;
}
/∗ Correct s e t t i n g s of f i r s t H in first interval , last H in last i n t e r v a l . ∗/
i f ( 0 == i d ) h new [ 1 ] = b o u n d a r y c o n d i t i o n ( x [ 1 ] , t i m e n e w ) ;
i f ( i d == p − 1 ) h new [ n ] = b o u n d a r y c o n d i t i o n ( x [ n ] , t i m e n e w ) ;
/∗ Update t i m e and t e m p e r a t u r e . ∗/
/∗ End o f t i m e l o o p . ∗/
}
Now that we have a source code file, let’s go through the process
of using it.
The first step is to compile the program.
On the MPI machine, a special version of the compiler
automatically knows how to include MPI:
mpicc -c myprog.c
We can only link on a machine that has the MPI libraries. So let’s
assume we’re on the MPI machine now.
To compile and link in one step:
mpicc myprog.c
mpicc myprog.o
mv a.out myprog
#!/bin/bash
#PBS -lwalltime=00:00:30
#PBS -lnodes=2:ppn=2
#PBS -W group_list=???
#PBS -q production_q
#PBS -A $$$
NUM_NODES=‘/bin/cat $PBS_NODEFILE | /usr/bin/wc -l \
| /usr/bin/sed "s/ //g"‘
cd $PBS_O_WORKDIR
export PATH=/nfs/software/bin:$PATH
jmdrun -printhostname \
-np $NUM_NODES \
-hostfile $PBS_NODEFILE \
./myprog &> myprog.txt
exit;
Burkardt Introductory MPI
Compiling, Linking, RUNNING
If you look, you can spot the most important line in this file, which
says to run myprog and put the output into myprog.txt.
The command -lnodes=2:ppn=2 says to use two nodes, and to
use that there are two processors on each node, for a total of four
processors. (System X uses dual core chips).
So to use the batch system, you first compile your program, then
send the job to be processed:
qsub myprog.sh
The system will accept your job, and report to you a queueing
number that can be used to locate the job while it is waiting, and
which will be part of the name of the log files at the end.
If your output does not show up in a reasonable time, you can
issue the command qstat to see its status.
We’ve gone very quickly over the steps involved in using MPI,
starting with and algorithm, writing a code that will use MPI, and
sending the code through the machine.
We will return to discuss some features of MPI a little more
carefully. In particular, we will look at the six MPI commands we
have used so far, add a few more, and explain how they work.
We will look at another programming example, which uses the
“master-worker” model, where one processor is in charge, and
issues work to the others.