Running MPI job on TACC

I tried to run a mpi code on TACC, but got some problem. So I test with the following simple code first.

And then submit this job file as shown on TACC user guide webside

But still got this error

Do you know what’s the problem here?

You don’t need ibrun mpi, ibrun is a proxy for mpi calls, so just do ibrun ./hello

This is what I did, I just named the excutable mpi_hello

Try running this code please

I ran this code and it still gives following error

My CMakeList and modules are following

Is there any problem with them?

And by the way if I just run it on login node with mpiexec, it ran and gave following results.

I used this file: as

Compiled with: mpiicpc -o mpi

Submission file:

#SBATCH -J mpi      # job name
#SBATCH -o mpi.o%j  # output and error file name (%j expands to jobID)
#SBATCH -A Material-Point-Metho # Project
#SBATCH -N 1               # number of nodes requested
#SBATCH -n 4              # total number of mpi tasks requested
#SBATCH -p skx-normal      # queue (partition) -- normal, development, etc.
#SBATCH -t 00:01:00        # run time (hh:mm:ss) - 10 hours
# Slurm email notifications are now working on Lonestar 5
#SBATCH --mail-type=fail   # email me when the job fails
# run the executable named a.out
ibrun ./mpi

Then it works fine:

TACC:  Starting up job 5999171 
TACC:  Starting parallel tasks... 
06 July 2020 08:06:08 AM

P0:  HELLO_MPI - Master process:
P0:    C++/MPI version
P0:    An MPI example program.

P0:    The number of processes is 4

P0:    'Hello, world!'
P0:    Elapsed wall clock time = P1:    'Hello, world!'
P2:    'Hello, world!'
P3:    'Hello, world!'
3.09944e-06 seconds.

P0:    Normal end of execution.

06 July 2020 08:06:08 AM
TACC:  Shutdown complete. Exiting. 

I only used the default modules already loaded.

Currently Loaded Modules:
  1) intel/18.0.2      3) impi/18.0.2   5) autotools/1.1    7) cmake/3.16.1   9) TACC
  2) libfabric/1.7.0   4) git/2.24.1    6) python2/2.7.15   8) xalt/2.8

Just found the reason…

Previously I use openmpi module locally downloaded using spack.

Now I use default cray_mpich module and it works.

1 Like