ubuntu 16.04 slurm srun failed with intel mpi?

I am trying to install slurm in a cluster running ubuntu 16.04.

I am using intel mpi and the installation directory is located at the head node /opt/intel/impi_5.01.

According to the slurm instruction, it needs to export the libpmi.so variable.

But, I installed slurm-llnl via ubuntu

sudo apt-get slurm-llnl

and I am not sure where the libpmi.so is located? So, I did a search and found a file here, is this the file I'm looking for?

/usr/lib/x86_64-linux-gnu/libpmi.so

Anyway, I export the variable and I tried

srun -p old -N3 -n24 hostname

It returns,

rolly@head:~$ srun -p old -N3 -n24 hostname
node02
node02
node02
node02
node02
node02
node02
node02
node01
node01
head
head
node01
head
head
head
node01
node01
head
node01
head
head
node01
node01

It appears working.

But as I run my task,

srun -p old -N3 -n24 ~/QE530-CPU/espresso-5.3.0/bin/pw.x

It produced errors,

mpiexec_node02: cannot connect to local mpd (/tmp/mpd2.console_rolly); possible causes: 1. no mpd is running on this host 2. an mpd is running but was started without a "console" (-n option)
mpiexec_node02: cannot connect to local mpd (/tmp/mpd2.console_rolly); possible causes: 1. no mpd is running on this host 2. an mpd is running but was started without a "console" (-n option)
mpiexec_node01: cannot connect to local mpd (/tmp/mpd2.console_rolly); possible causes: 1. no mpd is running on this host 2. an mpd is running but was started without a "console" (-n option)
mpiexec_node01: cannot connect to local mpd (/tmp/mpd2.console_rolly); possible causes: 1. no mpd is running on this host 2. an mpd is running but was started without a "console" (-n option)
mpiexec_node02: cannot connect to local mpd (/tmp/mpd2.console_rolly); possible causes: 1. no mpd is running on this host 2. an mpd is running but was started without a "console" (-n option)
mpiexec_node02: cannot connect to local mpd (/tmp/mpd2.console_rolly); possible causes: 1. no mpd is running on this host 2. an mpd is running but was started without a "console" (-n option)
mpiexec_node02: cannot connect to local mpd (/tmp/mpd2.console_rolly); possible causes: 1. no mpd is running on this host 2. an mpd is running but was started without a "console" (-n option)
mpiexec_node02: cannot connect to local mpd (/tmp/mpd2.console_rolly); possible causes: 1. no mpd is running on this host 2. an mpd is running but was started without a "console" (-n option)
mpiexec_node02: cannot connect to local mpd (/tmp/mpd2.console_rolly); possible causes: 1. no mpd is running on this host 2. an mpd is running but was started without a "console" (-n option)
mpiexec_node01: cannot connect to local mpd (/tmp/mpd2.console_rolly); possible causes: 1. no mpd is running on this host 2. an mpd is running but was started without a "console" (-n option)
mpiexec_node01: cannot connect to local mpd (/tmp/mpd2.console_rolly); possible causes: 1. no mpd is running on this host 2. an mpd is running but was started without a "console" (-n option)
mpiexec_node02: cannot connect to local mpd (/tmp/mpd2.console_rolly); possible causes: 1. no mpd is running on this host 2. an mpd is running but was started without a "console" (-n option)
mpiexec_node01: cannot connect to local mpd (/tmp/mpd2.console_rolly); possible causes: 1. no mpd is running on this host 2. an mpd is running but was started without a "console" (-n option)
mpiexec_node01: cannot connect to local mpd (/tmp/mpd2.console_rolly); possible causes: 1. no mpd is running on this host 2. an mpd is running but was started without a "console" (-n option)
mpiexec_node01: cannot connect to local mpd (/tmp/mpd2.console_rolly); possible causes: 1. no mpd is running on this host 2. an mpd is running but was started without a "console" (-n option)
mpiexec_node01: cannot connect to local mpd (/tmp/mpd2.console_rolly); possible causes: 1. no mpd is running on this host 2. an mpd is running but was started without a "console" (-n option)

I believe the error prompts are due to running mpiexec with intel-mpi, it should be using mpirun instead.

How can I correct the problem?

Thanks!

1 Answer

I found my solution.

1) sudo apt-get install mpich

2) srun --mpi=pmi2

3) mkl and intel related environmental variables are loaded correctly.

I hope this is going to help someone having similar problem.