I have this code that I would like to edit and run it as an MPI code. The array in the code mass_array1 is a multi-dimensional array with total 'iterations' i*j around 80 million. I mean if I flatten the array into 1 dimensional array, there are 80 million elements.
The code takes almost 2 days to run which is quite annoying as it is only small part of the whole project. Since I can log into a cluster and run the code through 20 or so processors (or even more), can someone help me edit this code to an MPI code?
Even writing the MPI code in C language works.
#Alotting Black Holes at z=6
from tqdm import tqdm
bhs=[0]*1000
for i in tqdm(range(0,1000),leave=True):
bhs[i]=np.zeros(len(mass_array1[i]))
for j in range (len(mass_array1[i])):
bhs[i][j]=np.random.lognormal(np.log(Mbhthfit6(mass_array1[i],6)[j]),np.log(5))
Current C program using MPI on that cluster:
int main(int argc,char **argv){
float epsran;
FILE *fp;
char str[256];
fp=fopen("parameterfile.dat","w");
fprintf(fp,
" cosmological parameter\n"
"h:%f\n"
"omegam:%f\n"
"omegab:%f\n"
"omegal:%f\n"
"sigma8:%f\n"
"rho0mMpc:%e\n"
"alpha:%f\n"
"deltac:%f\n",ndh,
omegam,omegab,omegal,sigma8,rho0mMpc,alpha,deltac);
fclose(fp);
/* MPI test */
int i,Petot,MyRank;
clock_t start,end;
start = clock();
MPI_Init(&argc, &argv);
MPI_Comm_size(MPI_COMM_WORLD, &Petot);
MPI_Comm_rank(MPI_COMM_WORLD, &MyRank);
srand((unsigned)(time(NULL)+MyRank));
//printf ("Hello World %d\n%d", MyRank,Petot);
float samples[100];
for(i=0;i<100/Petot;i++){
samples[i]=halo_samples(1.68,1000);
outputS(235,30,varipsapp(samples[i],0),MyRank*(100/Petot)+i);
}
printf("Length:%d",(int)(sizeof(samples)/sizeof(samples[0])));
/* FILE *fpw;
fpw = fopen("Minitial.dat","w");
for(i=0;i<MyRank*(100/Petot);i++){
fprintf(fpw,"%f\n",samples[i]);
}
fclose(fpw);*/
MPI_Finalize();
end = clock();
}
Submitting a job
After this, there is a job.sh file that looks something like this:
#!/bin/sh
#$ -S /bin/sh
#$ -cwd
#$ -V
#$ -N mergertree
#$ -q all.q@messier04
#$ -q all.q@messier05
#$ -pe openmpi10 20
#$ -o resultfile/out.txt
#$ -e resultfile/error.txt
mpirun -np $NSLOTS ./a.out
from Convert a normal python code to an MPI code
No comments:
Post a Comment