Suppose I have a single function delineation
. I want to run the same function multiple times for multiple parameters parallelly instead of sequentially one after the other.
import numpy
from pysheds.grid import Grid
xs = 82.1206, 72.4542, 65.0431, 83.8056, 35.6744
ys = 25.2111, 17.9458, 13.8844, 10.0833, 24.8306
def delineation(image_location):
variable_avg = []
for (x,y) in zip(xs,ys):
grid = Grid.from_raster(image_location, data_name='map')
grid.catchment(data=map, x=x, y=y, out_name='catch')
variable = grid.view('catch', nodata=np.nan)
variable = numpy.array(variable)
variablemean = (variable).mean()
variable_avg.append(variablemean)
return(variable_avg)
#calling function serially one after the other with different parameters and saving the results to a variable.
results1 = delineation(r'/home/test/image_1.tif')
results2 = delineation(r'/home/test/image_2.tif')
results3 = delineation(r'/home/test/image_3.tif')
For example, If I run delineation(r'/home/test/image_1.tif')
then delineation(r'/home/test/image_2.tif')
and then delineation(r'/home/test/image_3.tif')
, as shown in the above code, it will run sequentially one after the other and if it takes 5 minutes for one function to run then running these three will take 5x3=15 minutes. Hence, I am wondering if I can run these three parallelly/embarrassingly parallel so that it takes only 5 minutes to execute the function for all the three different parameters.
Help me with the fastest way to do this job. The script should be able to utilize all the resources/CPU/ram available by default to do this task.
from Fastest way to run a single function in python in parallel for multiple parameters
No comments:
Post a Comment