Monday, 14 September 2020

How to speed up the 'for' loop in a python function?

I have a function var. I want to know the best possible way to run the for loop (for multiple coordinates: xs and ys) within this function quickly by multiprocessing/parallel processing by utilizing all the processors, cores, and RAM memory the system has.

Is it possible using Dask module?

pysheds documentation can be found here.

import numpy as np
from pysheds.grid import Grid

xs = 82.1206, 72.4542, 65.0431, 83.8056, 35.6744, 82.1206, 72.4542, 65.0431, 83.8056, 35.6744
ys = 25.2111, 17.9458, 13.8844, 10.0833, 24.8306, 25.2111, 17.9458, 13.8844, 10.0833, 24.8306


def var(image_location):
    
    variable_avg = []
    for (x,y) in zip(xs,ys):
        grid = Grid.from_raster(r'/home/data/dir_data.tif', data_name='map')

        grid.catchment(data='map', x=x, y=y, out_name='catch', recursionlimit=15000000, xytype='label') 

        grid.clip_to('catch')

        grid.read_raster(image_location, data_name='variable', window=grid.bbox, window_crs=grid.crs)

        variablemask = grid.view('variable', nodata=np.nan)
        variablemask = np.array(variablemask)
        variablemean = np.nanmean(variablemask)
        variable_avg.append(variablemean)
    return(variable_avg)

Calling function: var(r'/home/test/image1.tif')

The dir_data.tiff used in grid = Grid.from_raster(r'/home/data/dir_data.tif', data_name='map') can be directly downloaded from here.

You can copy the same image with a different name in the directory and use it in var(r'/home/test/image1.tif') as well for testing.



from How to speed up the 'for' loop in a python function?

No comments:

Post a Comment