Monday 25 February 2019

What are alternative methods for pandas quantile and cut in pyspark 1.6

I'm newbie to pyspark. I have pandas code like below.

bindt = df[df[var].notnull()][var].quantile([0,.1,.2,.3,.4,.5,.6,.7,.8,.9,1]).unique()

df['{0}_quartile'.format(var)] = pd.cut(df[var], bindt, labels=False, include_lowest=True )

Could anyone please suggest how to rewrite above code in pyspark 1.6 dataframe.

Thanks in advance



from What are alternative methods for pandas quantile and cut in pyspark 1.6

No comments:

Post a Comment