Monday, 11 January 2021

ks_2samp returns p-value of 1.0

I have two pandas DataFrame data1 and data2, and both DataFrame have an integer column h filled with different values varying from 1 to 50.

data1 has a sample size of roughly 55000, whereas data2 has a sample size of roughly 8000. I am not able to upload the exact data due to their sizes, but below are the histograms I created of data1['h'] vs. data2['h']:

enter image description here

(I applied matplotlib.yscale('log') for an easier observation)

enter image description here

To compare the distribution, I used ks_2samp from scipy.stats. I composed one two-tailed test and two one-tailed tests to observe both directions of superiority:

# h indices are significantly different
print(ks_2samp(data1['h'], data2['h']))

# data1 h indices are greater
print(ks_2samp(data1['h'], data2['h'], alternative='greater'))

# data2 h indices are greater
print(ks_2samp(data1['h'], data2['h'], alternative='less'))

The results were the following:

Ks_2sampResult(statistic=0.1293719140156916, pvalue=3.448839769104661e-105)
Ks_2sampResult(statistic=0.0, pvalue=1.0)
Ks_2sampResult(statistic=0.1293719140156916, pvalue=1.5636837258561576e-105)

I have practiced ks_2samp before for other projects, but seeing such obscure p-values is quite new to me. The second result, especially, makes me wonder if I'm performing the test incorrectly, as p-value being 1.0 seems extremely absurd.

I've researched some similar issues including the following StackOverflow question (scipy p-value returns 0.0), but unfortunately this issue is not identical to any reported issues of yet.

I'd love to get any insights to interpret such results or to fix my approach.



from ks_2samp returns p-value of 1.0

No comments:

Post a Comment