Add seperate jniLibs reference for different module dependencies in Android Studio project

I am developing an android project with two different module dependencies:

1. A third party jar dependency.
2. An Android Library dependency that I have created myself.

Both of these modules are using different C++ libraries. The jar dependency refers to its libraries / .so files itself.

I have only added it to my project as a jar dependency. However, for android library dependency I have added the .so files in the libs/armeabi-v7a/ folder and have added their reference in module level build.gradle file as follows:

android {
 defaultConfig {

        ndk {
            abiFilters 'armeabi-v7a'
        }
    }


    sourceSets {
        main {
            jniLibs.srcDirs = ['libs']
        }
    }
}

Without adding this code to my android library dependency's build.gradle file, the .so files are not included in the build and I get an error when I try to load .so files. On the other hand, the jar dependency throws an error when it tries to refer to its own .so files when I add the above mentioned code in my android library dependency's build.gradle.

I have googled this issue and haven't found anything. Is there a way to reference the jniLibs directory module wise to resolve this sort of clash?

from Add seperate jniLibs reference for different module dependencies in Android Studio project