Phone Number with Password Authentication using Firebase

In my current project, I'm registering a user with email ID and then in profile section when the user updates a phone number, I authenticate that phone number & using PhoneAuthCredetials I'm merging both Authentication methods for the same user.

Now the real pain is, My user can sign in with Phone number or Email ID but for Email ID user has to enter password & for phone number it'll need OTP. (I Tried that, Entered phone number & password for signInWithEmailAndPassword() but not worked)

I wanted the user to log in with email or phone with a password. the way Flipkart does.

Is there any way I can manage that?

from Phone Number with Password Authentication using Firebase