How can I remove distortion introduced by librosa griffin lim?

I'm doing:

import librosa


D = librosa.stft(samples, n_fft=nperseg, 
                 hop_length=overlap, win_length=nperseg,
                 window=scipy.signal.windows.hamming)

spect, _ = librosa.magphase(D)

audio_signal = librosa.griffinlim(spect, n_iter=1024, 
                                  win_length=nperseg, hop_length=overlap, 
                                  window=signal.windows.hamming)
print(audio_signal, audio_signal.shape)
sf.write('test.wav', audio_signal, sample_rate)

And it is introducing noticeable distortion in the reconstructed audio signal. What can I do to improve that?

from How can I remove distortion introduced by librosa griffin lim?