RecyclerView's Items Position On The Screen Changes On The Run

I have this RecyclerView where any item can be clicked to open, and when it's opened, it grows (picture 1 is a closed item, picture 2 is an opened item)

The layout file of an item holds two states - the closed card, and the opened card. To switch between them, I change only the visibility of any state. Here are the methods that control an item's opening (expanding) or closing (shrinking) :

    /**
    * handles the expanding functionality.
    */
    public void expand() {
        shrink = true;
        if (expandedItem != -1) {
            notifyItemChanged(expandedItem);
        }
        expandedItem = getLayoutPosition();

        toggleExpandShrinkIcon(true);
        if (!openedFromParent[1]) {
            openedFromParent[1] = true;
        } else {
            openedFromParent[0] = false;
        }
        expandedContainer.setVisibility(View.VISIBLE);
        shrunkProgressBar.setVisibility(View.INVISIBLE);
    }

    /**
     * handles the shrinking functionality.
     */
    public void shrink() {
        toggleExpandShrinkIcon(false);
        expandedContainer.setVisibility(View.GONE);
        shrunkProgressBar.setVisibility(View.VISIBLE);
        shrink = false;
    }

These methods are located in the RecyclerView's adapter, inside of the ViewHolder's class, and they are public so I could use them also out of the RecyclerView's adapter class (not only by clicking), as I did when one item hovers another.

Recently I added drag-to-hover functionality (using this library) so that I can drag any item on top of any other item, and when one item hovers another item, the lower item gets opened.
When an item gets opened, it pushes all the other items below it to be able to expand without hiding the items under it (like in the first video).
When moving from hovering one item to another, say from the second item to the third, when hovering the second item it gets opened and the third item is pushed down, and when moving to the third item the second item gets closed, but the third item won't go back up. Then when hovering the third item, it gets opened on the fourth item (see the second video to understand better).

How can I solve this? (make the third item go back up, like when the drag is released, at the end of the second video)
Help would be highly appreciated! (:

First picture (closed item):

Second picture (opened item):

First Video (item gets opened & closed in the list):

Second video (item dragged):

from RecyclerView's Items Position On The Screen Changes On The Run