Autosizing TextViews in RecyclerView causes text size to decrease

I am trying to use Autosizing TextViews in a RecyclerView, but when I scroll a few times the text gets so small that it's obviously not working properly.

Example of my TextView:

<android.support.v7.widget.AppCompatTextView
        android:id="@+id/textview_unit_title"
        android:layout_width="@dimen/tile_image_size"
        android:layout_height="wrap_content"
        android:maxLines="2"
        android:textSize="@dimen/medium_size"
        android:textColor="@color/color_text"
        android:paddingTop="@dimen/padding_title"
        android:layout_marginRight="2dp"
        android:layout_marginEnd="2dp"
        app:autoSizeMaxTextSize="@dimen/style_medium"
        app:autoSizeTextType="uniform"
        app:layout_constraintRight_toRightOf="parent"
        app:layout_constraintLeft_toRightOf="@id/imageview_unit_icon"
        app:layout_constraintTop_toTopOf="parent"/>

Should I update this scaling somewhere else programmatically or is there another solution?

from Autosizing TextViews in RecyclerView causes text size to decrease