Heroku container:push always re-installs conda packages

I've followed the python-miniconda tutorial offered by Heroku in order to create my own ML server on Python, which utilizes Anaconda and its packages.

Everything seems to be in order, however each time I wish to update the scripts located at /webapp by entering

heroku container:push

A complete re-installation of the pip (or rather, Conda) dependencies is performed, which takes quite some time and seems illogical to me. My understanding of both Docker and Heroku frameworks is very shaky, so I haven't been able to find a solution which allows me to push ONLY my code while leaving the container as is without (re?)uploading an entire image.

Dockerfile:

FROM heroku/miniconda

ADD ./webapp/requirements.txt /tmp/requirements.txt

RUN pip install -qr /tmp/requirements.txt

ADD ./webapp /opt/webapp/ 
WORKDIR /opt/webapp

RUN conda install scikit-learn  
RUN conda install opencv

CMD gunicorn --bind 0.0.0.0:$PORT wsgi

from Heroku container:push always re-installs conda packages