Docker container with Python modules gets too big

I want my Docker container to use tensorflow lite (tflite) in a python script. My Dockerfile looks like this:

FROM arm32v7/python:3.7-slim-buster
COPY model.tflite /
COPY docker_tflite.py /
COPY numpy-1.20.2-cp37-cp37m-linux_armv7l.whl /
RUN apt-get update \
    && apt-get -y install libatlas-base-dev
RUN pip install numpy-1.20.2-cp37-cp37m-linux_armv7l.whl \
    && pip install --no-build-isolation --extra-index-url https://google-coral.github.io/py-repo/ tflite_runtime

CMD ["python", "docker_tflite.py"]

The Docker Container is too big for my microcontroller at 197 MB, is there any way to make it smaller?

---

UPDATE:

Following Itamar's answer, I have adjusted my Dockerfile:

FROM arm32v7/python:3.7-slim-buster as dev
COPY model.tflite /
COPY docker_tflite.py /
COPY numpy-1.20.2-cp37-cp37m-linux_armv7l.whl /
RUN apt-get update \
    && apt-get -y install libatlas-base-dev
RUN pip install --user numpy-1.20.2-cp37-cp37m-linux_armv7l.whl \
    && pip install --user --no-build-isolation --extra-index-url https://google-coral.github.io/py-repo/ tflite_runtime


FROM arm32v7/python:3.7-slim-buster as runtime
COPY model.tflite /
COPY docker_tflite.py /
COPY --from=dev /root/.local /root/.local
RUN apt-get update \
    && apt-get -y install libatlas-base-dev

CMD ["python", "docker_tflite.py"]

Meanwhile the Docker container is at 179 MB, which is already a progress, thank you very much. Is there any more optimization potential in my Dockerfile, e.g. in the apt-get statements?

from Docker container with Python modules gets too big