JS canvas zoom and pan with large data (100k-1M objects)

I want to implement fast and high-res zooming and panning on my Canvas. As you may already read in the title, I have a lot of data, so using CanvasRenderingContext2D.scale() and CanvasRenderingContext2D.translate() with redrawing isn't an option, because it is way to laggy. I already tried using Images, but I want to see all the data-points while zooming and panning. Drawing only the data-points that are visible is a good start, but this makes the zooming and panning only possible on a very close zoom, but I actually want to be able to zoom out until I can see all the data points. The data points are drawn with circles and are very close together if that helps you in any way. I appreciate every suggestion on how to handle this.

from JS canvas zoom and pan with large data (100k-1M objects)