Block all mouse events except scroll

Right now I'm using some kind of loading component (actually a absolute positioned div with a strong transparency and a loading spinner) to disable some inputs / areas and show they are loading.

But if its positioned above a scrollable element (div), I'd still like to be able to scroll it (using mousewheel / touch-input).

<div style="position: relative">
  <div
    style="
      position: absolute;
      left: 0;
      right: 0;
      top: 0;
      bottom: 0;
      background-color: white;
      opacity: 0.5;
      pointer-events: none;
    "
  ></div>

  <div style="width: 200px; height: 200px; overflow: auto">
    <div style="height: 1000px">
      <button onclick="alert('YOU CANNOT TRIGGER ME!')">
        YOU CANNOT TRIGGER ME
      </button>
    </div>
  </div>
</div>

Using pointer-events: none; sure works for scroll-events but does not block e.g. the button from being pressed. How to achieve this?

In this solution the content is blocked but not scrollable, too:

<div style="position: relative">
  <div
    style="
      position: absolute;
      left: 0;
      right: 0;
      top: 0;
      bottom: 0;
      background-color: white;
      opacity: 0.5;
    "
  ></div>

  <div style="width: 200px; height: 200px; overflow: auto">
    <div style="height: 1000px">
      <button onclick="alert('YOU CANNOT TRIGGER ME!')">
        YOU CANNOT TRIGGER ME
      </button>
    </div>
  </div>
</div>

This is a strongly simplfied minimal repro. I want the semi-transparent div to be a common component that can just be placed above any content without knowing / being able to modify it.

from Block all mouse events except scroll