Is it possible to avoid the shrinking of Chart.js pie charts when accompanied by labels?

To get labels on Chart.js Pie and Doughnut charts, there are plugins to do so, like: chartjs-plugin-labels but after doing so I noticed a big problem for my UI design:

The size of the actual chart shrinks so that the labels fit within the canvas. It makes sense that the labels need to be able to fit within the canvas they're rendered on, and thus the shrinking of the chart. But sometimes I may use labels, and other times not, and I need my pie charts to render the same size regardless.

Is there a solution in the settings of either Chart.js (the label plugin I'm using is compatible with < 3.0, I'm using 2.9) or chartjs-plugin-labels to maintain consistent chart size, regardless of whether labels are applied?

I tried applying an empty label to every chart, but the size of the chart actually shrinks based on the specific size of the labels being rendered, so aside from being a hacky solution, it doesn't consistently solve the uniformity problem.

For example maybe a way to make charts start off taking 50% of the canvas?

from Is it possible to avoid the shrinking of Chart.js pie charts when accompanied by labels?