Can I use pythonnet without .AddReference?

The usual way to integrate pythonnet in your project is the following:

import clr
clr.AddReference('My.Assembly')
import My.Assembly

My.Assembly.DoSomething()

What if I don't want the assembly namespace to be imported and be available globally. Is there any way to achieve something like this:

my_assembly = magic_loader('My.Assembly.dll')
my_assembly.DoSomething()

from Can I use pythonnet without .AddReference?