Swift library that supports both app and extension

I've created a library for my iOS app but I get the warning linking against a dylib which is not safe for use in application extensions.

I know this is because I haven't enabled Allow app extension API only for my library. So when I enable that setting my library gives errors where I have used UIApplication.shared.

I know I cannot use shared in my extension, and don't actually need to, but this library is used by both my app and my extension.

So the question is, how can I compile the library with a guard around UIApplication.shared?

I already use this:

#if !IS_EXTENSION
    // Cancel the background task
    UIApplication.shared.endBackgroundTask(bgTask)
#endif

and set IS_EXTENSION for Active Compilation Conditions in my extension target, and also IS_EXTENSION=1 for Preprocessor Macros in my app extension, however the library still gives me warnings on these lines.

from Swift library that supports both app and extension