Why does a Free Pascal dylib need DYLD environment variables?

0
votes

I have a Mac app mostly built with Xcode that uses a dynamic library (dylib) built with Free Pascal via Lazarus. When I turned on Apple's "Hardened Runtime" feature, the dylib stopped working, until I checked the option "Allow DYLD Environment Variables". This is described as "Allows an application to be impacted by DYLD environment variables, which can be used to inject code into the process." That code injection bit sounds like something I'd like to avoid. Any idea why this happens or what I can do about it?

To answer some questions asked in a comment:

I'm not sure exactly what failed, as I didn't write the dylib. It may have been trying to talk to a server over the Internet. All I know right now is that it returned an unexpected and unhelpful error code.

The app calls functions in the dylib using dlopen and dlsym. The dylib is in the Contents/Frameworks subdirectory of the app. The app does not set any environment variables, either in the Info.plist or in code.

otool -L output for dylib:

@rpath/lib<redacted>.dylib (compatibility version 0.0.0, current version 0.0.0)
/System/Library/Frameworks/Cocoa.framework/Versions/A/Cocoa (compatibility version 1.0.0, current version 22.0.0)
/usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)
/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1258.1.0)
/System/Library/Frameworks/Foundation.framework/Versions/C/Foundation (compatibility version 300.0.0, current version 1259.0.0)
/System/Library/Frameworks/AppKit.framework/Versions/C/AppKit (compatibility version 45.0.0, current version 1404.47.0)
/System/Library/Frameworks/ApplicationServices.framework/Versions/A/ApplicationServices (compatibility version 1.0.0, current version 48.0.0)
/System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices (compatibility version 1.0.0, current version 728.13.0)
/usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)

otool -L output for main executable:

/System/Library/Frameworks/Security.framework/Versions/A/Security (compatibility version 1.0.0, current version 58286.251.4)
/System/Library/Frameworks/AVFoundation.framework/Versions/A/AVFoundation (compatibility version 1.0.0, current version 2.0.0)
/System/Library/Frameworks/AudioToolbox.framework/Versions/A/AudioToolbox (compatibility version 1.0.0, current version 492.0.0)
/System/Library/Frameworks/Accelerate.framework/Versions/A/Accelerate (compatibility version 1.0.0, current version 4.0.0)
/System/Library/Frameworks/CoreMedia.framework/Versions/A/CoreMedia (compatibility version 1.0.0, current version 1.0.0)
@rpath/Quesa.framework/Versions/A/Quesa (compatibility version 1.6.0, current version 2.0.0)
@executable_path/../Frameworks/Ming.framework/Versions/A/Ming (compatibility version 0.0.0, current version 0.0.0)
@rpath/SBEngineLib4.framework/Versions/A/SBEngineLib4 (compatibility version 1.0.0, current version 1.0.0)
/System/Library/Frameworks/Quartz.framework/Versions/A/Quartz (compatibility version 1.0.0, current version 1.0.0)
/usr/lib/libsqlite3.dylib (compatibility version 9.0.0, current version 274.22.0)
/System/Library/Frameworks/ApplicationServices.framework/Versions/A/ApplicationServices (compatibility version 1.0.0, current version 50.1.0)
/System/Library/Frameworks/Carbon.framework/Versions/A/Carbon (compatibility version 2.0.0, current version 158.0.0)
/System/Library/Frameworks/Cocoa.framework/Versions/A/Cocoa (compatibility version 1.0.0, current version 23.0.0)
/System/Library/Frameworks/AddressBook.framework/Versions/A/AddressBook (compatibility version 1.0.0, current version 1893.0.0)
/System/Library/Frameworks/SystemConfiguration.framework/Versions/A/SystemConfiguration (compatibility version 1.0.0, current version 963.250.1)
/System/Library/Frameworks/IOKit.framework/Versions/A/IOKit (compatibility version 1.0.0, current version 275.0.0)
/System/Library/Frameworks/WebKit.framework/Versions/A/WebKit (compatibility version 1.0.0, current version 607.1.40)
/System/Library/Frameworks/Foundation.framework/Versions/C/Foundation (compatibility version 300.0.0, current version 1570.15.0)
/usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 400.9.4)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)
/System/Library/Frameworks/AppKit.framework/Versions/C/AppKit (compatibility version 45.0.0, current version 1671.40.118)
/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1570.15.0)
/System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices (compatibility version 1.0.0, current version 944.3.0)
/System/Library/Frameworks/CoreVideo.framework/Versions/A/CoreVideo (compatibility version 1.2.0, current version 1.5.0)
/System/Library/Frameworks/QuartzCore.framework/Versions/A/QuartzCore (compatibility version 1.2.0, current version 1.11.0)

Update: I set a breakpoint at dlopen, and found that the dylib in question was trying to open another dylib, libiconv.dylib, by name rather than full path. It turns out that fails in the hardened runtime without the special entitlement.

1
macoslazarusfreepascaldyld
There's nowhere near enough information here for us to help. In what way did the dylib stop working? What's the specific failure? How was the dylib built? How was the app built to use it? Is it linked against the dylib or does it load it dynamically (e.g. using dlopen())? Where is the dylib located in the app bundle? Does your app attempt to set any environment variables, perhaps via Info.plist? What does otool -L print when applied to both the main executable of your app and the dylib? - Ken Thomases
@KenThomases, I answered your questions as best I could. - JWWalker
Are you passing the absolute path to the library to dlopen()? Does the library have any companion files that it needs to locate? It doesn't appear to be linked to any non-system libraries or frameworks. Could it be attempting to dlopen() other libraries itself? With "Allow DYLD Environment Variables" enabled (so things work), you could try running your app with the DYLD_PRINT_OPTS, DYLD_PRINT_ENV, DYLD_PRINT_LIBRARIES_POST_LAUNCH, DYLD_PRINT_APIS, and DYLD_PRINT_RPATHS environment variables set to 1 (either from a shell or via the Xcode Run scheme). - Ken Thomases
@KenThomases, yes I'm passing an absolute path to dlopen. No there are no companion files. I doubt that it's trying to dlopen other libraries. I tried setting those environment variables, and get lots of output, but I'm not sure what I'm looking for. - JWWalker
What does it report about any DYLD environment variables that may be set. After that, you should see somewhere in the output your call to dlopen(). You can ignore stuff before that. Then, you're looking to see what dyld APIs the library might be using or what other libraries it might be loading. Basically, anything that might fail if DYLD environment variables are disallowed. Is anything loaded from non-system locations? Does anything appear to be found based on whatever environment variables are set? - Ken Thomases

1 Answers

1
votes

The Free Pascal dylib apparently does dlopen("libiconv.dylib", ...) with just the file name and no path. This relies on dyld searching in $LD_LIBRARY_PATH, $DYLD_LIBRARY_PATH, current working directory, and $DYLD_FALLBACK_LIBRARY_PATH (or, if that last is undefined, $HOME/lib, /usr/local/lib, and /usr/lib).

However, dyld won't search anywhere for hardened-runtime processes without the Allow DYLD Environment Variables entitlement. Only full paths will work.

The dylib will either have to be changed somehow (if you can), or you'll need to enable Allow DYLD Environment Variables despite the risks (which I don't think are particularly severe).