Monthly Archives: November 2021

Xcode’s Environmental Pollution

For a while now my build server has been issuing a large number of mysterious but important sounding warnings along these lines:

warning: include location '/usr/local/include' is unsafe for cross-compilation [-Wpoison-system-directories]

I’m allergic to warnings, and tend to abide by a 100% no-warnings-allowed policy, at least when it comes to release builds. So it was important to me to track this down and finally silence it.

I was really scratching my head because I don’t specify this include location anywhere in my build commands. Normally when an issue like this comes up, it’s easy enough to debug the problem by copying the build command out of the build log and pasting it into the Terminal. Invoking it independently of the xcodebuild process usually reproduces the same warning, and gives you the specific combination of command line options that is leading to the warning being generated.

In this case however, copying and pasting the command line to the Terminal did not yield the warning. What’s going on?

I figured I would take a step back and just try to reproduce the problem independently from my build scripts. Instead of copying and pasting the specific “clang” invocation that yields the warning, I’ll copy and paste the “xcodebuild” line instead. Bzzt! Still no warning.

After a lot of trial and error, I came across the strangest observation: if I invoke “xcodebuild” from within my Python-based build script, the warning is emitted. If I invoke it directly from the Terminal, it isn’t. In fact, if I simplify my build script to simply invoking “xcodebuild”, the warning happens. Stranger still? If I change the script from “python3” to just “python”, the warning goes away again.

I strongly suspected that the difference in behavior must be based in environment variables, so I decided to add a line to the top of the python script:

import os; print(os.environ)

Sure enough, the environment variables differed when I ran the script with “python” vs. “python3”. To get a feel on your own Mac for how the two commands differ, you could run this as a one-liner with python and with python3:

python3 -c "import os; print(os.environ)"

If your configuration is like mine, the output will include a “CPATH” value something like this:

{... 'CPATH': '/usr/local/include', 'LIBRARY_PATH': '/usr/local/lib'}

That “CPATH” entry for example only exists when invoking the script with python3, and it’s this very environment variable that is creating the unexpected Xcode warnings!

I was perplexed about how or why the version of Python could impact these environment variables, but then I remembered that python3 is bundled in Xcode itself, and the version at /usr/bin/python3 is a special kind of shim binary that directs Apple to locate and run the Xcode-bundled version of the tool. Apparently, a side-effect of this mechanism causes the problematic environment variable to be set! Running the python3 implementation directly where it lives in the Xcode installation:

`xcrun -f python3` -c "import os; print(os.environ);"

Does not exhibit the problematic environment variable!

So the issue is not about running python vs. python3 per se, but about invoking an Xcode build via any mechanism that leads to Apple’s “relocation” shim being executed and inadvertently mucking with environment variables that will impact the build process. I’ve filed FB9776086 requesting that the unwanted environment variables not be set when invoking commands, but in lieu of a system-wide fix, I’ll be adding this to the top of my Python build scripts:

import os
if "CPATH" in os.environ: os.environ.pop("CPATH")

Now my automated builds are beautifully warning-free again!