Category Archives: Xcode

Nutty Namesakes

When Apple introduced the Mac App Store, it was clear that I would need to produce two distinct versions of my apps: one custom-built so suit the specific requirements of the App Store, and one built the way it always ever was, to be distributed directly to customers.

The approach I took was to simply duplicate the application target in all of my Xcode projects, such that for every final app target, there is a counterpart for the App Store:

Xcode build settings showing MarsEdit App Store having the same product name as MarsEdit.

Because the vast majority of build settings for all of my products are specified in Xcode configuration files, there isn’t much duplication here. True, each of the targets has to list its dependencies, resources, linked libraries, etc., but for the most part I go about editing source files on a day-to-day basis and when I ship an app update it just comes down to building and archiving each target in turn.

If you look carefully at the screenshot above, you will see that the MarsEdit App Store target has “MarsEdit” set as its product name. This is good, because I want each of the resulting apps to be named “MarsEdit.app.” As far as I know, there is no getting around it: the two targets have to share the same product name in order for them to build differing products that nonetheless have the same name.

This mostly works fine, in spite of a few Xcode glitches. One example is the auto-generated “Products” group in the project navigator shows two entries called “MarsEdit.app,” and doesn’t give any indication of which one is which. This can make it slightly annoying to easily e.g. right-click on a product and reveal it in the Finder.

Now as it happens I also define multiple targets that share the same product name in some of my supporting framework projects. Particularly since Apple started supporting the use of frameworks for iOS code, I’ve changed my iOS static library bundles over to frameworks. So where I used to have RSFoundation.framework and RSFoundation.o, I now have two targets, each producing a framework for the desired platform. Here’s an example where Xcode actually handles the doppelgängers with some aplomb. If you go to add a framework to a target and there are duplicates with the same name, it takes care to differentiate them:

Screenshot showing Xcode finding multiple libraries with the same name

Recently I ran into a troubling behavior with Xcode’s testing schemes, that I’ve correlated with the scenario of having a testing bundle target in a project that specifies as its test host a specific one of two targets whose product names match. The bug, in a nutshell, is Xcode can end up deciding that both targets should be built as forced dependencies of the test bundle, leading to annoying, hard to explain behaviors. I wrote this up as Radar 19924983 if you’re interested in reading more. And here’s the sample project I included with the bug, if that is something you’re interested in checking out.

On the whole I’m pretty happy with the approach of using separate targets with the same name. I am pretty sure I could work around whatever bugs I come across by resorting to splitting the two targets up into independent Xcode projects, but that feels a lot less elegant to me. The fact that Xcode supports duplicate names as well as it does in e.g. the “add libraries” panel I showed above gives me hope that support for namesake product targets will continue to improve in the future.

Convenient Build Settings

I wrote almost ten years ago about a trick I use in my Xcode projects that lets me set the marketing version for my products in one place, a build setting, and then reference that build setting everywhere else the version number needs to be expressed. This is a cool trick that has saved me a bunch of dumb hand-editing over the years.

And yet, all these years I have been laboriously, and usually behind schedule, updating my source bases every year to advance the range of years that my copyright statement applies to. For example, here is the string for MarsEdit’s NSHumanReadableCopyright string:

MarsEdit ${APPLICATION_VERSION}, ©2003-2015 Red Sweater Software

You can see my use there of the aforementioned marketing version build setting. With the opportunity staring me in the face, why have I spent the past decade manually advancing the year from 2005 to 2006 to 2007 to … well, I guess the charitable explanation would be that I’ve had better things to do, but that particular tedium also ends today.

My projects make heavy use of Xcode configuration files, such that every application or framework target inherits consistent default build settings that reflect my opinions about which warnings should be enabled, what platforms to support, etc. My configuration files all take advantage of a kind of cascade of imported files such that ultimately, every target in every Xcode project in my source base inherits settings from a file called RS-Target-All.xcconfig, and every project likewise inherits settings from RS-Project-All.xccconfig.

Today I added a simple line to RS-Project-All.xcconfig:

CURRENT_YEAR = 2015

The result is that every project and target in my source base now has access to this custom build setting, thus I can change the contents of all my NSHumanReadableCopyright strings to e.g.:

MarsEdit ${APPLICATION_VERSION}, ©2003-${CURRENT_YEAR} Red Sweater Software

And I will never have to update the marketing string’s copyright year again, on any of my projects, for as many years as I keep updating and releasing new versions of them. Quite appropriately, the terminating year on the copyright span will always be the year when I built and shipped the app. All I have to do every January is edit the solitary Xcode configuration file.

Technically I could go a step further and incorporate the product name as a variable as well:

${PRODUCT_NAME} ${APPLICATION_VERSION}, ©2003-${CURRENT_YEAR} Red Sweater Software

But since the product name isn’t changing all the time, that just starts to feel a bit fussy.

Some people also take advantage of the Xcode option to preprocess the Info.plist file, which could potentially allow for even more fancy substitutions. Maybe you could use the C preprocessor’s built-in __DATE__ variable to extract just the current year somehow? I never turned on preprocessing of Info.plist files because, at least when I last considered it, there seemed to be some bugs with it.

For now, I’m satisfied by the little bits of time that defining this once will save me, not to mention the avoidance of inevitable errors I would make when editing each occurrence by hand.

Enable Clang Modules, Disable Auto Linking

When Apple announced support for clang modules a few years ago, I jumped right in. I was curious to see what the build-time improvements might be, among other promised advantages.

Unfortunately things didn’t go well at the beginning. I ran into at least one bug, and probably more if I scratch my memory a bit harder. As cool as modules might turn out to be, they were not ready, at least for me.

Time heals some wounds, so I decided to take another look recently. I was annoyed by mysterious Xcode build failures, and at least some of them had to do with precompiled headers. I remembered in the back of my mind that modules were meant to (at your discretion) completely supersede precompiled headers, so I decided to give them another try.

True to form, I ran into another bug. This time, though, I could see a workaround in breaking down my monolithic project structure into smaller pieces. It was something I wanted to do anyway, and it would work around the modules problem. Win, win?

When I say I wanted to “break down” a monolithic project, I mean that, for example, a single ABC.xcodeproj that contains targets for A.framework, B.framework, and C.framework should be broken up so that each framework has its own .xcodeproj file. The monolithic structure is not too problematic on the face of it, but when you end up harboring references to specific targets in the monolithic project from various levels of other projects, things get a little hairy. In the case of the modules bug I reported, it seemed that some amount of cyclical referencing was responsible for surfacing the issue.

Unfortunately, breaking an Xcode project up can be fraught with peril. There is no easy way to say “take this target from ABC.xcodeproj, and move it to A.xcodeproj.” On the face of it, you’re stuck making a new A.xcodeproj and then painstakingly recreating the target definitions, file references, build phases, etc., that make up the rules for generating the target’s products.

A trick I use in this situation is to literally duplicate the ABC.xcodeproj file, then go in and hack out everything that isn’t contributing to the target you care about preserving. In this case, to get A.xcodeproj, I make a copy, then delete from the project any files that are solely used for B.framework or C.framework. The result is a standalone project for building A.framework. Of course, to avoid harboring redundant build targets for A.framework, you also remove from ABC.xcodeproj any of the files that are solely meant for building it.

That solves part of the problem, and you end up with a shiny new A.xcodeproj. But now you have to go trawling through your source bases, looking for any references to ABC.xcodeproj that expect to link to A.framework through it. These builds will now fail, because ABC.xcodeproj doesn’t have a target for the desired build. Worse, the way Xcode copes with this failure (dependent target suddenly missing), is to simply remove the dependent framework from both the dependencies and linked libraries list for the higher-level target.

To solve the problem, you have to add a project reference to the new A.xcodeproj, and then reconstitute both the dependency and linkage rules for the target. Once you do this for all the possibly umpteen-million targets that previously referenced A.framework in ABC.xcodeproj, you will once again enjoy a working build.

Ah, but thanks to an insidiously helpful new feature of modules, you don’t strictly have to do all of this. Enabling modules for your Xcode targets also opens the door to a feature called automatic linking which will cause modules that are imported by a target’s sources to be automatically linked. So, in a nutshell as long as my source code still contains references like:

#import <A/A.h>

Then Xcode and clang will helpfully also add the required framework linkage when building my target.

Ugh, that is so frustrating!

This sounds so helpful, how could I possibly be against it? The problem with auto linking is it will quietly mask any absent-minded dependencies on frameworks that violate the separation of concerns you have diligently instilled on your codebase. Put more bluntly: auto linking encourages spaghetti code.

One of the great advantages of frameworks is that they put up firm put permeable walls between the various components of a source base. If a given framework in your source base is intended to operate independently of the user interface, it’s easy to maintain this discipline by, for example, abstaining from linking with UIKit, AppKit, or whatever other user-interface frameworks may be at your disposal. Then when you absent-mindedly add a dependency to NSImage in your Foundation-level code, you’ll be greeted by a compiler error when it goes to link.

Not so with auto linking. The build completes without error, and now your LowLevelStuff.framework depends on UI frameworks. The permeable walls afforded by frameworks are no longer firm. You might argue: “What’s the big deal? It’s one little NSImage reference.” True, but it’s one little chink in the dam that gives you any sense of ownership or sanity about the scope of your code. Let that dam erode and you have cross-references not just between UI and Foundation layers of your code, but among and between any and all frameworks of varying, allegedly precise purposes. I.e. spaghetti code.

Discipline is one of the greatest and most rewarding challenges of programming. To the extent we successfully impose effortless standards on ourselves, we make the task of maintaining discipline that much easier. The firm walls of framework dependency are one such standard: a gentle reminder in the course of our daily work that things are running, even if only slightly, off the rails. Without those gentle reminders we’re bound to discover only months or years from now just how far we’ve diverged from our intended course.

In summary: enable clang modules, they’re pretty good.

CLANG_ENABLE_MODULES = YES

But automatic framework linking leads to lack of insight about actual dependencies between targets.

CLANG_MODULES_AUTOLINK = NO

Now that my source base is completely converted to using modules and eschewing precompiled headers, I feel good. I haven’t noticed any particular speed improvements, but I feel on board for the future. Adopting them sets me up for easier interoperation with Swift, when the time comes, and will surely spare me any bugs that sprout up specifically with precompiled headers, which Apple is no longer strongly motivated to fix.

Unit Testing Precision

In the course of practicing my on-again, off-again unit testing discipline, I find myself frequently wanting to zero in on just a few tests, so that I can quickly invoke tests from Xcode (Cmd-U) without suffering the lengthy re-running of a bunch that I already trust will pass.

Luckily the Xcode scheme editor provides for this, allowing tests to be disabled or enabled by file and even by individual test method. Instead of testing with Cmd-U, use Cmd-Opt-U to bring up the scheme configuration panel.

However, I’ve found the logic behind whether Xcode will enable or disable a whole slew of tests to be somewhat unpredictable. You can hold down the option key while clicking to affect its behavior, but I inevitably wind up clicking it the wrong way, such that the test I actually want to click is no longer enabled (because its parent file is unchecked).

Today I discovered a handy trick which is probably an oversight on Apple’s part: even when the checkbox for a specific test is disabled, it can be selected by control-clicking on the test’s row and selecting “Enabled Test” from the popup menu.

EnableTest

The great thing about this method is it takes care of whatever other enabling needs to take place for the selected test to be allowed to be enabled. I’m sure I will use this trick often.

Isolating Xcode Builds

I maintain an extensive array of automated builds with Jenkins. The automatic reporting of inadvertent build failures empowers me to stay much more on top of the quality of my own code, which is shared across many projects and targets many platforms and OS releases.

Unfortunately, I have frequently run into trouble with “false negative” builds. Builds which fail for perplexing reasons that I can usually only chalk up to “welp, something changed in Xcode.”

Sometimes the problem has seemed to be rooted in its finicky behavior with respect to paths that have spaces in them. Fine, I’ve removed all the spaces from my paths.

Other times, the problems have been much more nuanced, having to do with the problematic (I guess) sharing of cached data between builds. I don’t know how much of this should be chalked up to the fact that I’m building multiple projects at once, or that I’m building multiple problems of the same name, but with different dependencies and platform target. In any case, I have come to realize that the key to making sure my builds, you know, build when they should is to isolate each build as much as possible from the others.

To this end, over the years I have taken advantage of Xcode build settings such as SYMROOT, the setting that defines where your “Built Products” should be stored. But that only affects where the final products are saved. To make sure intermediate files that are used to build a product don’t get mixed and matched, I also override OBJROOT.

Overriding those two gets you a long way towards isolating a build from other builds that may have happened recently, or that may be happening right now. But, as it turns out, they don’t get you far enough.

Apple has increasingly moved towards some valuable changes in their approach to handling “cache” type data, including precompiled headers, header maps, module caches, etc. To the extent that Apple can use these caching strategies to enable faster builds, I’m all for it. But on my build server, it’s inevitably led to confusion and disarray.

Recently I’ve run up against confounding build errors that I can’t quite trace to specific problems. All I know is that “wiping out the Derived Data folder” solves it. Adding yet more build setting overrides to my build process also seems to solve some of the issues. For example, faced with a litany of precompiled header errors along the lines of:

fatal error: file 'BlahBlah.h' has been modified since the precompiled header 'WhooHah_Prefix-fmvtpvxgevrsamfytqmaabjnyjyx/WhooHah_Prefix.pch.pch' was built

I have found some relief by adding another override, this time for the SHARED_PRECOMPS_DIR. Recently, I ran up against a similar kind of failure, presumably related to the fact that I have started taking advantage of Apple’s support for “modules”:

fatal error: malformed or corrupted AST file: 'Unable to load module "[...]/DerivedData/ModuleCache/LQKN4K4Q4BR6/Foundation-1SPG61SLRK6SC.pcm": module file out of date'

That’s fine, but where is the “modules” equivalent of SHARED_PRECOMPS_DIR? It probably exists, though it’s not documented on the aforelinked build settings reference. Worse, since the advent of the DerivedData folder, it seems some of the claims in the build settings reference about default values are no longer true.

“Fine,” I thought, “how the heck do I override the Derived Data folder location altogether?” I don’t know why it took me so long to pursue this line of thinking. Perhaps my allegiance to the idea of saving effort through shared caching was too strong for my own good. When it comes down to it: I don’t mind that builds take a little longer or use more disk space, so long as they are an utmost reliable reflection of the current status of my source bases.

But how do you override the Derived Data folder location? If there is a build setting for it, then like the module caches folder, is evidently too new to be well-documented. Examining the build process in Xcode, it seems to be defined at a higher level than any of the other environment-variable based settings that can be observed in the build log e.g. for custom build phases.

I know you can set the Derived Data folder location in Xcode’s UI, but that, I assumed, only affects the builds that happen within the app. The brilliant idea finally dawned on me to check the xcodebuild man page, and what do you know?

-derivedDataPath path

Overrides the folder that should be used for derived data when performing a build action on a scheme in a workspace.

Since the vast majority, if not all, of the various cache and cache-like folders I’m trying to override are in fact located by default within the “Derived Data” folder, overriding just it could be the lynchpin that ensures my builds are isolated from one another.

I adjusted my build scripts to no longer override OBJROOT, CACHE_ROOT, or SHARE_PRECOMPS_DIR. Instead, they simply pass -derivedDataPath to xcodebuild with a suitably unique path:

xcodebuild -project MyProj.xcodeproj -scheme "MyScheme" -derivedDataPath "./MyCache"

You know what? Suddenly all my Jenkins builds are passing all their tests. Son of a gun.

Brent’s Feedback

I love Brent Simmons’s style of responding to my last post, in which I described a cover class for NSURLSession that makes is easier for me to adopt it gradually throughout my source base. Brent:

This is the right way to do it. The callers – including the unit tests – don’t have to know anything about the implementation, since the interface is the same. That’s just good programming.

That’s just good etiquette. When responding to somebody with whom you have a fundamental difference of opinion, lead with a compliment. Brent goes on to say:

It’s also not how I would do it in this specific case.

Brent argues that cover classes have their place when it comes to adapting APIs that are not native to the frameworks or language being developed in. But when there is no impedance mismatch, he says:

I’d rather just use a thing directly, rather than write a class that wraps a built-in class.

All good food for thought. I will add here that in the particular case of my “RSLegacyHTTPRequest” class, I decided to make it a subclass of the working title for my previously mentioned “RSSpiffyRequest,” which is the much less glamorous “RSHTTPSession.” RSHTTPSession is a subclass of NSObject, and happens to own a subsystem of objects that “don’t matter to you” but deeply involve NSURLSession. In fact, interacting with RSHTTPSession will feel a lot like interacting with NSURLSession.

The idea, longer term, is that clients of RSLegacyHTTPRequest will move away from that antiquated interface and towards RSHTTPSession. The argument for subclassing in this case is I like the pattern when it allows me to gradually move good logic upwards, from an antique class to a modern class. Is it awkward that it’s called RSHTTPSession, instead of RSHTTPSessionManager? Maybe. I’ll change it if things get weird.

So I’m doing things my way, adapting an antique class to the future by providing a cover class that translates an old-and-busted interface to NSURLSession, and doing things Brent’s way, by basing the future of my “spiffy” NSURLSession convenience classes on a base class that inherits from NSObject, provides a stable interface, but fully embraces and exposes the NSURLSession philosophy to its clients.

Same Tests, Different Class

I recently decided to tackle a long-postponed refactoring of my source base: to move from my own, custom CFNetwork-based “URL request” class, to something based on the modern NSURLConnection or NSURLSession classes from Apple.

When I first started writing networking code for OS X, NSURLConnection didn’t exist yet. If you wanted a class that approximated the convenience of NSURLConnection as we know it today, you had to write your own wrappers around the lower-level CFNetwork HTTP library functions. If you needed to have precise control over the loading of network resources from the web, you simply had to use CFNetwork (or the 3rd-party open-source libcurl library, which some folks did do!.)

When I acquired MarsEdit from NewsGator in early 2007, I wasn’t surprised to learn that Brent Simmons (the original author) had also implemented his own CFNetwork-based HTTP class: RSHTTPDownloader. I took a look at it and my own class, RSHTTPRequest, and ended up migrating the best parts of his with the best parts of mine, keeping the name I had grown accustomed to: RSHTTPRequest. (It was a great convenience that Brent’s Ranchero Software and and my Red Sweater adopted the same class-name prefix).

Fun fact: I actually worked briefly on the CFNetwork team at Apple, and learned the ropes of that somewhat-daunting, low-level API. Suffice to say, I was less scared off by CFNetwork than some people, but even I didn’t consider it to be perfectly simple to use. However, it worked well enough and when NSURLConnection debuted later in 2007, I found it somewhat wanting. I can’t remember the details but there were little things that I’d come to appreciate about having direct access to the “streams” and I lost that by moving to NSURLConnection. So I stuck with RSHTTPRequest.

Fast-forward more than seven years and I’m still on RSHTTPRequest. But now, it’s clearly to my detriment. Not only has Apple been improving NSURLConnection and the rest of the networking stack all this time, a couple years ago they introduced the all new NSURLSession, which makes the whole setup easier to use and more powerful at the same time. Capping off the incentives is the fact that new functionality such as background network loading sessions are essentially required if you want to make good use of recent technologies such as Share Extensions. In short, it’s a great time to switch to NSURLSession.

Just Switch to NSURLSession!

The only problem is I’ve spent more than seven years building a pretty complex network-oriented app around this one, I hesitate to say little anymore, this one disgusting evolution of what started out as a simple wrapper around CFNetwork’s HTTP loading mechanism. RSHTTPRequest has every manner of convenience method, special cases, not to mention subclasses covering highly specialized networking operations such as POST’ing HTTP forms, communicating with most blogging APIs on the planet, etc. You get the idea: I can’t just switch to NSURLSession. So what am I going to do?

I’m a very paranoid person when it comes to code, and it soothes me to take small, predictable, measurably safe steps when it comes to situations like this. I already have a number of unit tests on RSHTTPRequest that verify its behavior to a reasonable extent. The basics are covered, and as I’ve discovered and addressed bugs over the years, I’ve usually taken the time to add unit tests covering the nuances. So to be perfectly clear, the main impediments to my switching to NSURLSession are:

  1. The interface to NSURLSession is totally different from my custom class.
  2. The default behavior of NSURLSession is totally different from my custom class.

So I could hunker down for a long weekend, write a new NSURLSession-based helper, RSSpiffyRequest, and then change every single line of code that deals with RSHTTPRequest to instead use the pertinent methods on the new class. One problem with this approach is that I have to come up with the all new RSSpiffyRequest interface from scratch, before I change a single line of code. The other problem is even if I come up with the perfect interface for my new class, I’ll inherit all the unpredictably different behaviors of NSURLSession. Things will randomly not work. Crashes will probably happen. It will suck.

The only way forward, to my paranoid mind, is to find a way to slow down the transition so that fewer parts are changing at once. Since RSHTTPRequest already satisfies a contract with countless clients, and because it’s already covered by a moderate number of unit tests, the safest way forward is to change RSHTTPRequest so that it’s based on NSURLSession.

But hold on, that’s too fast for me, too. I don’t want to reimplement RSHTTPRequest’s whole interface at once, hold my breath, and hope that all the code still works. Instead, I need a new class that implements the interface of RSHTTPRequest, can be tested like RSHTTPRequest, and can be moved to by client code gradually, as the expected behavior is confirmed.

Parallel Request Classes

The approach I decided to take is to add a new class, RSLegacyHTTPRequest. The name is future-proofed: the presence of the word “Legacy” in the name is to remind me that it’s not the end-all be-all NSURLSession subclass. It’s not the ideal, convenient networking object that should be used across my source base. Instead, it’s the nasty, pragmatic, full-of-history class interface of RSHTTPRequest that happens to be implemented on top of NSURLSession.

This gives me the freedom to willy-nilly switch client code from RSHTTPRequest to RSLegacyHTTPRequest, just to see what happens. A great place to start was in the RSHTTPRequest unit tests class I mentioned. It currently verifies the behavior of RSHTTPRequest for most all niggling network details I’ve encountered over seven years. I switched it from testing RSHTTPRequest to testing RSLegacyHTTPRequest by simply adding this to the top of the test file:

#define RSHTTPRequest RSLegacyHTTPRequest

Boom! Everything fails. That’s a nice start, and makes for an easy challenge to chip away at. Pick a failing unit test, and make it work. In my case this involved discovering how to simulate or approximate the interface of the old RSHTTPRequest, but on top of NSURLSession. Slowly but surely, I came to a point where I had a new RSLegacyHTTPRequest class, completely based on NSURLConnection, that passes all my existing tests.

I was far from done. I have other classes that use RSHTTPRequest, and themselves have unit tests of their own. I pulled a similar trick to uncover other unit testing edge cases that should have been in RSHTTPRequest, but weren’t. I was beefing up my test class, in some cases even finding residual bugs from my old implementation.

I got to a point today where I realized I had a more-or-less “fully” functional RSLegacyHTTPRequest class, but I am still not ready to abandon RSHTTPRequest. I have other clients to meander through, verifying that they behave as expected when switched over to the new class. In the mean time, I was stuck wondering whether the unit tests, those golden unit tests, should be left testing the old RSHTTPRequest or testing the new RSLegacyHTTPRequest? Answer: they’re both in use, so both should be tested.

Multi-Class Unit Tests

Finally we get to the namesake of this article’s title: how do I use the same precious collection of unit testing code but apply it to two (or more!) classes that happen to share the same interface? The answer is to subclass. When you implement and run unit tests in an Xcode project, the testing system looks at runtime for any classes in the test bundle that “have methods that start with -test”. What this means is that, if you wanted to, you could redundantly run all the unit tests in one of your test cases twice simply by subclassing it and leaving the subclass empty. If “MyTests2” is a subclass of “MyTests”, then at runtime, it’s going to find and execute all the tests in each of the classes, which will happen to be exactly the same tests.

For my situation, I want to run all the tests twice, but I want to run them on instances of different class. I decided to accomplish this by making some modest edits to the test class, replacing every alloc/init pair for RSHTTPRequest:

[[RSHTTPRequest alloc] init...]

With a bottleneck method that instantiates the class from an overridable Class object:

- (Class) classForHTTPRequestInstances
{
	return [RSHTTPRequest class];
}

- (RSHTTPRequest*) requestForTestingWithURL:(NSURL*)theURL
{
	return [[[self classForHTTPRequestInstances] alloc] init...];
}

I then added a new unit test class called “RSLegacyHTTPRequestTests”, which merely implements -classForHTTPRequestInstances to return RSHLegacyHTTPRequest. Now when I build and run unit tests, every single, painstakingly collected unit test gets run twice, once for each of the super-important URL request wrapper classes in my framework.

Over time, hopefully over a short period of time, I’ll move away from RSHTTPRequest to RSLegacyHTTPRequest. And then, hopefully over a similarly short period of time, I’ll start moving to RSSpiffyRequest. All the while, however, I’ll rest easy at night, knowing that to a very great extent, all the functionality I’ve come to expect over years of developing this code is being constantly confirmed by this glorious set of unit tests.

Product Packaging Oversign

For a long time I have used a custom script phase to deliberately code sign all the bundles, helper tools, frameworks, etc., that are bundled into a final shipping product. The idea behind this approach was to overcome limitations in Xcode’s built-in code signing flags, which e.g. made it difficult to be sure that subsidiary projects would be built with the same code signing identity.

Recently I started looking at the possibility of switching to Apple’s relatively new “Code Sign On Copy” option for Copy Files build phases that involve code. For example if you’re copying files to a Frameworks, Plugins, or similarly obvious target for code resources, a friendly checkbox appears to offer code signing “as it’s copied in.”

Screenshot showing code sign on copy for copy files phase.

Xcode’s default code signing behavior gets some things right: for example, it takes care to preserve the metadata and entitlements of the original code signing, if any. This means that subsidiary product that e.g. are aware of specific sandboxing rules that should be applied to them can sign themselves as part of their build process, and the “Code Sign On Copy” action later will only update the identity associated with the code signing.

But I noticed a problem which relates to my recent obsession with script phase performance. The problem is that Xcode’s default code signing occurs too often. In fact in my projects, every single time I build, it re-signs every code object that had been copied, even though it hasn’t changed. This isn’t a big deal for one or two small frameworks, but for an app with dozens or hundreds of code objects, it becomes a noticeable time lag every time you rebuild to test a small fix.

The specific issue I discovered boils down to:

  1. Having entitlements for your target. E.g. being sandboxed.
  2. Modifying the Info.plist file in a build phase. E.g. to update the build number for the bundle.

If your target meets these criteria, then by my estimation Xcode will resign all the “Code Sign On Copy” files every time you build, regardless of whether they’ve changed. I reported this as Radar #19264472. Until Apple fixes it, the only workarounds I know are to suck it up and wait out the code signing every build, stop modifying your Info.plist, or go back to a manual code signing script that can take finer care about how and when to sign subsidiary code.

Explicit Requirement Satisfied

If you do any fine-tuned customization to your app’s code signing “designated requirement,” you may come across situations where the code appears to be signed correctly, but codesign verification nonetheless yields a cryptic, terse failure message:

./Your.app: valid on disk
./Your.app: does not satisfy its designated Requirement

This problem is pretty insidious, because the failure of an app to meet its designated requirement does not actually prevent you distributing it and having users run it. The code is still signed by you, it just doesn’t identify itself to the code signing system in such a way that it will be considered trusted by the system as being “itself.” This is bad.

How bad? The most likely to affect your users if the app uses the system keychain at all to store secure data. Even data placed in the keychain by the app itself will be denied to the app unless the user approves of the access. Worse, even if the user approves the access with “Always Allow,” the system will continue to prompt the user each and every time it tries to access the data.

The reason for the failure is that the “designated requirement” serves as a test of identity for your app. If your app doesn’t meet that test, the system assumes you no longer have any business accessing the data that you previously placed in the keychain. So an app that starts out not even meeting its own designated requirement is invalid from the start. We need our apps to meet their own designated requirements.

For most apps this is a no-brainer: just let the code signing system provide a default designated requirement. Typically, the default asserts that the bundle identifier and certificate signing chain used when signing the app are later the same. In short: unless an app is signed with the same certificate and has the same bundle ID as it had when it stored the data, it will not be allowed to access the data now.

Why customize the designated requirement? The reasons are probably too varied for me to imagine, but one common scenario that happens to affect me is one in which two different bundle IDs may effectively refer to the same “app.” For example instances of MarsEdit may possess either the “com.red-sweater.marsedit” or “com.red-sweater.marsedit.macappstore” identifiers, depending upon whether they were built for the Mac App Store or for direct sale. Another scenario where a custom designated requirement would be appropriate is if an application changed developers and either of the old developer’s and new developer’s certificate is meant to be considered legitimate. If you’re curious, poke around at the designated requirements of some of the apps on your Mac:

codesign -d -r- /Applications/Google\ Chrome.app
designated => (identifier "com.google.Chrome" or identifier "com.google.Chrome.canary") and certificate leaf = H"85cee8254216185620ddc8851c7a9fc4dfe120ef"

In this case it appears that Google wants to confirm that a very specific certificate was used to sign Chrome, but wants the pre-release “Canary” builds to be treated as authentic copies of the app, even though they have a different bundle identifier.

Customizing the designated requirement can lead to a pretty lengthy specification string that can be hard to process mentally. When such a string produces an app that doesn’t meet its own requirement, it can be hard to guess at which specific clause is failing. The codesign tool’s error message is of no help, but you can take advantage of a useful option related to the tool’s verification functionality. Pass “-R” along with a specific requirement string, and it will be evaluated separately from the designated requirement. The long and short of this is you can feed the codesign tool pieces of the larger requirement string, and see which ones pass and fail. To provide a string right on the command line, start it with an “=”:

codesign -vv -R '=identifier "com.apple.mail"' /Applications/iTunes.app 
/Applications/iTunes.app: valid on disk
/Applications/iTunes.app: satisfies its Designated Requirement
test-requirement: code failed to satisfy specified code requirement(s)

In this contrived case, we ask codesign to confirm that iTunes has Mail’s bundle identifier, and of course it fails. When we instead provide a requirement string that makes sense…

codesign -vv -R '=identifier "com.apple.iTunes"' /Applications/iTunes.app
/Applications/iTunes.app: valid on disk
/Applications/iTunes.app: satisfies its Designated Requirement
/Applications/iTunes.app: explicit requirement satisfied

…the codesign tool confirms that it passes.

In the past I have been at a loss for how to more easily debug designated requirements issues. I only discovered the “-R” option today, but I’m sure it will be a great help in the months and years to come.

Speeding Up Custom Script Phases

Some Xcode projects use a handy feature for extending the build process called a Run Script build phase. These build phases can be added to the linear list of steps that Xcode steps through when building a target, such that custom tasks such as preparing dynamically generated source code, verifying build results, etc., can call be done as part of Xcode’s standard build routine.

A problem you might run into is that a long-running script phase can be a real drag to wait through every single time you build. You can work around this problem in a few ways, depending on what makes sense for your project. Two of the simplest options are:

  1. Check the box to run the script only “when installing.”
  2. This is particularly handy if the stuff that’s happening in the build phase really doesn’t need to be there until the final release build, or if you are willing to manually perform the steps during development.

  3. Add logic to your build phase script to detect the build configuration, and either perform a more expedient version of the task, or skip it altogether. For example, a build phase that does laborious profiling of your finished product may not need to be done on “Debug” builds, so you could add a clause to the script:
    if [ ${CONFIGURATION} == Profiling ] ; then
    	echo "Running a very long profiling task!"
    	...
    	echo "Finished running a very long profiling task!"
    else
    	echo "Skipping long profiling task! Whoo!"
    fi
    

    (Note: you don’t have to use bash for your shell script phases. In fact, you can run whatever interpreter you like. I typically use Python, but used bash here as a canonical example).

These tricks work out fine when a script phase is clearly only needed under special circumstances, but what about script phases that have to be done every time you build, rely upon files that may change over time, and take a considerable amount of time to finish? For this we can take advantage of another nuanced feature of Xcode script phases, which is the ability to specify arbitrary input and output files for the script. I think it’s time for an illustrative screenshot:

Screenshot of a configured Xcode script phase

Here’s the low down on the impact of specifying input and output files with script phases in Xcode:

  1. Lack of modification to any of the listed input files encourages Xcode not to run the script phase. Hooray! Fastness!
  2. Non-existence of any listed output file compels Xcode to run your script phase.
  3. The number of input and output file paths is passed to the script as ${SCRIPT_INPUT_FILE_COUNT} and ${SCRIPT_OUTPUT_FILE_COUNT} environment variables.
  4. Each input and output path is passed as e.g. ${SCRIPT_INPUT_FILE_0}, ${SCRIPT_OUTPUT_FILE_1}, etc.

In practice, what does this mean when you go looking to speed up your script phase? It means you should:

  1. List as an input every file or folder that may affect the results of your script phase.
  2. List at least one output, even if your script doesn’t leave any reliable artifacts.

One frustrating bug I should point out here is that Xcode (as of at least Xcode 6.0 and including 6.2 betas) is not as attentive as it should be about changes to the the input and output file lists. If you make changes to these lists, rebuild several times, and don’t see the efficiencies you expect being applied, you should close and reopen your Xcode project document. It seems to cache its notion about script phase dependencies, and will stubbornly stick to those rules until forced to re-evaluate them from scratch. I reported this to Apple as Radar #19233769.

Referring back to the screenshot above, you can that in this contrived example I have listed two input “files,” but in this case they are actually directories. This is to illustrate that if you list a directory, Xcode will trigger a re-running of your script phase whenever any shallow change occurs in that directory. For example, if I add, delete, or modify any files in ImportantStuff”, the script will run on the next build. However, if I modify a file within “ImportantStuff/Goodies”, it will not trigger the script to run because Xcode does not monitor for changes recursively.

You will also see that I’ve gone to the trouble of artificially adding a bogus file to the derived files folder, indicating the script has run. That has the effect of causing a “clean” of the project to wipe out the artificial file and thus cause the script phase to run again even though none of the input files may have changed. If I wanted to ensure that the script phase is skipped as aggressively as possible, I could provide e.g. a path to an output file that is guaranteed to exist, such as a stable file in my project like “$(SRCROOT)main.m”. Because the designation “Output File” in this case is only a hint to the Xcode build system, you don’t have to worry (and can’t expect) that the file will be created or altered unless you do so yourself in the script. This feels a little dodgy to me though, because Apple could choose to change the behavior in the future to e.g. imply that “clean” should delete all listed output files for a script phase.

Script build phases are a woefully under-documented, incredibly useful component of the Xcode build system. They are especially useful when they run blazingly fast or are skipped completely if not needed. I hope this information gives you the resources you need to take full advantage of them.