Category Archives: Xcode

Optional Emptiness

iOS, ObjC, Swift, Xcode

Objective-C developers are comfortable with many idioms that fall out of the safety of messaging nil. For example, consider a chunk of Objective-C code that tests the “emptiness” of a UITextField string:

if (myTextField.text.length == 0) {
    // Do something
}

If myTextField.text is nil, what happens? In Objective-C, a message sent to nil will return nil, or zero, depending on the return type of the message. In this case “length” returns an integer, which happens to be zero when text is an empty string, and zero when text is nil. So this code block perfectly expresses “if the text is empty, do something.”

Adapting this code to Swift, you immediately run up against the language’s strict handling of optionals. Because myTextField.text might be nil, it has to be unwrapped before the length method can be called. This leads to less terse code such as:

if myTextField.text?.isEmpty != false {
    // Do something
}

This works! But it’s harder to read, and harder to reason. Similarly to the way the Objective-C version requires deep understanding of that language’s nil-messaging behavior, the Swift version requires deep understanding of optional chaining and comparison of optional and non-optional values. Here’s another example:

if (myTextField.text ?? "").isEmpty {
    // Do something
}

This is much easier to understand: use the non-nil String from myTextField, or else a constant string that is guaranteed to return true for isEmpty. It’s still more cumbersome than the original Objective-C, though.

In my own Swift adventures, I’ve addressed this using Swift’s powerful extension mechanism. It turns out that in Swift, any type that conforms to the “Collection” protocol implements an “isEmpty” method. String is one of these types. So with a small extension, we can add the “isEmpty” method not only to String? but to all optionals that wrap a collection:

extension Optional where Wrapped: Collection {
	public var isEmpty: Bool {
		switch(self) {
		case .none:
			return true
		case .some(let concreteSelf):
			return concreteSelf.isEmpty
		}
	}
}

With this extension in place, our test becomes:

if myTextField.text.isEmpty {
    // Do something
}

Which is both highly readable, behaves correctly when “text” is nil, and doesn’t require any deep language understanding to comprehend.

Thanks to Hwee-Boon Yar for the Objective-C scenario that motivated this post, and to Michel Fortin for putting forward the Swift equivalents cited above. This question came up in the Core Intuition Slack, where interesting discussions like this often take place. Join us!

Swift Integration Traps

Apple, Mac, Swift, Xcode

In the nearly four years since Swift was announced at WWDC 2014, Mac and iOS developers have embraced the language with decreasing reluctance. As language features evolve, syntax stabilizes, and tooling improves, it’s easier than ever to leap into full-fledged Swift development.

Several months ago, I myself made this leap. Although the vast majority of my Mac source base consists of Objective-C files, I have enjoyed adding new source files in Swift, and even converting key files to Swift either as an exercise, or when I think I will gain specific advantages.

One remaining challenge in Swift is the lack of ABI stability. In layperson’s terms: the lack of ABI stability prevents compiled Swift code from one version of the Swift compiler and runtime from linking with and running in tandem with Swift code compiled for another version.

For most developers, this limitation simply means that the entire Swift standard library, along with glue libraries for linking to system frameworks, needs to be bundled with the application that is built with Swift. Although it’s a nuisance that several megabytes of libraries must be added to every single Swift app, in the big scheme of things, it’s not a big deal.

A worse consequence is the number of pitfalls that ABI instability present, that are difficult to understand intuitively, and in many cases impossible, or at least dangerous, to work around. These pitfalls lie mainly in areas where developer code is executed on behalf of a system service, in a system process. In this context, it is not possible for developers to ensure that the required version of Swift libraries will be available to support their code. Game over.

On the Mac, system integration plugins are a typical scenario for this problem. While iOS has evolved with a strong architecture for running developer code in standalone, sandboxed processes, on the Mac there are still many plugins that run in a shared system process alongside code from other developers. These plugins run the gamut from arcane, rarely used functionality, to very common, user-facing features where a plugin is effectively required in order to satisfy the platform behaviors prescribed by Apple and expected by end-users.

One example on the more arcane, or at least inessential, end of the spectrum, is the Screen Saver plugin interface. Create a new project in Xcode, and choose the “Screen Saver Plugin” template as your starting point. Notice how unlike most templates, Xcode doesn’t even offer a choice of language. Your source files will be Objective-C. At least they’re giving you a hint here.

On the more mainstream end of the spectrum are plugins such as System Preferences panels and QuickLook Plugins. Depending on the type of app you are developing, it may be essential, or at least very well-advised to implement one of these types of plugins. So what do you do if you have an existing Objective-C app that you want to port to Swift, or you are writing a Swift app from scratch, and need to support one of these plugin formats? In the case of System Preferences panels at least, you have a couple practical options:

  1. Implement the plugin code, and all supporting code in Objective-C.
  2. Move the functionality out of System Preferences and into the host app.

Each of these could be somewhat reasonable approaches for a System Preferences plugin. The content of these plugins is often fairly straightforward, standard UI, and the goal is usually to collect configuration data to convey to the host application. It’s also not unreasonable, and may even be preferable to move such configuration code out of System Preferences and into a native panel inside the host app.

QuickLook Plugins are another beast. Because the goal of a QuickLook Plugin is usually to convey a visual depiction of a native document type, it’s exceedingly common to take advantage of the very classes that present the document natively in the host app. Let’s say you’ve written an app in Swift, FancyGraphMaker. Apple encourages you to implement a QuickLook Plugin so that users will be able preview the appearance of your fancy graphs, both in the dedicated QuickLook interface, and by way of more unique looking icons in the Finder.

But once you’ve written the code to draw those fancy graphs in Swift, you’re locked out of using that code from a QuickLook Plugin. Worse? Finishing touches such as supporting Quick Look are liable to come later in the development of an app, so you’ve probably gone through the decision-making process of writing your app in Swift, before realizing that the decision effectively cuts you off from a key system feature. That’s a Swift Integration Trap.

Although the workarounds are not as straight-forward in this scenario as they are for a System Preferences pane, it is probably still technically possible to leverage Swift code in the implementation of a QuickLook Plugin. I have not tested this, but I imagine such a plugin could spawn an XPC process that is itself implemented in Swift and executes the bulk of the preview-generation work on behalf of the system-encumbered plugin code. The XPC process would be free to link to whatever bundled Swift libraries it requires, generate the desired preview data, and message it back to the host process. At least, I think that would work.

But I shouldn’t have to think that hard to get this to work, nor should any other developer. The problem with these Swift Integration Traps is twofold:

  1. If you don’t know about them, you end up stuck, potentially regretting the decision to move to Swift.
  2. If you do know about them, you might put off adopting Swift completely, or at least put off converting classes that are pertinent to QuickLook preview generation.

Each of these consequences is bad for developers, for users, and for Apple. Developers face a trickier decision process about whether to move to Swift, users face potential integration shortcomings for Swift-based apps, and Apple suffers either reduced adoption of Swift, reduced integration with system services, or both.

I filed Radar #38792518 requesting that QuickLook Plugins be supported by the App Extension model. Essentially, this would formalize the process of putting the generation code in a separate XPC process, as I speculated above would work around the problem. The App Extension system is designed to support, and in fact requires this approach. The faster Apple moves QuickLook Plugins, and other shared-process plugins to the App Extension model, the fast developers can embrace Swift with full knowledge that their efforts to integrate with the system will not be stymied.

Update: Thanks to a hint from Chris Liscio, I have learned that Apple has in fact made some progress on the QuickLook front, but it won’t help the vast majority of cases in which a QuickLook Plugin is used to provide previews for custom file types. It took me a while to hunt this down because it not very clearly documented, and Google searches do not lead to information about it.

At WWDC 2017, Apple announced support for a new QuickLook Preview Extension. It escaped my notice even while ardently searching for evidence of such a beast, because the news was shared in the What’s New in Core Spotlight session. Making matters worse, the term “QuickLook” does not appear once in the session transcript, although it turns out that “Quick Look” appears many times:

Core Spotlight is also coming to macOS and just like on iOS you can customize your preview. On macOS a preview is shown when you select a search result in the Spotlight window. Here you really do want to implement a Quick Look preview extension for your Core Spotlight item because Spotlight on macOS does not have a default preview.

Ooh, this sounds exciting! I’ve wondered over the years why such similar plugins, Spotlight importers, and QuickLook generators, shouldn’t be unified. Although the WWDC presentation emphasizes substantial parity in behavior for QuickLook Previews between iOS and macOS, there is a major gotcha:

Core Spotlight is great for databases and shoeboxes where your app has full control over the contents.
It’s not for items that the user monitors in the finder, for that the classic Spotlight API still exists and still works great.

I beg to differ with that “still works great” assessment, at least in the context of this post. Mac developers who want to integrate with QuickLook must still use a shared-process plugin. It’s still a Swift Integration Trap.

IDEBundleInjection Signing Failure

Debugging, Mac, Xcode

When a unit test bundle is built to be dynamically injected into a host app, Xcode performs a little dance at build time, in which it adds its own IDEBundleInjection.framework to the bundle, then re-signs it with the developer’s code signing identity.

Normally this all goes off without a hitch, but today when I went to build and test such a bundle, I was met with a rude code signing failure:

IDEBundleInjection.framework: unsealed contents present in the root directory of an embedded framework

I took all the usual steps when facing an obtuse error: clean the build directory, quit and restart Xcode, etc. Nothing fixed it, so I thought perhaps it was an issue with the 9.3 beta Xcode I was running. Nope. Same problem with 9.2. Finally, I made my own copy of the framework in question, and ran “codesign” against it myself from the Terminal. Same error!

This framework, stored within Xcode itself, has become unsignable. Running “codesign -v” against the framework in place also confirms that the code signing seal has been broken. What happened to my Xcode?

It occurred to me that I recently migrated from one Mac to another, and copied my Xcode when I did. I tried to use the Apple-standard migration assistant, but it failed, so I ended up using Finder, or ditto from the Terminal, to copy everything over. Maybe something was messed up in the transition?

The codesign utility is useful for letting me know that something is wrong, but doesn’t actually do me the favor of telling me what it is! Luckily, I have a backup of my whole disk and the original Xcode on that volume appear to have properly signed internal frameworks. Running a diff on IDEBundleInjection.framework between the two copies, I do see some reported distinctions. Where “.” is the current, misbehaving framework:

Only in .: .BC.D_QdfhyO
Only in .: .BC.D_mgLUu2
Only in ./Versions: .BC.D_gSVCxT

These appear to be redundant cruft correlating to the expected internal version links. For every link like:

IDEBundleInjection -> Versions/Current/IDEBundleInjection

I have one of these unexpected garbage links. The presence of these links are, of course, detected by codesign, and it throws everything off.

I don’t know why these mysterious gremlin files showed up on my Mac, but whatever the cause, there’s an easy solution. I’m taking a leap of faith that I don’t actually want any of these files:

cd /Applications/Xcode.app
find . -name ".BC.*" -delete

And now I can get back to unit testing my app.

Xcode’s Secret Performance Tests

Apple, Hacking, Swift, Testing, Xcode

I was inspired today, by a question from another developer, to dig into Xcode’s performance testing. This developer had observed that XCTestCase exposes a property, defaultPerformanceMetrics, whose documentation strongly suggests can be used to add additional performance metrics:

This method returns XCTPerformanceMetric_WallClockTime by default. Subclasses of XCTestCase can override this method to change the behavior of measureBlock:.

If you’re not already familiar, the basic approach to using Xcode’s performance testing infrastructure is you add unit tests to your project that wrap code with instructions to measure performance. From the default unit test template:

func testPerformanceExample() {
	// This is an example of a performance test case.
	self.measure {
		// Put the code you want to measure the time of here.
	}
}

Depending on the application under test, one can imagine all manner of interesting things that might be useful to tabulate during the course of a critical length of code. As mentioned in the documentation, “Wall Clock Time” is the default performance metric. But what else can be measured?

Nothing.

At least, according to any header files, documentation, WWDC presentations, or blunt Googling that I have encountered. There is exactly one publicly documented Xcode performance testing metric, and it’s XCTPerformanceMetric_WallClockTime.

I was curious whether supporting additional, custom performance metrics might be possible but under-documented. To test this theory, I added “beansCounted” to the list of performance metrics returned from my XCTestCase subclass. For some reason I couldn’t get Swift to accept the XCTPerformanceMetric pseudo-type, but it allowed me to override as returning an array of String:

override static func defaultPerformanceMetrics() -> [String] {
	return ["beansCounted"]
}

When I build and test, this fails with a runtime exception “Unknown metric: beansCounted”. The location of an exception like this is a great clue about where to go hunting for information about whether an uknown metric can be made into a known one! If there’s a trick to implementing support for my custom “beansCounted” metric, the answer lies in the method XCTestCase’s “measureMetrics(_: automaticallyStartMeasuring: forBlock:)”, which is where the exception was thrown.

By setting a breakpoint on this method and stepping through the assembly in Xcode, I can watch as the logic unfolds. To simplify what happens: first, a list of allowable metrics is computed, and then the list of desired metrics is iterated. If any metric is not in the list? Bzzt! Throw an exception.

I determined that things are relatively hardcoded such that it’s not trivial to add support for a new metric. I was hoping I could implement some magic methods in my test case, like “startMeasuring_beansCounted” and “stopMeasuring_beansCounted”

but that doesn’t appear to be the case. The performance metrics are supported internally by a private Apple class called XCTPerformanceMetric, and the list of allowable metrics is derived from a few metrics hardcoded in the “measureMetrics…” method:

  • “com.apple.XCTPerformanceMetric_WallClockTime”
  • “com.apple.XCTPerformanceMetric_UserTime”
  • “com.apple.XCTPerformanceMetric_RunTime”
  • “com.apple.XCTPerformanceMetric_SystemTime”

As well as a bunch of others exposed by a private “knownMemoryMetrics” method:

  • “com.apple.XCTPerformanceMetric_TransientVMAllocationsKilobytes”
  • “com.apple.XCTPerformanceMetric_TemporaryHeapAllocationsKilobytes”
  • “com.apple.XCTPerformanceMetric_HighWaterMarkForVMAllocations”
  • “com.apple.XCTPerformanceMetric_TotalHeapAllocationsKilobytes”
  • “com.apple.XCTPerformanceMetric_PersistentVMAllocations”
  • “com.apple.XCTPerformanceMetric_PersistentHeapAllocations”
  • “com.apple.XCTPerformanceMetric_TransientHeapAllocationsKilobytes”
  • “com.apple.XCTPerformanceMetric_PersistentHeapAllocationsNodes”
  • “com.apple.XCTPerformanceMetric_HighWaterMarkForHeapAllocations”
  • “com.apple.XCTPerformanceMetric_TransientHeapAllocationsNodes”

How interesting! There are a lot more metrics defined than the single “wall clock time” exposed by Apple. So, should we use them? Official answer: no way! This is private, unsupported stuff, and can’t be relied upon. Punkass Daniel Jalkut answer? Why not? They’re your tests, and your the only one who will get hurt if they suddenly stop working. In my opinion taking advantage of private, undocumented system behavior for private, internal gain is much different than shipping public software that relies upon such undocumented behaviors.

I modified my unit test subclass to return a custom array of tests based on the discoveries above, just to test a few:

override static func defaultPerformanceMetrics() -> [String] {
	return [XCTPerformanceMetric_WallClockTime, "com.apple.XCTPerformanceMetric_TransientHeapAllocationsKilobytes", "com.apple.XCTPerformanceMetric_PersistentVMAllocations", "com.apple.XCTPerformanceMetric_UserTime"]
}

The tests build and run with no exception. That’s a good sign! But these “secret peformance tests” are only useful if they can be observed and tracked the way the wall clock time can be. How does Xcode hold up? I made my demonstration test purposefully impactful on some metrics:

func testPerformanceExample() {
	self.measure {
		for _ in 1..<100 {
			print("wasting time")
		}
		let _ = malloc(3000)
	}
}

Now when I build and test, look what shows up in the Test navigator’s editor pane:

Screenshot of performance metrics after reducing the size of allocations and length of run.

Look at all those extra columns! And if I click the “Set Baselines…” button, then tweak my function to make it substantially less performant:

func testPerformanceExample() {
	self.measure {
		for _ in 1..<10000 {
			print("wasting time")
		}
		let _ = malloc(300000)
	}
}

Now the columns have noticably larger numbers:

Screenshot of Xcode's test results after running tests with

But more importantly, the test fails:

Screenshot of test errors generated by failing to meet performance baselines.

I already mentioned that by any official standard, you should not take advantage of these secret metrics. They are clearly not supported by Apple, may be inaccurate or have bugs, and could outright stop working at any time. I also said that, in my humble opinion, you should feel free to use them if you can take advantage of them. The fact that they are supported so well in Xcode probably implies that groups internal to Apple are using them and benefiting from them. Your mileage may vary.

The only rule is this: if Apple does do anything to change their behavior, or you otherwise ruin your day by deciding to play with them, you shouldn’t blame Apple, and you can’t blame me!

Enjoy.

Playground Graphs

iOS, Mac, Swift, Xcode

I was playing around with the Swift standard library’s “map” function, when I noticed a cool feature of Xcode Playgrounds. Suppose you are working with an array of numbers. In the Xcode Playgrounds “results” section, you can either click the Quick Look “eye” icon, or click the little results rectangle to get an inline results view of the expression you’re viewing:

Screenshot of Xcode Playgrounds's inline results view, revealing the values of an array of numb ers.

The linear list of values is revelatory and easy to read, but wouldn’t it be easier to understand as a graph? It turns out simply passing these values through the map function does just that:

Screenshot of the Xcode Playgrounds's result of the map function when returning numeric values.

I thought I had stumbled on some magical secret of Xcode, but it turns out the behavior is well documented, and applies to more than just the “map” function. You can even grab the edges of the result view and resize it to better suit your data. In fact, any looping numeric value seems to trigger the availability of this handy graphing functionality:

Screenshot of Xcode Playgrounds showing a graph of the results of Fibonacci sequence.

I am still frustrated by a lot of behaviors of Xcode Playgrounds, but little gems like these are nice to stumble upon.

Unified Swift Playgrounds

OS X, Swift, Xcode

The announcement of Swift Playgrounds 2.0 has me thinking again about Xcode Playgrounds: both about what a revelation they are, and about how disappointing they continue to be.

When Xcode Playgrounds were first introduced (as “Swift Playgrounds”) in 2014, they were received as a groundbreaking new way for developers to write Swift code interactively. There were lots of rough edges on the feature, but it seemed reasonable to expect that because they were released in tandem with the Swift programming language itself, those rough edges would be smoothed out on a parallel pace with the language itself.

Two years later, Apple announced Swift Playgrounds again, immediately introducing a nomenclature confusion. This time the name referred to a dedicated, closed-source iOS app designed for interactively teaching programming concepts with Swift. The previous Xcode-coupled technology, now known as “Xcode Playgrounds” or simply “Playgrounds,” had seen modest improvements over the years but continued to be frustratingly slow, unpredictable, and crash-prone.

Today, in early 2018, the release of Swift Playgrounds 2.0 for iOS appears to represent Apple’s commitment to driving that product forward into the future. The latest version of Xcode Playgrounds, on the other hand, offers a lackluster interface, slow responsiveness, and a tendency to crash both within the Playground, and in ways that take down the entire Xcode app. In short: they’re not a very fun place to play.

I propose that Apple eliminate Xcode playgrounds, and invest all of their work in the field of interactive coding into the Swift Playgrounds app. It has been a nagging shortcoming that Swift Playgrounds is only available for iOS. Many people who would benefit from the educational opportunities of Swift Playgrounds could do so from Macs, whether in schools, homes, or workplaces. Porting Swift Playgrounds to the Mac would address that problem.

Where does that leave developers? After eliminating Xcode Playgrounds as we know them today, I envision adapting the Mac version of Swift Playgrounds so that playgrounds can be run either independently in a Swift Playgrounds app, or in “developer mode” within Xcode. In effect, Xcode would become a dedicated Swift Playgrounds authoring app, where such authoring capabilities would incidentally provide all the benefits that standalone Xcode Playgrounds currently provide.

Taking this course would allow Apple to maximize the output of its engineering and design efforts while eliminating the naming confusion that currently exists between Swift Playgrounds and Xcode Playgrounds.

For students and educators, it would broaden device requirements for Swift Playground materials, opening up learning opportunities for people who have access to Macs but not to iOS devices.

Finally, and perhaps most importantly for the developer ecosystem as a whole, it would eliminate the frustratingly problematic Xcode Playgrounds and hopefully provide developers with something more inspiring, more functional, and more reliable.

Radar #36910249.

Treat Warnings as Errors in Swift

Bug Reports, Xcode

For years I have maintained a zero-tolerance policy for warnings in shipping code. To help me enforce this, my “Release” build configurations define the build setting that induces Xcode to “treat warnings as errors”:

GCC_TREAT_WARNINGS_AS_ERRORS = YES

The comedy of this build setting is that it references “GCC,” a compiler that increasingly few of us even remember Apple ever using for macOS or iOS development. Apple’s fork of the popular open-source compiler was the standard for years, until Apple debuted clang, which is built on the LLVM framework.

It’s always been kind of annoying that, as time moved on, we were stuck specifying important build settings with names that included “GCC,” but I accepted it. After all, I configure the vast majority of these settings in “.xcconfig” files that I reuse in all my projects, so I almost never interact with them directly.

As I’ve started to shift some of my code from Objective-C to Swift, I assumed that my ancient GCC build setting would ensure that I don’t ship any builds with warnings. But today I realized that I had built a release build that didn’t treat a warning as an error. What’s the deal?

It looks like Apple has decided to break with tradition and establish a new build setting for Swift:

SWIFT_TREAT_WARNINGS_AS_ERRORS = YES

Adding this to my .xcconfig file breaks the build when a warning is generated, and prevents me from accidentally shipping code that is known to be vulnerable to unexpected behavior. I think it would have been a nice touch if Apple had inferred SWIFT_TREAT_WARNINGS_AS_ERRORS when GCC_TREAT_WARNINGS_AS_ERRORS is set, so I’ve filed a bug requesting that it does. Radar #35352318.

Xcode 9 Signing Workarounds

Apple, Xcode

I wrote on Monday about issues with Xcode 9 relating to code signing. Although the gist of that post involved sandboxed Mac applications that launch sandboxed child processes, the fundamental issue is a bit broader: Xcode 9 adds a “com.apple.security.get-task-allow” entitlement to any binary it signs. For the majority of developers, this is probably not an issue, because the entitlement is removed when an Xcode archive is exported for distribution. Most developers, and particularly iOS developers, use Xcode archives.

For folks who don’t, side effects of this additional entitlement include, but may not be limited to:

  1. Inability to launch sandboxed child processes.
  2. Rejection from the Mac App Store.
  3. Unknown consequences of shipping with an unintended entitlement.

So, if you’re a developer who doesn’t use archives, what are your options? I’ve come up with four workarounds, and I present them here, roughly sorted by advisability and level of tedium:

  1. Use Xcode 8. The simplest solution is to not upgrade to Xcode 9 unless and until you need to. Xcode 8’s signing process does not impose the unintended entitlement, so there is no risk of shipping a product that has it, unless you add it yourself. The downside to sticking with Xcode 8 is you won’t enjoy any of the new features of Xcode 9, you’ll have to work to support either Swift 4, macOS 10.13, or iOS 11 SDK features in your app.

  2. Manually re-sign the built-product. Code signing is code signing, and you’re free to sign anything you like to suit your needs, using the “codesign” command line tool. It frankly sounds like a pain in the neck to recursively re-sign every binary in the app bundle, ensuring that the suitable entitlements (minus the unwanted one) are preserved, but I’m sure it can be done.

  3. Use Xcode archives. It strikes me as a little obnoxious to have to use Xcode archives when they don’t offer any added benefits for my dibstrution workflow. But as a long term solution, this is probably the safest bet. The new behavior in Xcode 9 strongly suggests that Apple expects most developers to use archives, and joining the crowd is usually a good idea when it comes to avoiding trouble with Apple’s developer tools.

    If you are using Xcode archives for the first time, particularly with a complex project, you might discover that the resulting archives are not suitable for exporting a signed application. If you get a “Generic Xcode Archive” after running Build -> Archive, you know you’ve got a problem. By default the archive process builds all targets with an “install” option, rendering their built products into a file hierarchy that will be used to build the archive. If your project includes helper apps, for example, they will be “installed” alongside your main app, resulting in a generic archive of two apps, instead of the expected archive of a single app.

    The solution for this problem is to ensure that the “SKIP_INSTALL” build setting is set to YES for any such helper app. Just archive your main app, export the “Built Products” from the resulting archive, and look at the file hierarchy to determine whether you have subtargets that need to have installation disabled.

  4. Hack Xcode 9. In a hurry to ship an update to your app, and you’ve only got Xcode 9 handy? It turns out the imposition of this “com.apple.security.get-task-allow” entitlement is controlled by a single property list file inside Xcode’s application bundle. As a test, I edited the file:

    Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/Library/Xcode/PrivatePlugIns/IDEOSXSupportCore.ideplugin/Contents/Resources/BaseEntitlements.plist

    It contains a single entitlement, the one that’s causing our grief. I deleted the entitlement from the list, saved the file, and relaunched Xcode. After doing so, everything is “back to normal.”

    I can’t strongly encourage you to hack your copy of Xcode because I don’t know what the consequences might be. “It seems fine,” but you’re on your own if you decide to do this.

This small change in Xcode 9 causes a lot of unexpected grief for folks who don’t use Xcode archives. I am curious to know how widespread the problem is, and enthusiastic to get the word out about it so that affected folks can work around the problem, or at least be aware of it. Myself, I’ll probably end up adopting the workaround of using Xcode archives, but I’m hopeful that Apple will see the merit of providing an option in an update to Xcode 9 that supports disabling the addition of this entitlement without archiving and exporting a built product.

Sandbox Inheritance Tax

Apple, Bug Reports, Xcode

I ran into a subtle bug with Xcode 9 that I think is worth sharing. Specifically, this bug affects Mac applications that:

  1. Are sandboxed.
  2. Launch a sandboxed subprocess with NSTask (or posix_spawn).
  3. Configure the subprocess to inherit the parent’s sandbox.

When such an app is compiled with Xcode 9, the subprocess will crash whenever the parent process launches it. A canonical example of something that might suffer from this problem is a bundled crash-monitor. I embed one with my apps to keep an eye on the running status of the parent process, and to present a crash-reporting interface to users if the host app terminates prematurely. When I build and run my app with Xcode 9, the bundled crash monitor dies instantly upon being launched.

It took me a while to realize that the subprocess is dying because it fails to satisfy the contract for inheriting a sandbox. From Apple’s “Enabling App Sandbox Inheritance“:

To enable sandbox inheritance, a child target must use exactly two App Sandbox entitlement keys: com.apple.security.app-sandbox and com.apple.security.inherit. If you specify any other App Sandbox entitlement, the system aborts the child process.

Well, that’s funny because my child process does specify only those two keys, but the system is aborting it anyway. It turns out that Xcode 9 is inserting a third entitlement without my permission. Clicking on the detail of the “Process Product Packaging” build phase in Xcode’s log navigator, I can see that there are three entitlements for my target:

Xcode build log detail showing the wrong entitlements.

When my subprocess is launched, the system sees that extra “com.apple.security.get-task-allow” entitlement in the context of “com.apple.security.inherit”, and unceremoniously crashes my the child process.

I’m not sure what Apple’s reasoning is for imposing this entitlement on sandboxed targets, but it appears to be doing so across the board, for literally every sandboxed target in my app. I confirmed that all of my apps, XPC processes, helper tools, etc., are all getting this bonus entitlement.

I searched Xcode’s files, and discovered the entitlement listed in this file inside the Xcode app bundle:

Contents/Developer/Platforms/MacOSX.platform/Developer/Library/Xcode/PrivatePlugIns/IDEOSXSupportCore.ideplugin/Contents/Resources/BaseEntitlements.plist

Putting aside the question of whether it’s appropriate for Xcode to surreptitiously add entitlements that are not specified by the developer’s own list of permissions, the addition of the entitlement for these particular targets, ones that inherit their parent’s sandbox, turns out to be a fatal move.

Ideally I would be able to work around this by adding a custom build phase to manually tweak the generated entitlements file, removing the unwanted key. But the “Process Product Packaging” build phase happens so late in the build process that it’s after the last user-specified custom build phase. There’s no room in Xcode’s current design for fixing up the problematic entitlements before they are incorporated into the signed product. As far as I can tell the only clean workaround would be to redundantly re-sign the child app with a custom script, and corrected entitlements, after Xcode’s build process is completed.

I filed Radar #34628449, “Sandboxed project build with Xcode 9 cannot launch child process.”

Update: Colin Barrett pointed out on Twitter that the entitlement in question here, “com.apple.security.get-task-allow”, may be required in order to attach to and debug a process. If true, then I think this is something that was handled in a different way in Xcode 8. I can confirm that my apps do not have the entitlement imposed on them by Xcode 8, yet I am able to attach to and debug them.

If Apple changed the debugger infrastructure in Xcode 9 so that the relationship between the debugger and target processes is more locked down, requiring a specific entitlement, then that’s probably a good thing. But if this change was made without thinking about the implications for the above-cited “strict two entitlement” rule for sandbox inheritance, then probably some flexibility needs to be applied to that rule.

Finally, as I noted above the entitlement is being applied to all my targets. What I didn’t clarify is that the entitlement is added even when Building and Archiving. A release build’s binaries are endowed with this additional entitlement, which may also bring additional security vulnerabilities to the app.

I would not ship a sandboxed Mac app that is built with Xcode 9, until we understand more about when Xcode applies this entitlement, and whether it can be prevented for Release builds at the very least.

Update 2: I’ve learned that Xcode’s “Export Archive” functionality causes the unwanted entitlement to be removed. Apparently the assumption is that everybody creates Xcode archives as part of their build and release process. I am sure this is true for most (all?) iOS deployments, but for Developer-ID signed apps on the Mac, there has traditionally been less of an incentive to do this. Got a properly signed Mac application? Zip it up, put it on a web server, and you’re done.

I’m not sure yet whether I’ll switch my build process to use archiving, or whether I’ll pull some other stunt to redo the code signing with corrected entitlements. In any case this has been quite an adventure today getting to the bottom of this. I updated my bug report with Apple to request that they provide some standard build flag that would prevent the problematic entitlement from being added from the start. In the mean time, I’ll explore one of the workarounds and get my builds back to fully functional!

Better Swift Completion

Bug Reports, ObjC, Swift, Xcode

Apple released Xcode 9 earlier this week, and in spite of a few glitches here and there, I have found the update to be an overall improvement over Xcode 8. It’s nice that Apple continues to invest in the core tools for Mac and iOS developers.

I’ve been dabbling in more and more Swift development lately, and it’s brought to light a shortcoming in Xcode’s code completion which has unfortunately not improved in Xcode 9: completion of Swift function calls when there is a large quantity of candidates.

Take for example NSAttributedString. If I want to initialize a new instance in Swift, I type “NSAttributedString(” to bring up the list of compatible init methods I can choose from:

SwiftCompletion

The problem at this point is that I have to navigate the menu by hand. I can’t narrow down the list of completions any further by typing, because the very next character I type will be interpreted as the manual filling out of parameters of the NSAttributedString initializer.

BadCompletion

This is a situation where Objective-C gets much nicer treatment in the editor. Because completion in Objective-C begins when I start typing “init”, and because the named first parameter is part of the init message name, I can winnow down the results quite a bit:

Pasted Image 9 22 17 11 24 AM

Better still, because Xcode performs a fuzzy match on the typing, I can proceed to type the names of additional parameters to zero in completely on the variation I want:

MEAppController AppDelegate m Edited

When I accept the completion, all of my typing is replaced with the expected, templated parameter placeholders for the chose initializer.

I filed Radar #34594940 requesting better completion for Swift.