Category Archives: Apple

Sandbox Inheritance Tax

Apple, Bug Reports, Xcode

I ran into a subtle bug with Xcode 9 that I think is worth sharing. Specifically, this bug affects Mac applications that:

  1. Are sandboxed.
  2. Launch a sandboxed subprocess with NSTask (or posix_spawn).
  3. Configure the subprocess to inherit the parent’s sandbox.

When such an app is compiled with Xcode 9, the subprocess will crash whenever the parent process launches it. A canonical example of something that might suffer from this problem is a bundled crash-monitor. I embed one with my apps to keep an eye on the running status of the parent process, and to present a crash-reporting interface to users if the host app terminates prematurely. When I build and run my app with Xcode 9, the bundled crash monitor dies instantly upon being launched.

It took me a while to realize that the subprocess is dying because it fails to satisfy the contract for inheriting a sandbox. From Apple’s “Enabling App Sandbox Inheritance“:

To enable sandbox inheritance, a child target must use exactly two App Sandbox entitlement keys: com.apple.security.app-sandbox and com.apple.security.inherit. If you specify any other App Sandbox entitlement, the system aborts the child process.

Well, that’s funny because my child process does specify only those two keys, but the system is aborting it anyway. It turns out that Xcode 9 is inserting a third entitlement without my permission. Clicking on the detail of the “Process Product Packaging” build phase in Xcode’s log navigator, I can see that there are three entitlements for my target:

Xcode build log detail showing the wrong entitlements.

When my subprocess is launched, the system sees that extra “com.apple.security.get-task-allow” entitlement in the context of “com.apple.security.inherit”, and unceremoniously crashes my the child process.

I’m not sure what Apple’s reasoning is for imposing this entitlement on sandboxed targets, but it appears to be doing so across the board, for literally every sandboxed target in my app. I confirmed that all of my apps, XPC processes, helper tools, etc., are all getting this bonus entitlement.

I searched Xcode’s files, and discovered the entitlement listed in this file inside the Xcode app bundle:

Contents/Developer/Platforms/MacOSX.platform/Developer/Library/Xcode/PrivatePlugIns/IDEOSXSupportCore.ideplugin/Contents/Resources/BaseEntitlements.plist

Putting aside the question of whether it’s appropriate for Xcode to surreptitiously add entitlements that are not specified by the developer’s own list of permissions, the addition of the entitlement for these particular targets, ones that inherit their parent’s sandbox, turns out to be a fatal move.

Ideally I would be able to work around this by adding a custom build phase to manually tweak the generated entitlements file, removing the unwanted key. But the “Process Product Packaging” build phase happens so late in the build process that it’s after the last user-specified custom build phase. There’s no room in Xcode’s current design for fixing up the problematic entitlements before they are incorporated into the signed product. As far as I can tell the only clean workaround would be to redundantly re-sign the child app with a custom script, and corrected entitlements, after Xcode’s build process is completed.

I filed Radar #34628449, “Sandboxed project build with Xcode 9 cannot launch child process.”

Update: Colin Barrett pointed out on Twitter that the entitlement in question here, “com.apple.security.get-task-allow”, may be required in order to attach to and debug a process. If true, then I think this is something that was handled in a different way in Xcode 8. I can confirm that my apps do not have the entitlement imposed on them by Xcode 8, yet I am able to attach to and debug them.

If Apple changed the debugger infrastructure in Xcode 9 so that the relationship between the debugger and target processes is more locked down, requiring a specific entitlement, then that’s probably a good thing. But if this change was made without thinking about the implications for the above-cited “strict two entitlement” rule for sandbox inheritance, then probably some flexibility needs to be applied to that rule.

Finally, as I noted above the entitlement is being applied to all my targets. What I didn’t clarify is that the entitlement is added even when Building and Archiving. A release build’s binaries are endowed with this additional entitlement, which may also bring additional security vulnerabilities to the app.

I would not ship a sandboxed Mac app that is built with Xcode 9, until we understand more about when Xcode applies this entitlement, and whether it can be prevented for Release builds at the very least.

Update 2: I’ve learned that Xcode’s “Export Archive” functionality causes the unwanted entitlement to be removed. Apparently the assumption is that everybody creates Xcode archives as part of their build and release process. I am sure this is true for most (all?) iOS deployments, but for Developer-ID signed apps on the Mac, there has traditionally been less of an incentive to do this. Got a properly signed Mac application? Zip it up, put it on a web server, and you’re done.

I’m not sure yet whether I’ll switch my build process to use archiving, or whether I’ll pull some other stunt to redo the code signing with corrected entitlements. In any case this has been quite an adventure today getting to the bottom of this. I updated my bug report with Apple to request that they provide some standard build flag that would prevent the problematic entitlement from being added from the start. In the mean time, I’ll explore one of the workarounds and get my builds back to fully functional!

JavaScript OSA Handler Invocation

Apple, Automation, Bug Reports, OS X

When Apple added support to macOS to support JavaScript for Automation, they did so in a way that more or less allows folks who invoke AppleScripts to invoke JavaScript for Automation scripts as if they were exactly the same. An abstraction in Apple’s Open Script Architecture (OSA) makes it easy for script-running tools to theoretically handle any number of scripting languages without concern for the implementation details of those languages.

This mostly works, but I recently received a bug report that shed light on a problem with Apple’s implementation of JavaScript with respect to invoking a specific named handler. The OSA provides a mechanism for loading and running a specific handler, or function, within a script. My app FastScripts takes advantage of this to query a script about whether it would prefer to be invoked in another process or not. Unfortunately, when it comes to JavaScript, Apple’s implementation runs the whole script in addition to running just the specific, named handler.

If you’ve got Xcode handy, you can use this simple playground content to observe the problem:

import OSAKit

if let javaScriptLanguage = OSALanguage(forName: "JavaScript") {
   let scriptSource = "Application('Safari').activate();" +
         "function boo() { ObjC.import('Cocoa'); $.NSBeep(); }"
   let myScript = OSAScript(source: scriptSource, language: javaScriptLanguage)

  // Only the behavior of boo should be observed
  myScript.executeHandler(withName: "boo", arguments: [], error: nil)
}

// Give time for the beep to sound
RunLoop.current.run(until: Date(timeIntervalSinceNow:5))

The named function “boo()” only invokes NSBeep, so when this playground is run, all that should happen is a beep should be emitted from the Mac. Instead, when it runs Safari becomes the active application. This is because in addition to running the “boo()” handler, it also runs the whole script at the top level.

A workaround to the bug is to wrap the top level functionality of a script in a “run()” handler, so where the scriptSource is declared above, instead use:

   let scriptSource = "function run() { Application('Safari').activate(); }" +
         "function boo() { ObjC.import('Cocoa'); $.NSBeep(); }"

I hope this helps the one other person on earth who cares about invoking JavaScript for Automation methods indvidually! (Radar #33962901, though I’m not holding my breath on this one!)

Xcode GitHub Integration

Apple, Bug Reports, Open Source, Xcode

Apple’s beta release of Xcode 9 features impressive improvements to its source control features, including streamlined integration with GitHub. There’s even a fancy “Open in Xcode” button when you go to clone a project:

Screen capture of the GitHub interface for cloning a project

This integration is amazing. You just click the button, specify a save folder in Xcode, and boom! You’re off and …

Screen capture of build failure indicating a missing signing certificate

Oh, right. Code signing. The otherwise stellar GitHub integration in Xcode underscores a longstanding deficiency in how it manages code signing identities for multi-team, collaborative projects. Precisely the kinds of projects you’re liable to find on GitHub.

The problem could be solved, or at least diminished greatly, by providing some mechanism for declaring that a project should be code signed “with the user’s own default developer team.” The default branch of any open source project targeting Apple platforms, would specify the DEVELOPMENT_TEAM as something like:

DEVELOPMENT_TEAM = Automatic

Xcode would provide a user-level setting for “Default Development Team”, and in the absence of any overriding setting, that team would be used whenever a project was configured as above.

I wrote about this problem once before, but with all the work being put into streamlining the experience of cloning from and pushing to GitHub, now is an ideal time for Apple to embrace a fix. Radar #32614751.

Another issue that stops short the cloning, and immediate building and running, of open source projects, is the need to fulfill external dependencies. In some cases this might require manually downloading and installing libraries, or cloning projects, but in the vast majority of cases the dependencies will be specified using built-in Git submodule support, or a popular package manager. In each of these cases, it should be trivial for Xcode to detect that the project it has just cloned also has dependencies:

  • Git submodules: there is a .gitmodules directory.
  • Carthage: there is a Cartfile file.
  • CocoaPods: there is a Podfile file.
  • Swift Package Manager: there is a Swift.package file.

If Xcode sees evidence of any of these techniques at play, it could do the favor of checking them out immediately after cloning the project. Radar #32615265.

The GitHub integration coming in Xcode 9 provides a nearly effortless capability for cloning, building, and running open source projects that target Apple platforms. Ideally it would also go the extra mile and provide for variable, dynamic development teams, as well as conduct a rudimentary check for dependencies that must be checked out before commencing work on the project.

Window Tabbing Pox

Apple, Bug Reports, OS X

macOS Sierra introduces a new system-wide window tabbing feature, which by default enables tabs within the windows of most document-based apps. Apple’s own TextEdit is a canonical example: open the app and select View -> Show Tab Bar to see how it looks.

Unfortunately, the default tabbing behaviors doesn’t make sense in all apps, even if they are document-based. I’ve had to disable the functionality in both MarsEdit and Black Ink, at least for the time being. Doing so is easy. Just add the following line somewhere early in the lifetime of your app:

NSWindow.allowsAutomaticWindowTabbing = NO;

This class property on NSWindow shuts the whole window tabbing system out of your app, so you don’t have to fuss with how to disable it on a window-by-window basis. Handy!

Unfortunately, setting that property to NO doesn’t do anything about windows that may already have been created, and which have their tab bar visible. I added the line to my app delegate’s applicationDidFinishLaunching: method, assuming that would take care of everything. After all, any documents will be instantiated later in the app’s lifetime, right? Right?

Wrong. Not with window restoration, at least, which causes a freeze-drying and restoration of open document windows when a user quits and relaunches the app. The window in this circumstance is actually created before applicationDidFinishLaunching. If it had its tab bar visible when you quit, it will have the tab bar visible when it’s restored, even if you’ve since disabled all window tabbing for the app.

What’s worse? Showing or hiding the tab bar on any window sets that choice as the default for all future windows in the app. So even new documents that are created by users, and which don’t have their tab bar visible because you’ve disabled it app-wide, will have a tab bar appended when they are restored at launch time, because “Show Tab Bar” was the last user action before disabling tabbing altogether.

The long and short of it? An app stuck in this situation will not have a View -> Show/Hide Tab Bar, and none of its windows will support tabbing, except for any document that is restored at launch time. Even new documents that are created without tab bars will have the tab bar imposed the next launch.

I filed Radar 28578742, suggesting that setting NSWindow.allowsAutomaticWindowTabbing=NO should also turn off window tabbing for any open windows that have it enabled.

If your app gets stuck in this predicament, one workaround is to move the NSWindow.allowsAutomaticWindowTabbing=NO line to an even earlier point in your app’s lifetime, such as in your app’s main.m or main.swift file.

Xcode 6 On Sierra

Apple, Hacking, OS X, Xcode

Xcode 6 and Xcode 7 are not supported by Apple on macOS Sierra, and should not be used for production work.

But what if you have a good reason for running one or the other? Maybe you want to compare a behavior in the latest Xcode 8 with an earlier version of the app. Instead of keeping a virtual machine around, or a second partition with an older OS release, it is liberating to be able to run and test against older versions of Xcode.

So far, it appears that Xcode 7 “mostly works” in spite of being unsupported by Apple. I’ve run into some launch-time crashes, but reopening the app tends to succeed.

Xcode 6 will flat out fail to launch, because one of its internal plugins depends on a private framework (Ubiquity.framework) that is no longer present on macOS Sierra. If you were willing to hack a copy of Xcode 6, however, you could get it running. You definitely shouldn’t do this, but if you’re curious how it could be done, here’s how:

  1. Always have a backup copy of any data that is important to you.
  2. Locate a copy of /System/Library/PrivateFrameworks/Ubiquity.framework from the previous OS X release.
  3. Copy the framework to within Xcode 6’s own Contents/Frameworks bundle subfolder: 
    ditto /my/old/System/Library/PrivateFrameworks/Ubiquity.framework ./Xcode.app/Contents/Frameworks/Ubiquity.framework
  4. Navigate to the problematic Xcode plugin and modify its library lookup table so that it points to the app-bundled copy of Ubiquity.framework, instead of the non-existent system-installed copy. 
    cd Xcode.app/Contents/PlugIns/iCloudSupport.ideplugin/Contents/MacOS
install_name_tool -change /System/Library/PrivateFrameworks/Ubiquity.framework/Versions/A/Ubiquity @rpath/Ubiquity.framework/Versions/A/Ubiquity ./iCloudSupport
  5. Now that you've modified Xcode, its code signature is invalid. You can repair it by signing it with your own credentials or with an ad hoc credential: 
    codesign --deep -f -s - ./Xcode.app
  6. Did I mention you really shouldn't do this?

Apple has good reason to warn people off running older versions of Xcode, but if you absolutely need to get something running again, it's often possible.

Swift Maturity

Apple, Swift

Ted Kremenek of Apple announced on the Swift evolution announcements mailing list that the team will no longer accept source-breaking changes for Swift 3. That is, changes that would require developers’ own Swift code to change. He notes that this means many desirable features will not make the cut, and will have to be pushed to Swift 3.1 or beyond:

The challenge of course is reconciling these diametrically opposing goals: maintaining source stability while having the ability to incorporate more core (and important) language changes that are possibly source-breaking.

How will they balance this going forward? He hints that the team wants to support a mechanism whereby developers can specify a version of Swift as a parameter to the compiler. Your code builds against Swift 3.1? The Swift 4 compiler will be able to handle that:

Our goal is to allow app developers to combine a mix of Swift modules (e.g., SwiftPM packages), where each module is known to compile with a specific version of the language (module A works with Swift 3, module B works with Swift 3.1, etc.), then combine those modules into a single binary.

This is great news for developers, but only strengthens my argument that Swift needs a mechanism for SDK-conditional compilation. At this point, a developer who wishes to maintain source code that compiles against, say, iOS 9 and iOS 10, must conditionalize on the version of Swift:

#if swift(>=2.3)
	// iOS 10 only code
#else
	// iOS 9 friendly code
#endif

When and if Ted Kremenek’s promise of a multiversioned Swift compiler comes to pass, it will presumably mean multiple versions of Swift can compile against the same SDK, so this fragile workaround will no longer … work.

Update: It occurs to me, multiple versions of Swift already do build against the same SDK. Currently we have Swift 2.3 and Swift 3 building against Apple’s latest beta SDKs. It’s the “>=” in the workaround that guarantees a suitable SDK match for now.

Brent’s Swift Tension

Apple, Links, Swift

Brent Simmons has been writing new code exclusively in Swift for a while now, and he recognizes omissions from the language that he still anticipates will be hard to overcome as Apple presumably moves toward incorporating Swift into its developer-facing frameworks. The Tension of Swift:

The Objective-C runtime allows AppKit and UIKit to provide some powerful features that I’d hate to do without: the Responder Chain, xib and storyboard loading, KVC, and the undo manager.

A key point he gets at is that however great Swift is, it’s only possible to use it to develop functional iOS and Mac apps because of the Objective C runtime still operating behind the scenes. Both AppKit and UIKit not only lean on the functionality of the runtime, but are designed with the runtime in mind.

Some major design priorities of the Swift language, namely type safety and compile time dependency binding, are at odds with the design priorities of 20 years of evolution in Apple’s frameworks. How and if that disparity will be reckoned by Apple remains to be seen.

I’m optimistic, because the Swift team has already made many concessions to make the language more compatible with the Objective C runtime. It strikes me as possibly non-optimal that a language that strikes the right compromise between Swift’s priorities and Objective C’s would start at the opposite extreme and work its way backwards, but that is what Apple seems to be doing.

Let’s hope they continue in that direction, and surprise us all with how well it all works out in the end.

Mac App Store Sandbox Testing

Apple, Mac, OS X

For months, many of us Mac developers have noticed that apps built for Mac App Store submission could no longer be tested using iTunes Connect “sandbox” users. Previously, a sandbox user account could be used to authenticate and download a _MASReceipt for a Mac app, to ensure that in-app receipt validation is working as expected. After updating to 10.11.2, many of us noticed that this functionality suddenly stopped working:

At first, we assumed it was a bug. But as time went on, it started to seem as though it could be related to Apple’s announcement that one of its key certificates was expiring.

Still, the communication from Apple about this issue was poor enough that it wasn’t obvious what exactly we needed to do. Even though the page linked above has a section explicitly listing what Mac developers are expected to do:

You can verify your receipt validation code is compatible with the renewed certificate in the test and production environments. A pre-release version of the Mac App Store Update for OS X Snow Leopard beta is now available.

The linked “pre-release version” was no-doubt once a valid link, but at least through my account, it now leads to a permission-denied type failure page.

So what do we do? Fortunately, after chatting through the problem with some friends, Chris Liscio deduced the key, somewhat-obvious in retrospect steps to test your Mac app for compliance with the new certificate, while getting sandbox testing working again at the same time:

  1. Install the new certificate from Apple. In my case, I opened it in Keychain access and added it to the System keychain, where the older, expiring certificate currently resides.
  2. Reboot.

The second step is the important one. If you just install the certificate and expect everything to work, you’ll be sadly rebuffed with continued failures. Reboot. Let the system soak in the new certificate, then try re-launching your Mac app built for submission to the Mac App Store. It will prompt you, as you had previously expected it to, for your sandbox credentials. When you enter them, instead of insisting you set up a new iTunes customer profile, it will just launch. Or, if it doesn’t, maybe you’ve got some problems to work though in your receipt validation code.

Presenting HTML On Apple TV

Apple, tvOS

In a recent post on my Bitsplitting blog, I complained that Apple’s forbiddance of web views on Apple TV would limit many developers who use HTML tastefully in the construction of their interfaces. I suggested that Apple might allow developers to use web views, but limit their usefulness as full-fledged web browsers.

Since writing that article, I discovered a potentially useful “backdoor” of sorts that could allow developers to continue using HTML to some extent for visual formatting of content in their user interfaces.

UITextView supports attributed strings, and NSAttributedString supports being initialized with HTML. Historically on the Mac at least, this capability was famously poor, but I seem to recall reading that it had been boosted at some point by using WebKit behind the scenes to do a more proper conversion.

Here’s an example of how an Apple TV app can convert literal HTML content into a visual form. This is, so far as I can tell, compliant with both the letter and the spirit of Apple’s guidelines for using the SDK:

NSString* staticHTMLContent = @"<div style='font-size:6em;'><strong>Hello</strong> <span style='font-family:courier;'>there</span>, I'm <span style='color:red;'>HTML</span>!</div>";

NSAttributedString* myHTMLString = [[NSAttributedString alloc] initWithData:[staticHTMLContent dataUsingEncoding:NSUTF8StringEncoding] options:@{NSDocumentTypeDocumentAttribute : NSHTMLTextDocumentType} documentAttributes:nil error:nil];

[self.textView setAttributedText:myHTMLString];

This example code is run from a UIViewController whose self.textView is an IBOutlet to a UITextView in the user interface. And here’s how it looks in the Apple TV simulator:

HTML rendered on the Apple TV simulator

Granted, this is a far cry from a fully-functional web view. I’m sure it won’t serve the needs of all developers who currently rely upon UIWebView or WKWebView, but I expect that in some cases it will be a valuable workaround to the otherwise total omission of support for rendering HTML on Apple TV.

NSURLSession Authentication Challenge Disparity

Apple, Bug Reports

Thanks to a bunch of my networking-related unit tests failing on 10.11, I came to the conclusion that NSURLSession’s authentication challenge mechanism changed from 10.11 with respect to the way HTTP Basic Auth challenges are handled.

In 10.10 a data task that is created for a resource protected by HTTP Basic Auth will result in a callback whose protection space method is identified as “NSURLAuthenticationMethodDefault”, while in 10.11 the same code accessing the same resource yields a protection space method “NSURLAuthenticationMethodHTTPBasic”.

The problem here is that existing challenge-handling code may have been written to handle the 10.10 behavior, looking for HTTP Basic Auth challenges like this:

if ([[challenge protectionSpace] authenticationMethod] == NSURLAuthenticationMethodDefault)
{
	// Handle the challenge
	...
}

While on 10.11 the “Handle the challenge” code will never be reached, so this shipping code will fail to function.

I think a robust workaround (that unfortunately requires re-compiling and re-shipping) is to test HTTP authentication challenges for either NSURLAuthenticationMethodDefault or NSURLAuthenticationMethodHTTPBasic, and treat them both as equivalent.

I filed this as Radar #21918904, and wrote a message in the developer forums in case people want to discuss the issue or the merits of various workarounds.