Monthly Archives: September 2017

Swatch Your Step

Bug Reports, Cocoa, Mac, OS X

Shortly after macOS 10.13 was released, I received an oddly specific bug report from a customer, who observed that the little square “swatches” in the standard Mac color panel no longer had any effect on MarsEdit’s rich text editor.

Screenshot of the macOS standard color panel.

I was able to reproduce the problem in the shipping 3.7.11 version of MarsEdit, which for various reasons is still built using an older version of Xcode, against the 10.6 SDK. The MarsEdit 4 Beta, which is built against the 10.12 SDK, does not exhibit the problem.

It’s not unusual for the behavior of Apple’s frameworks to vary based on the version of SDK an application was built against. The idea is usually to preserve the old behaviors of frameworks, so that any changes do not defy the expectations of a developer who has not been able to build and test their app against a later SDK. Sometimes, the variations in behavior lead to bugs like this one.

Using a totally straightforward demo app, consisting only of an NSTextView and a button to bring up the color panel, I was able to confirm that the bug affects an app that links against the macOS 10.9 SDK, but does not affect an app that links against the 10.10 SDK.

I filed Radar #34757710: “NSColorPanel swatches don’t work on apps linked against 10.9 or earlier.” I don’t know of a workaround yet, other than compiling against a later SDK.

Xcode 9 Signing Workarounds

Apple, Xcode

I wrote on Monday about issues with Xcode 9 relating to code signing. Although the gist of that post involved sandboxed Mac applications that launch sandboxed child processes, the fundamental issue is a bit broader: Xcode 9 adds a “com.apple.security.get-task-allow” entitlement to any binary it signs. For the majority of developers, this is probably not an issue, because the entitlement is removed when an Xcode archive is exported for distribution. Most developers, and particularly iOS developers, use Xcode archives.

For folks who don’t, side effects of this additional entitlement include, but may not be limited to:

  1. Inability to launch sandboxed child processes.
  2. Rejection from the Mac App Store.
  3. Unknown consequences of shipping with an unintended entitlement.

So, if you’re a developer who doesn’t use archives, what are your options? I’ve come up with four workarounds, and I present them here, roughly sorted by advisability and level of tedium:

  1. Use Xcode 8. The simplest solution is to not upgrade to Xcode 9 unless and until you need to. Xcode 8’s signing process does not impose the unintended entitlement, so there is no risk of shipping a product that has it, unless you add it yourself. The downside to sticking with Xcode 8 is you won’t enjoy any of the new features of Xcode 9, you’ll have to work to support either Swift 4, macOS 10.13, or iOS 11 SDK features in your app.

  2. Manually re-sign the built-product. Code signing is code signing, and you’re free to sign anything you like to suit your needs, using the “codesign” command line tool. It frankly sounds like a pain in the neck to recursively re-sign every binary in the app bundle, ensuring that the suitable entitlements (minus the unwanted one) are preserved, but I’m sure it can be done.

  3. Use Xcode archives. It strikes me as a little obnoxious to have to use Xcode archives when they don’t offer any added benefits for my dibstrution workflow. But as a long term solution, this is probably the safest bet. The new behavior in Xcode 9 strongly suggests that Apple expects most developers to use archives, and joining the crowd is usually a good idea when it comes to avoiding trouble with Apple’s developer tools.

    If you are using Xcode archives for the first time, particularly with a complex project, you might discover that the resulting archives are not suitable for exporting a signed application. If you get a “Generic Xcode Archive” after running Build -> Archive, you know you’ve got a problem. By default the archive process builds all targets with an “install” option, rendering their built products into a file hierarchy that will be used to build the archive. If your project includes helper apps, for example, they will be “installed” alongside your main app, resulting in a generic archive of two apps, instead of the expected archive of a single app.

    The solution for this problem is to ensure that the “SKIP_INSTALL” build setting is set to YES for any such helper app. Just archive your main app, export the “Built Products” from the resulting archive, and look at the file hierarchy to determine whether you have subtargets that need to have installation disabled.

  4. Hack Xcode 9. In a hurry to ship an update to your app, and you’ve only got Xcode 9 handy? It turns out the imposition of this “com.apple.security.get-task-allow” entitlement is controlled by a single property list file inside Xcode’s application bundle. As a test, I edited the file:

    Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/Library/Xcode/PrivatePlugIns/IDEOSXSupportCore.ideplugin/Contents/Resources/BaseEntitlements.plist

    It contains a single entitlement, the one that’s causing our grief. I deleted the entitlement from the list, saved the file, and relaunched Xcode. After doing so, everything is “back to normal.”

    I can’t strongly encourage you to hack your copy of Xcode because I don’t know what the consequences might be. “It seems fine,” but you’re on your own if you decide to do this.

This small change in Xcode 9 causes a lot of unexpected grief for folks who don’t use Xcode archives. I am curious to know how widespread the problem is, and enthusiastic to get the word out about it so that affected folks can work around the problem, or at least be aware of it. Myself, I’ll probably end up adopting the workaround of using Xcode archives, but I’m hopeful that Apple will see the merit of providing an option in an update to Xcode 9 that supports disabling the addition of this entitlement without archiving and exporting a built product.

Unordered Directory Contents

Cocoa, OS X

Since I updated to macOS 10.13 High Sierra, some of my unit tests broke. Examining the failures more carefully, I discovered that they were making assumptions about the order that Foundation’s FileManager.contentsOfDirectory(atPath:) would return items.

I wrote a quick playground to test the behavior on a 10.12 machine:

import Foundation

let array = try! FileManager.default.contentsOfDirectory(atPath: "/Applications/Utilities")
print("\(array.debugDescription)")

The results come back alphabetically ordered by file name:

[".DS_Store", ".localized", "Activity Monitor.app", "Adobe Flash Player Install Manager.app", "AirPort Utility.app", "Audio MIDI Setup.app", "Bluetooth File Exchange.app", "Boot Camp Assistant.app", "ColorSync Utility.app", "Console.app", "Digital Color Meter.app", "Disk Utility.app", "Grab.app", "Grapher.app", "Keychain Access.app", "Migration Assistant.app", "Script Editor.app", "System Information.app", "Terminal.app", "VoiceOver Utility.app"]

The same playground on 10.13 tells a different story:

["AirPort Utility.app", "VoiceOver Utility.app", "Terminal.app", "Activity Monitor.app", ".DS_Store", "Grapher.app", "Audio MIDI Setup.app", ".localized", "System Information.app", "Keychain Access.app", "Grab.app", "Migration Assistant.app", "Script Editor.app", "ColorSync Utility.app", "Console.app", "Disk Utility.app", "Bluetooth File Exchange.app", "Boot Camp Assistant.app", "Digital Color Meter.app"]

I thought at first this might have been related to the APFS conversion that 10.13 applied to my boot volume, but the same ordering discrepancy occurs for items on my HFS+ volumes as well.

After checking the 10.13 release notes for clues, and finding none, I consulted the documentation. Well, what do you know?

The order of the files in the returned array is undefined.

So, mea culpa. The test code in question probably shouldn’t have ever made assumptions about the ordering of items returned from this method. While it has evidently always been undefined, it appears they are only making good on that promise in 10.13. You have been warned!

Update: It turns out I have some real bugs in my apps, not just in my tests, because of assuming the results of this call will be reasonably sorted. Luckily I use a bottleneck method for obtaining the list of files, and I can impose my own sorting right at the source. If you’re looking to make the same kinds of changes to your app, be sure to heed Peter Maurer’s advice and use “localizedStandardCompare” (available since macOS10.6/iOS4) to obtain Finder-like ordering of the results.

Sandbox Inheritance Tax

Apple, Bug Reports, Xcode

I ran into a subtle bug with Xcode 9 that I think is worth sharing. Specifically, this bug affects Mac applications that:

  1. Are sandboxed.
  2. Launch a sandboxed subprocess with NSTask (or posix_spawn).
  3. Configure the subprocess to inherit the parent’s sandbox.

When such an app is compiled with Xcode 9, the subprocess will crash whenever the parent process launches it. A canonical example of something that might suffer from this problem is a bundled crash-monitor. I embed one with my apps to keep an eye on the running status of the parent process, and to present a crash-reporting interface to users if the host app terminates prematurely. When I build and run my app with Xcode 9, the bundled crash monitor dies instantly upon being launched.

It took me a while to realize that the subprocess is dying because it fails to satisfy the contract for inheriting a sandbox. From Apple’s “Enabling App Sandbox Inheritance“:

To enable sandbox inheritance, a child target must use exactly two App Sandbox entitlement keys: com.apple.security.app-sandbox and com.apple.security.inherit. If you specify any other App Sandbox entitlement, the system aborts the child process.

Well, that’s funny because my child process does specify only those two keys, but the system is aborting it anyway. It turns out that Xcode 9 is inserting a third entitlement without my permission. Clicking on the detail of the “Process Product Packaging” build phase in Xcode’s log navigator, I can see that there are three entitlements for my target:

Xcode build log detail showing the wrong entitlements.

When my subprocess is launched, the system sees that extra “com.apple.security.get-task-allow” entitlement in the context of “com.apple.security.inherit”, and unceremoniously crashes my the child process.

I’m not sure what Apple’s reasoning is for imposing this entitlement on sandboxed targets, but it appears to be doing so across the board, for literally every sandboxed target in my app. I confirmed that all of my apps, XPC processes, helper tools, etc., are all getting this bonus entitlement.

I searched Xcode’s files, and discovered the entitlement listed in this file inside the Xcode app bundle:

Contents/Developer/Platforms/MacOSX.platform/Developer/Library/Xcode/PrivatePlugIns/IDEOSXSupportCore.ideplugin/Contents/Resources/BaseEntitlements.plist

Putting aside the question of whether it’s appropriate for Xcode to surreptitiously add entitlements that are not specified by the developer’s own list of permissions, the addition of the entitlement for these particular targets, ones that inherit their parent’s sandbox, turns out to be a fatal move.

Ideally I would be able to work around this by adding a custom build phase to manually tweak the generated entitlements file, removing the unwanted key. But the “Process Product Packaging” build phase happens so late in the build process that it’s after the last user-specified custom build phase. There’s no room in Xcode’s current design for fixing up the problematic entitlements before they are incorporated into the signed product. As far as I can tell the only clean workaround would be to redundantly re-sign the child app with a custom script, and corrected entitlements, after Xcode’s build process is completed.

I filed Radar #34628449, “Sandboxed project build with Xcode 9 cannot launch child process.”

Update: Colin Barrett pointed out on Twitter that the entitlement in question here, “com.apple.security.get-task-allow”, may be required in order to attach to and debug a process. If true, then I think this is something that was handled in a different way in Xcode 8. I can confirm that my apps do not have the entitlement imposed on them by Xcode 8, yet I am able to attach to and debug them.

If Apple changed the debugger infrastructure in Xcode 9 so that the relationship between the debugger and target processes is more locked down, requiring a specific entitlement, then that’s probably a good thing. But if this change was made without thinking about the implications for the above-cited “strict two entitlement” rule for sandbox inheritance, then probably some flexibility needs to be applied to that rule.

Finally, as I noted above the entitlement is being applied to all my targets. What I didn’t clarify is that the entitlement is added even when Building and Archiving. A release build’s binaries are endowed with this additional entitlement, which may also bring additional security vulnerabilities to the app.

I would not ship a sandboxed Mac app that is built with Xcode 9, until we understand more about when Xcode applies this entitlement, and whether it can be prevented for Release builds at the very least.

Update 2: I’ve learned that Xcode’s “Export Archive” functionality causes the unwanted entitlement to be removed. Apparently the assumption is that everybody creates Xcode archives as part of their build and release process. I am sure this is true for most (all?) iOS deployments, but for Developer-ID signed apps on the Mac, there has traditionally been less of an incentive to do this. Got a properly signed Mac application? Zip it up, put it on a web server, and you’re done.

I’m not sure yet whether I’ll switch my build process to use archiving, or whether I’ll pull some other stunt to redo the code signing with corrected entitlements. In any case this has been quite an adventure today getting to the bottom of this. I updated my bug report with Apple to request that they provide some standard build flag that would prevent the problematic entitlement from being added from the start. In the mean time, I’ll explore one of the workarounds and get my builds back to fully functional!

Better Swift Completion

Bug Reports, ObjC, Swift, Xcode

Apple released Xcode 9 earlier this week, and in spite of a few glitches here and there, I have found the update to be an overall improvement over Xcode 8. It’s nice that Apple continues to invest in the core tools for Mac and iOS developers.

I’ve been dabbling in more and more Swift development lately, and it’s brought to light a shortcoming in Xcode’s code completion which has unfortunately not improved in Xcode 9: completion of Swift function calls when there is a large quantity of candidates.

Take for example NSAttributedString. If I want to initialize a new instance in Swift, I type “NSAttributedString(” to bring up the list of compatible init methods I can choose from:

SwiftCompletion

The problem at this point is that I have to navigate the menu by hand. I can’t narrow down the list of completions any further by typing, because the very next character I type will be interpreted as the manual filling out of parameters of the NSAttributedString initializer.

BadCompletion

This is a situation where Objective-C gets much nicer treatment in the editor. Because completion in Objective-C begins when I start typing “init”, and because the named first parameter is part of the init message name, I can winnow down the results quite a bit:

Pasted Image 9 22 17 11 24 AM

Better still, because Xcode performs a fuzzy match on the typing, I can proceed to type the names of additional parameters to zero in completely on the variation I want:

MEAppController AppDelegate m Edited

When I accept the completion, all of my typing is replaced with the expected, templated parameter placeholders for the chose initializer.

I filed Radar #34594940 requesting better completion for Swift.