Category Archives: Xcode

Xcode 9 Signing Workarounds

I wrote on Monday about issues with Xcode 9 relating to code signing. Although the gist of that post involved sandboxed Mac applications that launch sandboxed child processes, the fundamental issue is a bit broader: Xcode 9 adds a “com.apple.security.get-task-allow” entitlement to any binary it signs. For the majority of developers, this is probably not an issue, because the entitlement is removed when an Xcode archive is exported for distribution. Most developers, and particularly iOS developers, use Xcode archives.

For folks who don’t, side effects of this additional entitlement include, but may not be limited to:

  1. Inability to launch sandboxed child processes.
  2. Rejection from the Mac App Store.
  3. Unknown consequences of shipping with an unintended entitlement.

So, if you’re a developer who doesn’t use archives, what are your options? I’ve come up with four workarounds, and I present them here, roughly sorted by advisability and level of tedium:

  1. Use Xcode 8. The simplest solution is to not upgrade to Xcode 9 unless and until you need to. Xcode 8’s signing process does not impose the unintended entitlement, so there is no risk of shipping a product that has it, unless you add it yourself. The downside to sticking with Xcode 8 is you won’t enjoy any of the new features of Xcode 9, you’ll have to work to support either Swift 4, macOS 10.13, or iOS 11 SDK features in your app.

  2. Manually re-sign the built-product. Code signing is code signing, and you’re free to sign anything you like to suit your needs, using the “codesign” command line tool. It frankly sounds like a pain in the neck to recursively re-sign every binary in the app bundle, ensuring that the suitable entitlements (minus the unwanted one) are preserved, but I’m sure it can be done.

  3. Use Xcode archives. It strikes me as a little obnoxious to have to use Xcode archives when they don’t offer any added benefits for my dibstrution workflow. But as a long term solution, this is probably the safest bet. The new behavior in Xcode 9 strongly suggests that Apple expects most developers to use archives, and joining the crowd is usually a good idea when it comes to avoiding trouble with Apple’s developer tools.

    If you are using Xcode archives for the first time, particularly with a complex project, you might discover that the resulting archives are not suitable for exporting a signed application. If you get a “Generic Xcode Archive” after running Build -> Archive, you know you’ve got a problem. By default the archive process builds all targets with an “install” option, rendering their built products into a file hierarchy that will be used to build the archive. If your project includes helper apps, for example, they will be “installed” alongside your main app, resulting in a generic archive of two apps, instead of the expected archive of a single app.

    The solution for this problem is to ensure that the “SKIP_INSTALL” build setting is set to YES for any such helper app. Just archive your main app, export the “Built Products” from the resulting archive, and look at the file hierarchy to determine whether you have subtargets that need to have installation disabled.

  4. Hack Xcode 9. In a hurry to ship an update to your app, and you’ve only got Xcode 9 handy? It turns out the imposition of this “com.apple.security.get-task-allow” entitlement is controlled by a single property list file inside Xcode’s application bundle. As a test, I edited the file:

    Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/Library/Xcode/PrivatePlugIns/IDEOSXSupportCore.ideplugin/Contents/Resources/BaseEntitlements.plist
    

    It contains a single entitlement, the one that’s causing our grief. I deleted the entitlement from the list, saved the file, and relaunched Xcode. After doing so, everything is “back to normal.”

    I can’t strongly encourage you to hack your copy of Xcode because I don’t know what the consequences might be. “It seems fine,” but you’re on your own if you decide to do this.

This small change in Xcode 9 causes a lot of unexpected grief for folks who don’t use Xcode archives. I am curious to know how widespread the problem is, and enthusiastic to get the word out about it so that affected folks can work around the problem, or at least be aware of it. Myself, I’ll probably end up adopting the workaround of using Xcode archives, but I’m hopeful that Apple will see the merit of providing an option in an update to Xcode 9 that supports disabling the addition of this entitlement without archiving and exporting a built product.

Sandbox Inheritance Tax

I ran into a subtle bug with Xcode 9 that I think is worth sharing. Specifically, this bug affects Mac applications that:

  1. Are sandboxed.
  2. Launch a sandboxed subprocess with NSTask (or posix_spawn).
  3. Configure the subprocess to inherit the parent’s sandbox.

When such an app is compiled with Xcode 9, the subprocess will crash whenever the parent process launches it. A canonical example of something that might suffer from this problem is a bundled crash-monitor. I embed one with my apps to keep an eye on the running status of the parent process, and to present a crash-reporting interface to users if the host app terminates prematurely. When I build and run my app with Xcode 9, the bundled crash monitor dies instantly upon being launched.

It took me a while to realize that the subprocess is dying because it fails to satisfy the contract for inheriting a sandbox. From Apple’s “Enabling App Sandbox Inheritance“:

To enable sandbox inheritance, a child target must use exactly two App Sandbox entitlement keys: com.apple.security.app-sandbox and com.apple.security.inherit. If you specify any other App Sandbox entitlement, the system aborts the child process.

Well, that’s funny because my child process does specify only those two keys, but the system is aborting it anyway. It turns out that Xcode 9 is inserting a third entitlement without my permission. Clicking on the detail of the “Process Product Packaging” build phase in Xcode’s log navigator, I can see that there are three entitlements for my target:

Xcode build log detail showing the wrong entitlements.

When my subprocess is launched, the system sees that extra “com.apple.security.get-task-allow” entitlement in the context of “com.apple.security.inherit”, and unceremoniously crashes my the child process.

I’m not sure what Apple’s reasoning is for imposing this entitlement on sandboxed targets, but it appears to be doing so across the board, for literally every sandboxed target in my app. I confirmed that all of my apps, XPC processes, helper tools, etc., are all getting this bonus entitlement.

I searched Xcode’s files, and discovered the entitlement listed in this file inside the Xcode app bundle:

Contents/Developer/Platforms/MacOSX.platform/Developer/Library/Xcode/PrivatePlugIns/IDEOSXSupportCore.ideplugin/Contents/Resources/BaseEntitlements.plist

Putting aside the question of whether it’s appropriate for Xcode to surreptitiously add entitlements that are not specified by the developer’s own list of permissions, the addition of the entitlement for these particular targets, ones that inherit their parent’s sandbox, turns out to be a fatal move.

Ideally I would be able to work around this by adding a custom build phase to manually tweak the generated entitlements file, removing the unwanted key. But the “Process Product Packaging” build phase happens so late in the build process that it’s after the last user-specified custom build phase. There’s no room in Xcode’s current design for fixing up the problematic entitlements before they are incorporated into the signed product. As far as I can tell the only clean workaround would be to redundantly re-sign the child app with a custom script, and corrected entitlements, after Xcode’s build process is completed.

I filed Radar #34628449, “Sandboxed project build with Xcode 9 cannot launch child process.”

Update: Colin Barrett pointed out on Twitter that the entitlement in question here, “com.apple.security.get-task-allow”, may be required in order to attach to and debug a process. If true, then I think this is something that was handled in a different way in Xcode 8. I can confirm that my apps do not have the entitlement imposed on them by Xcode 8, yet I am able to attach to and debug them.

If Apple changed the debugger infrastructure in Xcode 9 so that the relationship between the debugger and target processes is more locked down, requiring a specific entitlement, then that’s probably a good thing. But if this change was made without thinking about the implications for the above-cited “strict two entitlement” rule for sandbox inheritance, then probably some flexibility needs to be applied to that rule.

Finally, as I noted above the entitlement is being applied to all my targets. What I didn’t clarify is that the entitlement is added even when Building and Archiving. A release build’s binaries are endowed with this additional entitlement, which may also bring additional security vulnerabilities to the app.

I would not ship a sandboxed Mac app that is built with Xcode 9, until we understand more about when Xcode applies this entitlement, and whether it can be prevented for Release builds at the very least.

Update 2: I’ve learned that Xcode’s “Export Archive” functionality causes the unwanted entitlement to be removed. Apparently the assumption is that everybody creates Xcode archives as part of their build and release process. I am sure this is true for most (all?) iOS deployments, but for Developer-ID signed apps on the Mac, there has traditionally been less of an incentive to do this. Got a properly signed Mac application? Zip it up, put it on a web server, and you’re done.

I’m not sure yet whether I’ll switch my build process to use archiving, or whether I’ll pull some other stunt to redo the code signing with corrected entitlements. In any case this has been quite an adventure today getting to the bottom of this. I updated my bug report with Apple to request that they provide some standard build flag that would prevent the problematic entitlement from being added from the start. In the mean time, I’ll explore one of the workarounds and get my builds back to fully functional!

Better Swift Completion

Apple released Xcode 9 earlier this week, and in spite of a few glitches here and there, I have found the update to be an overall improvement over Xcode 8. It’s nice that Apple continues to invest in the core tools for Mac and iOS developers.

I’ve been dabbling in more and more Swift development lately, and it’s brought to light a shortcoming in Xcode’s code completion which has unfortunately not improved in Xcode 9: completion of Swift function calls when there is a large quantity of candidates.

Take for example NSAttributedString. If I want to initialize a new instance in Swift, I type “NSAttributedString(” to bring up the list of compatible init methods I can choose from:

SwiftCompletion

The problem at this point is that I have to navigate the menu by hand. I can’t narrow down the list of completions any further by typing, because the very next character I type will be interpreted as the manual filling out of parameters of the NSAttributedString initializer.

BadCompletion

This is a situation where Objective-C gets much nicer treatment in the editor. Because completion in Objective-C begins when I start typing “init”, and because the named first parameter is part of the init message name, I can winnow down the results quite a bit:

Pasted Image 9 22 17 11 24 AM

Better still, because Xcode performs a fuzzy match on the typing, I can proceed to type the names of additional parameters to zero in completely on the variation I want:

MEAppController AppDelegate m Edited

When I accept the completion, all of my typing is replaced with the expected, templated parameter placeholders for the chose initializer.

I filed Radar #34594940 requesting better completion for Swift.

Xcode GitHub Integration

Apple’s beta release of Xcode 9 features impressive improvements to its source control features, including streamlined integration with GitHub. There’s even a fancy “Open in Xcode” button when you go to clone a project:

Screen capture of the GitHub interface for cloning a project

This integration is amazing. You just click the button, specify a save folder in Xcode, and boom! You’re off and …

Screen capture of build failure indicating a missing signing certificate

Oh, right. Code signing. The otherwise stellar GitHub integration in Xcode underscores a longstanding deficiency in how it manages code signing identities for multi-team, collaborative projects. Precisely the kinds of projects you’re liable to find on GitHub.

The problem could be solved, or at least diminished greatly, by providing some mechanism for declaring that a project should be code signed “with the user’s own default developer team.” The default branch of any open source project targeting Apple platforms, would specify the DEVELOPMENT_TEAM as something like:

DEVELOPMENT_TEAM = Automatic

Xcode would provide a user-level setting for “Default Development Team”, and in the absence of any overriding setting, that team would be used whenever a project was configured as above.

I wrote about this problem once before, but with all the work being put into streamlining the experience of cloning from and pushing to GitHub, now is an ideal time for Apple to embrace a fix. Radar #32614751.

Another issue that stops short the cloning, and immediate building and running, of open source projects, is the need to fulfill external dependencies. In some cases this might require manually downloading and installing libraries, or cloning projects, but in the vast majority of cases the dependencies will be specified using built-in Git submodule support, or a popular package manager. In each of these cases, it should be trivial for Xcode to detect that the project it has just cloned also has dependencies:

  • Git submodules: there is a .gitmodules directory.
  • Carthage: there is a Cartfile file.
  • CocoaPods: there is a Podfile file.
  • Swift Package Manager: there is a Swift.package file.

If Xcode sees evidence of any of these techniques at play, it could do the favor of checking them out immediately after cloning the project. Radar #32615265.

The GitHub integration coming in Xcode 9 provides a nearly effortless capability for cloning, building, and running open source projects that target Apple platforms. Ideally it would also go the extra mile and provide for variable, dynamic development teams, as well as conduct a rudimentary check for dependencies that must be checked out before commencing work on the project.

Debugging Swift: Error in Auto-Import

Have you ever tried debugging Swift code in an embedded framework, and met resistance from lldb in the form of a cryptic AST context error?

error: in auto-import:
failed to get module 'RSAppKit' from AST context:

<module-includes>:1:9: note: in file included from <module-includes>:1:
#import "Headers/RSAppKit.h"
        ^
error: [...]/RSAppKit.h:1:9: error: 'RSAppKit/SomeHeader.h' file not found
#import <RSAppKit/SomeHeader.h>
        ^

error: could not build Objective-C module 'RSAppKit'

After hours of trying to unravel this mystery, I discovered the root cause: the framework that is embedded in my app does not, in fact, contain any headers. They were stripped by Xcode when it copied the framework into the app.

In my opinion, Xcode and/or lldb should be smart enough to handle this situation, by preferring the version of the framework in the “Built Products” directory, which still has its header files in-tact. Radar #31502879 requests this, hopefully Apple will fix it.

In the mean time, you can work around the problem by setting the REMOVE_HEADERS_FROM_EMBEDDED_BUNDLES build setting to NO in the app that embeds the framework:

Xcode build settings showing REMOVE_HEADERS_FROM_EMBEDDED_BUNDLES set to NO for DEBUG builds.

You probably want to make sure it remains set to YES for Release builds, so that you don’t ship your framework’s header files to your customers.

System Level Breakpoints in Swift

Any great software developer must inevitably become a great software debugger. Debugging consists largely of setting breakpoints, then landing on them to examine the state of an app at arbitrary points during its execution. There are roughly two kinds of breakpoints: those you set on your own code, and those you set on other people’s code.

Setting a breakpoint on your own code is simple. Just find the line of source code in your Xcode project, and tap the area in the gutter next to the pertinent line:

MotoAppSwift

But what if you need to set a breakpoint on a system API, or a method implemented in a drop-in library for which you don’t have source code? For example, imagine you are hunting down a layout bug and decide it might be helpful to observe any calls to Apple’s own internal layoutSubviews method on UIView. Historically, to an Objective-C programmer, this is not a huge challenge. We know the form for expressing such a method symbolically and to break on it, we just drop into Xcode’s lldb console (View -> Debug Area -> Activate Console), and set a breakpoint manually by specifying its name. The “b” shorthand command in lldb does a bit of magic regex matching to expand what we type to its full, matching name:

(lldb) b -[UIView layoutSubviews]
Breakpoint 3: where = UIKit`-[UIView(Hierarchy) layoutSubviews], address = 0x000000010c02f642
(lldb) 

If you’re intimidated by the lldb console, or you want the breakpoint to stick around longer than the current debug session, you can use Xcode’s own built-in symbolic breakpoint interface (Debug -> Breakpoints -> Create Symbolic Breakpoint) to achieve the same thing:

Image of Xcode's symbolic breakpoint editor

In fact, if you add this breakpoint to your iOS project and run your app, I am pretty sure you will run into a breakpoint on Apple’s layoutSubviews method. Pop back into the lldb console and examine the object that is being sent the message:

(lldb) po $arg1
<UIClassicWindow: 0x7f8e7dd06660; frame = (0 0; 414 736); userInteractionEnabled = NO; gestureRecognizers = <NSArray: 0x60000004b7c0>; layer = <UIWindowLayer: 0x600000024260>>

Now, continue and break on the symbol again. And again. Examine the target each time by typing “po $arg1” into the lldb console. You can imagine how handy it might be to perform this kind of analysis while tracking down a tricky bug.

But what about the poor Swift programmers who have come to our platforms, bright-eyed and full of enthusiasm for Swift syntax? They who have read Apple’s documentation, and for whom “-[UIView layoutSubviews]” is impossible to parse, whereas “UIView.layoutSubviews” not only looks downright obvious, but is correct for Swift?

Unfortunately, setting a breakpoint on “UIView.layoutSubviews” simply doesn’t work:

(lldb) b UIView.layoutSubviews
Breakpoint 3: no locations (pending).
WARNING:  Unable to resolve breakpoint to any actual locations.
(lldb) 

This fails because there is no Swift type named UIView implementing a method called layoutSubviews. It’s implemented entirely in Objective-C. In fact, a huge number of Objective-C methods that are exposed to Swift get compiled down to direct Objective-C message sends. If you type something like “UIView().layoutIfNeeded()” into a Swift file, and compile it, no Swift method call to layoutIfNeeded ever occurs.

This isn’t the case for all Cocoa types that are mapped into Swift. For example, imagine you wanted to break on all calls to “Data.write(to:options:)”. You might try to set a breakpoint on “Data.write” in the hopes that it works:

(lldb) b Data.write
Breakpoint 11: where = libswiftFoundation.dylib`Foundation.Data.write (to : Foundation.URL, options : __ObjC.NSData.WritingOptions) throws -> (), address = 0x00000001044edf10

And it does! How about that? Only it doesn’t, really. This will break on all calls that pass through libswiftFoundation on their way to -[NSData writeToURL:options:error:], but it won’t catch anything that calls the Objective-C implementation directly. To catch all calls to the underlying method, you need to set the breakpoint on the lower level, Objective-C method.

So, as a rule, Swift programmers who want to be advanced debuggers on iOS or Mac platforms, also need to develop an ability for mapping Swift method names back to their Objective-C equivalents. For a method like UIView.layoutSubviews, it’s a pretty direct mapping back to “-[UIView layoutSubviews]”, but for many methods it’s nowhere near as simple.

To map a Swift-mapped method name back to Objective-C, you have to appreciate that many Foundation classes are stripped of their “NS” prefix, and the effects of rewriting method signatures to accommodate Swift’s API guidelines. For example, a naive Swift programmer may not easily guess that in order to set a breakpoint on the low-level implementation for “Data.write(to:options)”, you need to add back the “NS” prefix, explicitly describe the URL parameter, and add a mysterious error parameter, which is apparently how cranky greybeards used to propagate failures in the bad old days:

(lldb) b -[NSData writeToURL:options:error:]
Breakpoint 13: where = Foundation`-[NSData(NSData) writeToURL:options:error:], address = 0x00000001018328c3

Success!

For those of you mourn the thought of having to develop this extensive knowledge of Objective-C message signatures and API conventions, I offer a little hack that will likely get you through your next challenge. If the API has been rewritten using one of these rules, it’s almost certain that the Swift name of the function is a subset of the ObjC method name. You can probably leverage the regex matching ability of lldb to zero in on the method you want to set a breakpoint on:

(lldb) break set -r Data.*write
Breakpoint 14: 107 locations.

Now type “break list” and see the massive number of likely matches lldb has presented at your feet. Among them are a number of Swift cover methods that are part of libswiftFoundation, but you’ll also find the target method in question. In fact, you’ll also see a few other low-level Objective-C methods that you may want to break on as well.

To make the list more manageable, given your knowledge that the target methods are in a given Objective-C framework, add the “-s” flag to limit matches to a specific shared library by name:

(lldb) break set -s Foundation -r Data.*write
Breakpoint 17: 8 locations.

Among these breakpoints there are a few false hits on the NSPageData class, but the list is altogether more manageable. The single breakpoint “17” has all of its matches identified by sub-numbers. Prune the list of any breakpoints that get in your way, and you’re good to go:

(lldb) break disable 17.6 17.7 17.8
3 breakpoints disabled.
(lldb) c

Apple’s mapping of Objective-C API to Swift creates an altogether more enjoyable programming experience for Swift developers, but it can lead to great confusion if you don’t understand some of the implementation details, or how to work around lack of understanding. I hope this article gives you the tools you need to debug your Swift apps, and the Objective-C code that you are unavoidably leveraging, more effectively.

Update: I filed two related bugs: Radar #31115822 requesting automatic mapping from Swift method format back to underlying Objective-C methods, and Radar #31115942 requesting that lldb be more intuitive about evaluating terse Swift method signatures.

Implicit Swift Dependencies

If you’re developing in Swift for Mac or iOS, you need to ensure that any standard Swift libraries are also copied into your app bundle. Typically this is handled automatically by Xcode when it detects the presence of any Swift files in your app. If your app is entirely Objective-C, but you link against your own frameworks that themselves depend on Swift, you have to ensure the required libraries are embedded. This can be done by setting the “Always Embed Swift Standard Libraries” checkbox in your target’s build settings to “Yes”:

AlwaysEmbed

When this option is set, Xcode scans the bundled frameworks in your app and, if any of them contains Swift code, it copies the pertinent libraries into your app’s bundle.

This approach falls down a bit when it comes to Unit Testing targets. Because a Unit Test bundle doesn’t typically copy the framework it is testing into its own bundle, Xcode seems to deduce dependencies by examining the list of frameworks the test bundle links to. If you enable the “Always Embed” option above for a Unit Testing bundle, then any framework the bundle links against will be examined for Swift code, and the required libraries copied in.

But what if none of the linked frameworks themselves require Swift, but they depend upon other frameworks that do? In projects of a substantial size, you may have higher-level frameworks implemented entirely in Objective-C, that depend upon lower level frameworks that use Swift. In this scenario, Xcode’s CopySwiftLibs build phase does not identify the deeper dependency, and neglects to embed the required Swift libraries. When your unit tests run, you’ll see a runtime error like this:

The bundle “RSTextFilteringTests” couldn’t be loaded because it is damaged or missing necessary resources. Try reinstalling the bundle.
(dlopen_preflight([...]/RSTextFilteringTests): Library not loaded: @rpath/libswiftAppKit.dylib
  Referenced from: [...]/RSAppKit.framework/Versions/A/RSAppKit
  Reason: image not found)
Program ended with exit code: 82

So “Always Embed Swift Standard Libraries” doesn’t always embed them. I’ve filed Radar #30832923, requesting that the CopySwiftLibs build phase look harder for Swift dependencies. In my opinion it should look not only at the directly linked frameworks, but at any developer-supplied frameworks that those in turn depend upon. This would cause “Always Embed” to behave as expected when a developer adds Swift dependencies to frameworks at any level of their code base.

There are many possible workarounds to the problem, and each has its own drawbacks. Here are a few:

  1. Add a Swift-based unit test file to your test bundle. This simple workaround will work in most situations, because the test itself will impose many Swift library dependencies on your bundle. There is a shortcoming, though: if the test code doesn’t generate all the Swift dependencies that your frameworks do, you’ll still be left with missing Swift libraries. For example if a dependent library requires the Swift library for Apple’s Contacts.framework, your Swift test file is not sufficient to cue Xcode to copy the required libSwiftContacts library.
  2. Manually link to Swift-dependent frameworks. Suppose your unit tests target HighLevel.framework, and LowLevel.framework depends upon Swift. By adding LowLevel.framework to the list of linked libraries for your testing bundle, you will cue Xcode to also embed the required Swift libraries for that framework. This has the advantage of ensuring that whatever Swift libraries are required get copied, but it has the drawback of requiring explicit dependency on a lower level framework from your unit test bundle.
  3. Stop! Abort mission! Revert to Objective-C only. Many Swift dabblers probably feel this emotion from time to time. The unexpected costs of adding a modest amount of Swift to a project are frustrating, and one can’t help but wonder whether they would be better off waiting until Swift support is more baked in to the system. The obvious downside of this approach is that Apple’s progress with Swift is moving quickly, and the longer you put off testing the waters, the harder it may be for you to catch up when ignoring it is no longer a viable option.

The workaround I’ve settled on for now is #2. Adding unnecessary direct dependencies to my test bundle feels a little inappropriate, but it is both easily undone in the future, and unlikely to cause any real problems in practice.

In the course of investigating this problem, it also occurred to me that unit test bundles in particular are such a special case, that Apple should probably give them special treatment to eliminate a number of pitfalls. I filed Radar #30832983 suggesting that unit test bundles should default to “Always Embed Swift Standard Libraries,” on the assumption that Swift code may be present in some library.

Thinking further on that point, it seems that Unit Tests could be safely run in an environment in which all Swift support libraries are available at runtime to be resolved by DYLD. Because test bundles are not built to be shipped to customers, it’s of primary that they link and run successfully. Such an approach would do away with the need for Unit Test bundles to either specify that Swift libraries be bundled, or to fret about which of those libraries in particular need to be copied.

Interface Builder: View Is Clipping Its Content

If you have Xcode’s “Show live issues” enabled, you’re used to seeing (usually) helpful warnings and notices about your source code and interface files. These notices appear even if you haven’t yet built your project:

Image of Xcode Interface Builder warning about a view clipping its content.

If you click the “View is clipping its content” notice, it takes you right to the view in question:

Image of a popup button on the Interface Builder canvas.

At this point you can usually just “size to fit” and Interface Builder will use its knowledge of the control’s class, and that class’s ability to size itself to suit its content. Or, if you’re using Auto Layout, it might mean that you need to ask Interface Builder to update the items’s frame, allowing Auto Layout to essentially size to fit for you.

In this case however I have a conundrum: both “size to fit” and AutoLayout insist this is the proper size and placement for the control, yet Interface Builder is still convinced the control will clip its content (the text of the menu item title).

What happens if I naively attempt to increase the width of the popup button?

Image of a popup button with error messages indicating it is the wrong width.

The clipping area is gone, as Interface Builder is convinced the button is now wide enough, but that width is in conflict with what Auto Layout is convinced is the right width.

I can’t win: if I let Auto Layout have it’s way, I get an annoying clipping notice. If I let the clipping notice have its way, Auto Layout throws a fit.

One workaround, when working with Auto Layout, is to provide a bogus constraint that forces the width of the popup button to the width that I’ve manually imposed. By setting it to “Remove at Build Time” it should not have any effect on the behavior of your interface, except in Xcode itself.

WidthConstraint

See that empty “Identifier” field? I have actually taken advantage of that field as an opportunity to add a memo to myself for future reference: “Work around bug 25938”. This references my internal bug tracking the issue, so I can re-acquaint myself with the problem if I find myself wondering about this bogus constraint in the future.

It seems to me the bug here is either that NSPopUpButton sizes to fit at too narrow a size, technically causing “clipping” of its subviews. Alternatively, Interface Builder’s deduction of a view’s size causing clipping has some bug in it. Either way, I’ve filed the issue as Radar #30222922.

Update, January 28, 2017: Thanks to a tweet from Mark Lilback, I discovered the notice about clipping is a bit less pervasive than I thought. The notice only seems to appear when Xcode has the xib file actively open for editing with Interface Builder. What this means practically is that you have to click on the xib file at some point and have the editor appear for it, before the notice appears. It also means that if you cause Xcode to close the file for editing, the notice disappears. You can close any file in Xcode by focusing on its editor and then selecting File -> “Close <filename>”, or by pressing Cmd-Ctrl-W.

I have always found these kinds of notices in Xcode to be somewhat unpredictable. The fact that the file has to be actively opened for editing, combined with the fact that files remain open in Xcode after their editor is visually swapped out, explains most of the apparent inconsistencies I’ve seen.

Unbuffering Jenkins Output

For years I have used Jenkins to manage all my automated Mac and iOS builds. I have jobs set up to build and release apps “on demand,” as well as integration builds that kick off whenever a change is made to a pertinent source code repository.

For years I have also put up with a small but vexing nuisance that when something goes wrong with a build, I have to search in the most unlikely place for pertinent informational messages printed by my build scripts: at the very bottom of the log.

I had chalked it up for so long to being a nuanced Jenkins bug, that I never took the time to get down to basics and really unwrap what is going on. Today, I did that, and also came up with an easy solution.

I use a custom script to drive the whole process for each of my automated builds. This script is responsible for printing diagnostic information and for invoking other tools (such as xcodebuild) to get the actual work of building, testing, and uploading done.

The problem with the misplaced log lines in my console output has to do with a feature of the scripting language I use to drive the process: standard output buffering. I use Python, but the problem could also affect you if you use a language such as Ruby to drive your automation.

Here’s a contrived example of a build script that will exhibit the problem. If you create a new, empty Jenkins build job, and configure it with a single “Execute Shell” build task:

#!/usr/bin/python 

import os

print "start"
os.system("ls -ld /Applications/TextEdit.app")
print "end"

You will find the generated console output upon “building” this job is demonstrably out of order:

[workspace] $ /usr/bin/python /var/folders/wh/f_vmqvxx34d69c1pm191t0tr0000gq/T/hudson321577740740596707.sh
drwxr-xr-x@ 3 root  wheel  102 Dec 13 14:00 /Applications/TextEdit.app
start
end

The message from the “ls” subprocess prints right away, but Python buffers the output until it’s done running, leading a nonsensical printing of the “start” and “end” messages after the job is completely done.

Now I will make a tiny change, passing the “-u” parameter to the Python invocation in the script. This option is documented to “Force stdin, stdout and stderr to be totally unbuffered”:

#!/usr/bin/python -u

Running the job again in Jenkins produces the expected output in Jenkins:

[workspace] $ /usr/bin/python -u /var/folders/wh/f_vmqvxx34d69c1pm191t0tr0000gq/T/hudson3255283214356026397.sh
start
drwxr-xr-x@ 3 root  wheel  102 Dec 13 14:00 /Applications/TextEdit.app
end

This is a small annoyance in this example, but in a real-world scenario it means that the pertinent xcodebuild invocation, which I print to the console before calling, will now show up where it belongs in the console log, making it much easier for me to reason about the behavior of my automated builds when they aren’t working as they should.

Installing Symbols For WatchOS

For months I have been plagued by Xcode’s persistent failure to successfully install WatchOS device support on my Mac. If I open Xcode, and my iPhone (paired with an Apple Watch) is attached to the Mac, I am greeted by a progress indicator such as this:

InterfaceController swift

Because I don’t actually do any watchOS development, this has mostly been a mild annoyance. The bug doesn’t seem to affect the performance of my Mac, and other actions I perform in Xcode continue without delay. However, for those of you who are actually doing Watch development, if you run into this problem, you’ll be perpetually greeted by this error when you try to debug on a real device:

Xcode panel refusing to install on a device because of missing Apple Watch symbols.

The worst side-effect of this bug for me, and the problem that finally encouraged me to hunker down and solve the issue, is the fact that each attempt to “Install Symbols” is associated with a fresh download of a larger-than-200MB file from Apple to my Mac. This is no big deal on my home network, but when I’m roaming and tethered to my bandwidth-metered iPhone, it can lead to costly overage charges.

I’ve been to hell and back tracking down exactly how Xcode installs these symbols, where it fails, and how to work around the problem. I’ll give an overview of how Xcode’s symbol installation process works, and walk you through one concrete method for working around the problem, getting those precious symbols installed, and breaking the cycle of perpetual failed installations.

Symbol Installation Overview

First I want to give you a high level idea of how Xcode downloads and installs symbols. Here are the steps that take place:

  1. A device suitable for development is connected.
  2. Xcode looks for existing symbols in the suitable home directory location
  3. If symbols are not already installed, the symbol-installation process begins.
  4. A disk image file containing the required materials is downloaded to a temporary folder.
  5. The disk image file is copied to a Caches subfolder in your home directory.
  6. The disk image is mounted at a temporary location on your Mac
  7. A macOS Installer package on the disk image is used to install symbol files directly into your home folder.
  8. The temporary disk image is unmounted.
  9. The cached disk image file is deleted.

In my tests, the failure to successfully install watchOS symbols broke down in step 7, where the package is supposed to successfully copy symbol files into my home folder. I was able to determine that the Apple infrastructure responsible for the installation is failing on my Mac with a cryptic authentication error:

Could not download and install Symbols for watchOS 3.1 (14S471). Authorization is required to install the packages

The failure is not happening in Xcode per se, but in a helper tool that is part of the private PackageKit.framework. I filed Radar #29568241, in the hopes that somebody at Apple will have insight as the root cause of the problem, and how it can be fixed for good in either Xcode, or in the configurations of affected users’ Macs.

Fix It Yourself

To manually accomplish what Xcode is failing to do, we need to:

  1. Get a copy of the downloaded disk image.
  2. Copy the pertinent files out of the disk image’s installer package.
  3. Copy the pertinent files into the correct installation path in ~/Library/Developer/Xcode/

Because simply creating the directory at the installation path is enough to stop Xcode attempting to install symbols, I’m going to cover that first. Folks who don’t need or want the symbols for Watch development will solve the problem sufficiently after this step.

Determine the Installation Path

Unfortunately, the path is quite specific to the Watch device at hand, and may be difficult to guess. Here’s one way of figuring it out. Making sure you have an iPhone connected that triggers the symbol installation in Xcode, do the following:

  1. Quit Xcode.
  2. From the Terminal, navigate to Xcode’s binary executable folder:
    cd /Applications/Xcode.app/Contents/MacOS/
  3. Run Xcode with a special flag to cause additional logging to display:
    ./Xcode -DVTDownloadableLogLevel 3
  4. Wait for Xcode to launch and start installing symbols.
  5. Scan the output in Terminal for a line like this:
    Starting a download for Watch1,2 3.1 (14S471), to file path /Volumes/Data/daniel/Library/Developer/Xcode/watchOS DeviceSupport/Watch1,2 3.1 (14S471)/Symbols

    Of course, your output will be slightly different, but depending on the version of Watch and watchOS you have, the important part is relative to your home folder. In my case, I need to make sure this folder exists:

    ~/Library/Developer/Xcode/watchOS DeviceSupport/Watch1,2 3.1 (14S471)/Symbols

If you don’t care about Watch development, this is your big chance: just create that folder now. Quit and relaunch Xcode again, and you’ll see that your problems are over. No more symbol installations required. If you skim the logging output from above you’ll see that the “InstalledIfAllPathsArePresent” key in one of the dictionaries indicates that the path merely being present is enough to convince Xcode it’s “installed.”

If you do care about Watch development, you’re going to want those symbols. Let’s tackle that problem next.

Getting the Disk Image

Remember back in “Symbol Installation Overview,” I described the installation failing in step 7: where a package on a mounted volume is attempted to be installed. Unfortunately, this failure does not prevent steps 8 and 9 from proceeding, so when the whole procedure is over, there are no obvious artifacts left around to work with. The disk image, that was copied to ~/Library/Caches/com.apple.dt.Xcode/Downloads in step 5, is available for such a short period of time, you’re unlikely to nab it before Xcode deletes it upon failure.

You may have noticed in the Xcode logging output that many URLs to web resources are passed around. It’s likely that one of these contains the required disk image content, but I thought it would be safer to catch Xcode in the act and use precisely the disk image file that it itself was intending to use.

Yet another “secret” Xcode user default key can help us here. Once again, navigate to the Terminal to execute Xcode manually. This time, we’ll pass a flag that instructs it explicitly not to delete the coveted disk image file after it’s done:

./Xcode -DVTDownloadableLeaveTemporaryFiles 1

This time you’ll have to wait again until the process tries, and fails, but when you’re done you should find a suitable disk image file waiting for you at:

~/Library/Caches/com.apple.dt.Xcode/Downloads/Xcode.Symbols.Watch1,2.14S471-3.1.dmg

(Or with whatever name is suitable for the version of Watch and OS you are accommodating.)

Copying the Symbol Files Out

Double-click the disk image to mount it on your Mac. You’ll find a volume appears with a suitable device-oriented name, and a matching file within called e.g. “Watch1,1_Watch1,2.pkg”.

Warning: Do not open and install this package willy-nilly without reading the following carefully. This is a standard Installer package, and double-clicking it on your Mac will launch the system-standard Installer application. However, agreeing to let it “just install” on your Mac could have unintended consequences. The files in this package are named very similarly to many system files on your Mac, but of course these files are binaries for a ARM based Watch device, and not an x86_64 personal computer.

There are many techniques for getting the files out of an Installer Package without actually, you know, installing. You could use an app like Pacifist to extract them directly to a folder on your disk. There may also be a way to invoke the command-line “installer” tool in such a way that it simply spits the files out where you want them. For that matter, it might be safe to simply “Choose Folder…” from the Installer app, and trust that it will install everything in that folder instead of on the root of the volume.

Me? I decided to strike a semi-paranoid compromise. I used the Installer app, but instead of installing to my Mac’s main volume, I created a writable disk image in Disk Utility, and targeted that volume explicitly:

WatchHack

When the installation was complete, I had a mounted volume, filled with the contents that Xcode has presumably been intending to copy into my home folder.

WatchHack

Finish the Job

At this point you know correct install location in your home folder, and you have a copy of all the files that should be there. It should be as simple as selecting the files from “WatchHack” and dragging them to the “Symbols” folder in the appropriate ~/Library/Developer subfolder.

When the symbol files have been copied in, quit and relaunch Xcode. Not only should it avoid that nasty “Installing Symbols” phase, but building and running on your physical watchOS device should once again work as expected.