Unified Swift Playgrounds

The announcement of Swift Playgrounds 2.0 has me thinking again about Xcode Playgrounds: both about what a revelation they are, and about how disappointing they continue to be.

When Xcode Playgrounds were first introduced (as “Swift Playgrounds”) in 2014, they were received as a groundbreaking new way for developers to write Swift code interactively. There were lots of rough edges on the feature, but it seemed reasonable to expect that because they were released in tandem with the Swift programming language itself, those rough edges would be smoothed out on a parallel pace with the language itself.

Two years later, Apple announced Swift Playgrounds again, immediately introducing a nomenclature confusion. This time the name referred to a dedicated, closed-source iOS app designed for interactively teaching programming concepts with Swift. The previous Xcode-coupled technology, now known as “Xcode Playgrounds” or simply “Playgrounds,” had seen modest improvements over the years but continued to be frustratingly slow, unpredictable, and crash-prone.

Today, in early 2018, the release of Swift Playgrounds 2.0 for iOS appears to represent Apple’s commitment to driving that product forward into the future. The latest version of Xcode Playgrounds, on the other hand, offers a lackluster interface, slow responsiveness, and a tendency to crash both within the Playground, and in ways that take down the entire Xcode app. In short: they’re not a very fun place to play.

I propose that Apple eliminate Xcode playgrounds, and invest all of their work in the field of interactive coding into the Swift Playgrounds app. It has been a nagging shortcoming that Swift Playgrounds is only available for iOS. Many people who would benefit from the educational opportunities of Swift Playgrounds could do so from Macs, whether in schools, homes, or workplaces. Porting Swift Playgrounds to the Mac would address that problem.

Where does that leave developers? After eliminating Xcode Playgrounds as we know them today, I envision adapting the Mac version of Swift Playgrounds so that playgrounds can be run either independently in a Swift Playgrounds app, or in “developer mode” within Xcode. In effect, Xcode would become a dedicated Swift Playgrounds authoring app, where such authoring capabilities would incidentally provide all the benefits that standalone Xcode Playgrounds currently provide.

Taking this course would allow Apple to maximize the output of its engineering and design efforts while eliminating the naming confusion that currently exists between Swift Playgrounds and Xcode Playgrounds.

For students and educators, it would broaden device requirements for Swift Playground materials, opening up learning opportunities for people who have access to Macs but not to iOS devices.

Finally, and perhaps most importantly for the developer ecosystem as a whole, it would eliminate the frustratingly problematic Xcode Playgrounds and hopefully provide developers with something more inspiring, more functional, and more reliable.

Radar #36910249.

App Store Upload Failures

I’ve been running into failures to connect to iTunes Connect through Application Loader. Others corroborate similar problems uploading through Xcode. The nut of the problem comes down to a failure to authenticate a specific Apple ID. The failure string is “Unable to process authenticateWithArguments request at this time due to a general error”:

Image of the Sign In window for Application Loader showing a failure message

Whatever is going on, it’s somewhat sporadic. It started failing for me about 48 hours ago, and briefly started working again yesterday, but only for a few hours. Changes of network, computer, even clearing out all the app preferences and data for Application Loader, seem to have no impact on the issue.

I’ve filed a bug with Apple (Radar #36435867), and discovered a workaround you can use in a pinch: add another Developer ID to your team. I was able to grant admin privileges to a second Apple ID I control, and use it to log in and upload to my account. I’m not sure if this workaround requires a “company” style developer account, or if it can also be used for individual accounts.

Intrinsic String Encoding

I was baffled today while investigating a bug in MarsEdit, which a customer reported as only seeming to affect the app when writing in Japanese.

I pasted some Japanese text into the app and was able to reproduce the bug easily. What really confused me, though was that the bug persisted even after I replaced the Japanese text with very straight-forward ASCII-compatible English. I opened a new editor window, copied and pasted the English text in, and the bug disappeared. I copied and pasted back into the problematic editor, and the bug returned. What the heck? Two windows with identical editors, containing identical text, exhibiting varying behavior? I knew this was going to be good.

It turns out there’s a bug in my app where I erroneously ask for a string’s “fastestEncoding” in the process of converting it. The bug occurs when fastestEncoding returns something other than ASCII or UTF8. For example, with a string of Japanese characaters, the fastestEncoding tends to be NSUnicodeStringEncoding.

But why did the bug continue to occur even after I replaced the text with plain English? Well…

The documentation for NSString encourages developer to view it as a kind of encoding-agnostic repository of characters, which can be used to manipulate arbitrary strings, converting a specific encoding only as needed:

An NSString object encodes a Unicode-compliant text string, represented as a sequence of UTF–16 code units. All lengths, character indexes, and ranges are expressed in terms of 16-bit platform-endian values, with index values starting at 0.

This might lead you to believe that no matter how you create an NSString representation of “Hello”, the resulting objects will be identical both in value and in behavior. But it’s not true. Once I had worked with Japanese characters in my NSTextView, the editor’s text storage must have graduated to understanding its content as intrinsically unicode based. Thus when I proceeded to copy the string out of the editor and manipulate it, it behaved differently from a string that was generated in an editor that had never contained non-ASCII characters.

In a nutshell: NSString’s fastestEncoding can return different values for the same string, depending upon how the string was created. An NSString constant created from ASCII-compatible bytes in an Objective-C source file reports NSASCIIStringEncoding (1) for both smallest and fastest encoding:

printf("%ld\n", [@"Hello" fastestEncoding]);	// ASCII (1)

And a Swift string constant coerced to NSString at creation behaves exactly the same way:

let helloAscii =  "Hello" as NSString
helloAscii.fastestEncoding			// ASCII (1)

But here’s the same plain string constant, left as a native Swift String and only bridged to NSString when calling the method:

let helloUnicode = "Hello"
helloUnicode.fastestEncoding		// Unicode (10)

As confusing as I found this at first, I have to concede that the behavior makes sense. The high level documentation describing NSString representing “a sequence of UTF-16 code units” says nothing about the implementation details. It’s a conceptual description of the class, and for the most part all methods operating on an NSString comprising the same characters should be heave the same way. But the documentation for fastestEncoding is actually pretty clear:

“Fastest” applies to retrieval of characters from the string. This encoding may not be space efficient.

As I said earlier, my usage of fastestEncoding was erroneous, so the solution to my bug involves removing the call to the method completely. In fact, I don’t expect most developers will ever have a legitimate needs to call this method. Forthose who do, be very aware that it can and does behave differently, depending on the provenance of your string data!

Treat Warnings as Errors in Swift

For years I have maintained a zero-tolerance policy for warnings in shipping code. To help me enforce this, my “Release” build configurations define the build setting that induces Xcode to “treat warnings as errors”:


The comedy of this build setting is that it references “GCC,” a compiler that increasingly few of us even remember Apple ever using for macOS or iOS development. Apple’s fork of the popular open-source compiler was the standard for years, until Apple debuted clang, which is built on the LLVM framework.

It’s always been kind of annoying that, as time moved on, we were stuck specifying important build settings with names that included “GCC,” but I accepted it. After all, I configure the vast majority of these settings in “.xcconfig” files that I reuse in all my projects, so I almost never interact with them directly.

As I’ve started to shift some of my code from Objective-C to Swift, I assumed that my ancient GCC build setting would ensure that I don’t ship any builds with warnings. But today I realized that I had built a release build that didn’t treat a warning as an error. What’s the deal?

It looks like Apple has decided to break with tradition and establish a new build setting for Swift:


Adding this to my .xcconfig file breaks the build when a warning is generated, and prevents me from accidentally shipping code that is known to be vulnerable to unexpected behavior. I think it would have been a nice touch if Apple had inferred SWIFT_TREAT_WARNINGS_AS_ERRORS when GCC_TREAT_WARNINGS_AS_ERRORS is set, so I’ve filed a bug requesting that it does. Radar #35352318.

Selective Selector Mapping

I ran into an interesting challenge while porting some Objective-C code to Swift. The class in question served both as an NSTableView delegate and data source, meaning that it implemented methods both for controlling the table view’s behavior and for supplying its content.

Historically in Cocoa, most delegate relationships were established as informal protocols. If you wanted a particular class to be a table view data source, you simply implemented the required methods. For example, to populate a cell based table view, a data source would implement various methods, including one to indicate how many rows the view should have:

- (NSInteger) numberOfRowsInTableView:(NSTableView *)tableView;

In recent years, Apple has increasingly converted these informal protocols to formal Objective-C protocols. These give the compiler the opportunity to generate errors if a particular class declares compliance, but neglects to implement a required method. At runtime, however, the compliance-checking is still pretty loose. NSTableView consults its data source, checks to see that it implements a required subset of methods, and dynamically dispatches to them if it does.

The dynamic nature of NSTableView hasn’t changed with Swift. An @objc class in Swift that complies with NSTableViewDataSource must still implement the required methods such that Apple’s Objective-C based NSTableView can dynamically look up and dispatch to the required delegate methods. Swift’s method rewriting “magic” even ensures that a delegate method can be written in modern Swift style, yet still appear identically to older Objective-C code:

class MyDataSource: NSObject {
	@objc func numberOfRows(in tableView: NSTableView) -> Int {
		return 0

Given an instance of MyDataSource, I can use the Objective-C runtime to confirm that a the legacy “numberOfRowsInTableView:” selector is actually implemented by the class above:

let thisSource = MyDataSource()
thisSource.responds(to: Selector("numberOfRowsInTableView:")) // false

Or can I? False? That’s no good. I’m using the discouraged “Selector” initializer here to ensure I get Swift to look for a very specific Selector, even if it doesn’t appear to be correct to the Swift-adapted side of the runtime.

I was scratching my head, trying to figure out why Objective-C could not see my method. Did I forget an @objc marker? No. Did I forget to make MyDataSource a subclass of NSObject? No. I finally discovered that I could second-guess the default Swift selector mapping to obtain a result that “worked”:

class MyDataSource: NSObject {
	@objc func numberOfRowsInTableView(_ tableView: NSTableView) -> Int {
		return 0

let thisSource = MyDataSource()
thisSource.responds(to: Selector("numberOfRowsInTableView:")) // true

Instances of MyDataSource will get the job done for Objective-C calls to “numberOfRowsInTableView:”, but I’ve lost all the pretty formatting that I expected to be able to use in Swift.

There’s something else I’m missing out in my Swift implementation: type checking of MyDataSource’s compliance with the NSTableViewDataSource protocol. Old habits die hard, and I had initially ported my class over with an old-fashioned, informal approach to complying with NSTableViewDataSource: I declared a plain NSObject that happens to implement the informal protocol.

It turns that adding that protocol conformance onto my class declaration not only gains me Swift’s protocol type checking, but changes the way key functions are mapped from Swift to Objective-C:

class MyDataSource: NSObject, NSTableViewDataSource {
	func numberOfRows(in tableView: NSTableView) -> Int {
		return 0

let thisSource = MyDataSource()
thisSource.responds(to: Selector("numberOfRowsInTableView:")) // true

Armed with the knowledge that my class intends to comply with NSTableViewDataSource, Swift generates the expected mapping to Objective-C. Notice in this final case, I don’t even have to remember to mark the function as @objc. I guess when Swift is creating the selector mapping for a function, it does so in a few phases, prioritizing more explicit scenarios over more general:

  1. First, it defers to any explicit annotation with the @objc attribute. If I tag my “numberOfRows…” func above with “@objc(numberOfDoodads:)” then the method will be made available to Objective-C code dynamically looking for “numberOfDoodads:”.
  2. If there’s no @objc specialization, it tries to match function implementations with declarations in superclasses or protocols the class complies with. This is what gives us the automatic mapping of Swift’s “numberOfRows(in:)” to Objective-C’s “numberOfRowsInTableView:”.
  3. Finally it resorts to a default mapping based on Swift API Design Guidelines. This is what yielded the default “numberOfRowsIn:” mapping that I first encountered.

This is an example of a Swift growing pain that is particularly likely to affect folks who are adapting older source bases (and older programming mindsets!) to Swift. If you run across a completely vexing failure of Objective-C to acknowledge your Swift class’s protocol compliance, start by making sure that you’ve actually declared the compliance in your class declaration!

Swatch Your Step

Shortly after macOS 10.13 was released, I received an oddly specific bug report from a customer, who observed that the little square “swatches” in the standard Mac color panel no longer had any effect on MarsEdit’s rich text editor.

Screenshot of the macOS standard color panel.

I was able to reproduce the problem in the shipping 3.7.11 version of MarsEdit, which for various reasons is still built using an older version of Xcode, against the 10.6 SDK. The MarsEdit 4 Beta, which is built against the 10.12 SDK, does not exhibit the problem.

It’s not unusual for the behavior of Apple’s frameworks to vary based on the version of SDK an application was built against. The idea is usually to preserve the old behaviors of frameworks, so that any changes do not defy the expectations of a developer who has not been able to build and test their app against a later SDK. Sometimes, the variations in behavior lead to bugs like this one.

Using a totally straightforward demo app, consisting only of an NSTextView and a button to bring up the color panel, I was able to confirm that the bug affects an app that links against the macOS 10.9 SDK, but does not affect an app that links against the 10.10 SDK.

I filed Radar #34757710: “NSColorPanel swatches don’t work on apps linked against 10.9 or earlier.” I don’t know of a workaround yet, other than compiling against a later SDK.

Xcode 9 Signing Workarounds

I wrote on Monday about issues with Xcode 9 relating to code signing. Although the gist of that post involved sandboxed Mac applications that launch sandboxed child processes, the fundamental issue is a bit broader: Xcode 9 adds a “com.apple.security.get-task-allow” entitlement to any binary it signs. For the majority of developers, this is probably not an issue, because the entitlement is removed when an Xcode archive is exported for distribution. Most developers, and particularly iOS developers, use Xcode archives.

For folks who don’t, side effects of this additional entitlement include, but may not be limited to:

  1. Inability to launch sandboxed child processes.
  2. Rejection from the Mac App Store.
  3. Unknown consequences of shipping with an unintended entitlement.

So, if you’re a developer who doesn’t use archives, what are your options? I’ve come up with four workarounds, and I present them here, roughly sorted by advisability and level of tedium:

  1. Use Xcode 8. The simplest solution is to not upgrade to Xcode 9 unless and until you need to. Xcode 8’s signing process does not impose the unintended entitlement, so there is no risk of shipping a product that has it, unless you add it yourself. The downside to sticking with Xcode 8 is you won’t enjoy any of the new features of Xcode 9, you’ll have to work to support either Swift 4, macOS 10.13, or iOS 11 SDK features in your app.

  2. Manually re-sign the built-product. Code signing is code signing, and you’re free to sign anything you like to suit your needs, using the “codesign” command line tool. It frankly sounds like a pain in the neck to recursively re-sign every binary in the app bundle, ensuring that the suitable entitlements (minus the unwanted one) are preserved, but I’m sure it can be done.

  3. Use Xcode archives. It strikes me as a little obnoxious to have to use Xcode archives when they don’t offer any added benefits for my dibstrution workflow. But as a long term solution, this is probably the safest bet. The new behavior in Xcode 9 strongly suggests that Apple expects most developers to use archives, and joining the crowd is usually a good idea when it comes to avoiding trouble with Apple’s developer tools.

    If you are using Xcode archives for the first time, particularly with a complex project, you might discover that the resulting archives are not suitable for exporting a signed application. If you get a “Generic Xcode Archive” after running Build -> Archive, you know you’ve got a problem. By default the archive process builds all targets with an “install” option, rendering their built products into a file hierarchy that will be used to build the archive. If your project includes helper apps, for example, they will be “installed” alongside your main app, resulting in a generic archive of two apps, instead of the expected archive of a single app.

    The solution for this problem is to ensure that the “SKIP_INSTALL” build setting is set to YES for any such helper app. Just archive your main app, export the “Built Products” from the resulting archive, and look at the file hierarchy to determine whether you have subtargets that need to have installation disabled.

  4. Hack Xcode 9. In a hurry to ship an update to your app, and you’ve only got Xcode 9 handy? It turns out the imposition of this “com.apple.security.get-task-allow” entitlement is controlled by a single property list file inside Xcode’s application bundle. As a test, I edited the file:


    It contains a single entitlement, the one that’s causing our grief. I deleted the entitlement from the list, saved the file, and relaunched Xcode. After doing so, everything is “back to normal.”

    I can’t strongly encourage you to hack your copy of Xcode because I don’t know what the consequences might be. “It seems fine,” but you’re on your own if you decide to do this.

This small change in Xcode 9 causes a lot of unexpected grief for folks who don’t use Xcode archives. I am curious to know how widespread the problem is, and enthusiastic to get the word out about it so that affected folks can work around the problem, or at least be aware of it. Myself, I’ll probably end up adopting the workaround of using Xcode archives, but I’m hopeful that Apple will see the merit of providing an option in an update to Xcode 9 that supports disabling the addition of this entitlement without archiving and exporting a built product.

Unordered Directory Contents

Since I updated to macOS 10.13 High Sierra, some of my unit tests broke. Examining the failures more carefully, I discovered that they were making assumptions about the order that Foundation’s FileManager.contentsOfDirectory(atPath:) would return items.

I wrote a quick playground to test the behavior on a 10.12 machine:

import Foundation

let array = try! FileManager.default.contentsOfDirectory(atPath: "/Applications/Utilities")

The results come back alphabetically ordered by file name:

[".DS_Store", ".localized", "Activity Monitor.app", "Adobe Flash Player Install Manager.app", "AirPort Utility.app", "Audio MIDI Setup.app", "Bluetooth File Exchange.app", "Boot Camp Assistant.app", "ColorSync Utility.app", "Console.app", "Digital Color Meter.app", "Disk Utility.app", "Grab.app", "Grapher.app", "Keychain Access.app", "Migration Assistant.app", "Script Editor.app", "System Information.app", "Terminal.app", "VoiceOver Utility.app"]

The same playground on 10.13 tells a different story:

["AirPort Utility.app", "VoiceOver Utility.app", "Terminal.app", "Activity Monitor.app", ".DS_Store", "Grapher.app", "Audio MIDI Setup.app", ".localized", "System Information.app", "Keychain Access.app", "Grab.app", "Migration Assistant.app", "Script Editor.app", "ColorSync Utility.app", "Console.app", "Disk Utility.app", "Bluetooth File Exchange.app", "Boot Camp Assistant.app", "Digital Color Meter.app"]

I thought at first this might have been related to the APFS conversion that 10.13 applied to my boot volume, but the same ordering discrepancy occurs for items on my HFS+ volumes as well.

After checking the 10.13 release notes for clues, and finding none, I consulted the documentation. Well, what do you know?

The order of the files in the returned array is undefined.

So, mea culpa. The test code in question probably shouldn’t have ever made assumptions about the ordering of items returned from this method. While it has evidently always been undefined, it appears they are only making good on that promise in 10.13. You have been warned!

Update: It turns out I have some real bugs in my apps, not just in my tests, because of assuming the results of this call will be reasonably sorted. Luckily I use a bottleneck method for obtaining the list of files, and I can impose my own sorting right at the source. If you’re looking to make the same kinds of changes to your app, be sure to heed Peter Maurer’s advice and use “localizedStandardCompare” (available since macOS10.6/iOS4) to obtain Finder-like ordering of the results.

Sandbox Inheritance Tax

I ran into a subtle bug with Xcode 9 that I think is worth sharing. Specifically, this bug affects Mac applications that:

  1. Are sandboxed.
  2. Launch a sandboxed subprocess with NSTask (or posix_spawn).
  3. Configure the subprocess to inherit the parent’s sandbox.

When such an app is compiled with Xcode 9, the subprocess will crash whenever the parent process launches it. A canonical example of something that might suffer from this problem is a bundled crash-monitor. I embed one with my apps to keep an eye on the running status of the parent process, and to present a crash-reporting interface to users if the host app terminates prematurely. When I build and run my app with Xcode 9, the bundled crash monitor dies instantly upon being launched.

It took me a while to realize that the subprocess is dying because it fails to satisfy the contract for inheriting a sandbox. From Apple’s “Enabling App Sandbox Inheritance“:

To enable sandbox inheritance, a child target must use exactly two App Sandbox entitlement keys: com.apple.security.app-sandbox and com.apple.security.inherit. If you specify any other App Sandbox entitlement, the system aborts the child process.

Well, that’s funny because my child process does specify only those two keys, but the system is aborting it anyway. It turns out that Xcode 9 is inserting a third entitlement without my permission. Clicking on the detail of the “Process Product Packaging” build phase in Xcode’s log navigator, I can see that there are three entitlements for my target:

Xcode build log detail showing the wrong entitlements.

When my subprocess is launched, the system sees that extra “com.apple.security.get-task-allow” entitlement in the context of “com.apple.security.inherit”, and unceremoniously crashes my the child process.

I’m not sure what Apple’s reasoning is for imposing this entitlement on sandboxed targets, but it appears to be doing so across the board, for literally every sandboxed target in my app. I confirmed that all of my apps, XPC processes, helper tools, etc., are all getting this bonus entitlement.

I searched Xcode’s files, and discovered the entitlement listed in this file inside the Xcode app bundle:


Putting aside the question of whether it’s appropriate for Xcode to surreptitiously add entitlements that are not specified by the developer’s own list of permissions, the addition of the entitlement for these particular targets, ones that inherit their parent’s sandbox, turns out to be a fatal move.

Ideally I would be able to work around this by adding a custom build phase to manually tweak the generated entitlements file, removing the unwanted key. But the “Process Product Packaging” build phase happens so late in the build process that it’s after the last user-specified custom build phase. There’s no room in Xcode’s current design for fixing up the problematic entitlements before they are incorporated into the signed product. As far as I can tell the only clean workaround would be to redundantly re-sign the child app with a custom script, and corrected entitlements, after Xcode’s build process is completed.

I filed Radar #34628449, “Sandboxed project build with Xcode 9 cannot launch child process.”

Update: Colin Barrett pointed out on Twitter that the entitlement in question here, “com.apple.security.get-task-allow”, may be required in order to attach to and debug a process. If true, then I think this is something that was handled in a different way in Xcode 8. I can confirm that my apps do not have the entitlement imposed on them by Xcode 8, yet I am able to attach to and debug them.

If Apple changed the debugger infrastructure in Xcode 9 so that the relationship between the debugger and target processes is more locked down, requiring a specific entitlement, then that’s probably a good thing. But if this change was made without thinking about the implications for the above-cited “strict two entitlement” rule for sandbox inheritance, then probably some flexibility needs to be applied to that rule.

Finally, as I noted above the entitlement is being applied to all my targets. What I didn’t clarify is that the entitlement is added even when Building and Archiving. A release build’s binaries are endowed with this additional entitlement, which may also bring additional security vulnerabilities to the app.

I would not ship a sandboxed Mac app that is built with Xcode 9, until we understand more about when Xcode applies this entitlement, and whether it can be prevented for Release builds at the very least.

Update 2: I’ve learned that Xcode’s “Export Archive” functionality causes the unwanted entitlement to be removed. Apparently the assumption is that everybody creates Xcode archives as part of their build and release process. I am sure this is true for most (all?) iOS deployments, but for Developer-ID signed apps on the Mac, there has traditionally been less of an incentive to do this. Got a properly signed Mac application? Zip it up, put it on a web server, and you’re done.

I’m not sure yet whether I’ll switch my build process to use archiving, or whether I’ll pull some other stunt to redo the code signing with corrected entitlements. In any case this has been quite an adventure today getting to the bottom of this. I updated my bug report with Apple to request that they provide some standard build flag that would prevent the problematic entitlement from being added from the start. In the mean time, I’ll explore one of the workarounds and get my builds back to fully functional!

Better Swift Completion

Apple released Xcode 9 earlier this week, and in spite of a few glitches here and there, I have found the update to be an overall improvement over Xcode 8. It’s nice that Apple continues to invest in the core tools for Mac and iOS developers.

I’ve been dabbling in more and more Swift development lately, and it’s brought to light a shortcoming in Xcode’s code completion which has unfortunately not improved in Xcode 9: completion of Swift function calls when there is a large quantity of candidates.

Take for example NSAttributedString. If I want to initialize a new instance in Swift, I type “NSAttributedString(” to bring up the list of compatible init methods I can choose from:


The problem at this point is that I have to navigate the menu by hand. I can’t narrow down the list of completions any further by typing, because the very next character I type will be interpreted as the manual filling out of parameters of the NSAttributedString initializer.


This is a situation where Objective-C gets much nicer treatment in the editor. Because completion in Objective-C begins when I start typing “init”, and because the named first parameter is part of the init message name, I can winnow down the results quite a bit:

Pasted Image 9 22 17 11 24 AM

Better still, because Xcode performs a fuzzy match on the typing, I can proceed to type the names of additional parameters to zero in completely on the variation I want:

MEAppController AppDelegate m Edited

When I accept the completion, all of my typing is replaced with the expected, templated parameter placeholders for the chose initializer.

I filed Radar #34594940 requesting better completion for Swift.