App Sandbox Control Tool

I wrote only yesterday about how troubling I find the opaqueness of Apple’s App Sandbox.

Well, don’t I feel ignorant today.

% man asctl

NAME
     asctl -- App Sandbox Control Tool

SYNOPSIS
     asctl [-p] [-l] command [arguments]

DESCRIPTION
     asctl is a facility for manipulating the filesystem container for an applications using App Sandbox.  A container is a per-application filesytem hierarchy rooted in ~/Library/Containers.

This tool appears to offer extensive insight into the sandbox’s understanding of containers. I wish I had discovered it earlier!

Sandbox Container Ownership

I have a knack for finding bugs and edge cases, and my experience adapting to the Mac sandboxing facility over the past few years has been no exception.

The latest issue is something of a conundrum: I’m managed to produce two versions of my app, one of which causes the sandbox container to be apparently unwritable to the other after running! Specifically, preferences are not saved and console messages indicate an attempt to write preferences outside the host app’s sandbox.

I suspect it has something to do with changes I’ve made in anticipation of a bundle identifier change. As with so many of my struggles with sandboxing, I’m trying to smooth out the process of migration from one sandbox container to another. To that end I have added some temporary entitlements for preferences and shared file access, as well as modifying the code signing designated requirement to cover both bundle identifiers.

Long story short: if I run app version 1, then run app version 2, then run app version 1, app version 1 is denied access to the container (which had previously been its own!).

These kinds of issues scare the bejeezus out of me because I really fret my users running into data migration problems after I ship an update, and because the relative opacity of the sandboxing system makes a lot of issues very hard to debug.

I suspect there is some association made between a sandbox container and the owning app’s designated requirement, and that perhaps by changing it I’ve yielded ownership to the later version of the app. The Info.plist file inside a sandbox container has various keys such as “Identity” and “SandboxProfileData” which seem likely to pertain to this issue.

Apple supplies an App Sandbox in Depth which alludes to how ownership is determined, but alas does not go quite as deep as I’d like. Has anybody dived deeply enough into the sandbox to understand this issue well?

Update: I probably should have scratched more than the surface of the Info.plist before posting. There are other interesting keyed values in there such as SandboxProfileDataValidationInfo that includes keyed values such as full paths of various apps that share ownership (?) in the container. In summary: I guess I’m curious if anybody has dealt with issues like the one I’m seeing as has debugging tips to share.

Swifty SDK Changes

I’m not quite ready to leap to Swift 3.0, so when the Xcode 8 beta asked me to migrate my existing Swift code, I opted to switch to Swift 2.3.

Swift 2.3 is notable for its utter dearth of new behaviors, compared to Swift 2.2.1. Since no code changes are required, you should be able to opt in to 2.3 on Xcode 8, while continuing to build and distribute with Swift 2.2.1 on Xcode 7.

On the other hand, you may run into some system SDK changes that force you to accommodate distinctions between, for example, the macOS 10.11 and 10.12 SDKs. Here’s an example of a build failure I ran into when I set to building my app on Xcode 8:

Image of Xcode exhibiting a build-time error because of a Swift method call

“invalid use of ‘()’ to call a value of non-function type ‘Selector'”

What does it mean? It means that in the 10.11 SDK, “action” is defined as a method:

@protocol NSValidatedUserInterfaceItem
- (nullable SEL)action;
- (NSInteger)tag;
@end

While in the 10.12 SDK, it’s been upgraded to a property:

@protocol NSValidatedUserInterfaceItem
@property (readonly, nullable) SEL action;
@property (readonly) NSInteger tag;
@end

Objective-C doesn’t care whether you access the value as a property, or by invoking the method that implements the property. Swift … does.

So, the fix in Xcode 8, with the 10.12 SDK, is to delete those parentheses so Swift appreciates that it’s accessing a property. But doing so will break the build back on the 10.11 SDK. What’s a forward-looking, backward-looking developer to do? You might think you could take advantage of Swift’s #available operator, and that in fact best expresses what we’re trying to do: to behave differently depending on the specific SDK that is being used:

if #available(OSX 10.12, *) {
	thisAction = anObject.action
}
else {
	thisAction = anObject.action()
}

But this still produces a build error, because Swift compiles both code paths of an #available block. Unfortunately, I think the only way to convince Swift to absolutely ignore the problematic line is to use a Swift compiler check, instead. Note that this is a bad solution because we are fundamentally not concerned about the version of Swift being used. If Swift 2.3 were made available in Xcode 7, against earlier SDKs, then the logic of our test would fail:

#if swift(>=2.3)
	thisAction = anObject.action
#else
	thisAction = anObject.action()
#endif

In my opinion, we need a compile time, conditional code exclusion that operates on the SDK, similarly to #available. Something like “#if sdk(OSX 10.12)”. As far as I know, nothing like this exists.

I’ll report a bug to Apple requesting such functionality, but I wonder if I’m overlooking something obvious that already fits the bill. Any ideas?

Update: Radar #26895983.

Dynamic Development Team

The first Xcode 8 beta was released at WWDC, and includes a major revision to the way code signing is handled by default. You can watch the video or read the slides on Apple’s WWDC Videos page.

A major change from Xcode 7 is that Xcode now requires that every target in your project specify a development team, even when the manual code signing option is selected.

This is a nuisance to developers who have fine-tuned their code signing process such that signing is configured elsewhere: for example, in a centralized Xcode configuration file that imparts signing information to all projects in an organization.

The problem is worse for open source projects, where a project may be expected to be checked out and built by a variety of organizations. In this scenario, each organization will have to somehow automate the insertion of a valid team name and ID in the project file after every checkout or update.

The DevelopmentTeam and DevelopmentTeamName values are stored in the Xcode project file, so they can’t be defined in Xcode configuration files the way so many other settings that affect build behaviors can.

Xcode needs some mechanism for providing these values without manually, permanently altering the affected project files. I would welcome either a mechanism for specifying the values in the Xcode configuration file specified for the project, or a wildcard type value that can be selected to indicate the “default team” should be used.

(Radar #26892618)

Update: It turns out my wishes are already answered. The new PROVISIONING_PROFILE_SPECIFIER build setting will impart the desired team information to Xcode, even if provisioning profiles are not used. So, something like this in my Xcode configuration file:

PROVISIONING_PROFILE_SPECIFIER = 493CVA9A35/

Quiets all of Xcode’s gripes about requiring a team name, and doesn’t require that I tediously update every last target with explicit team information. This doesn’t exactly solve the challenge for open source projects, because organizations using an open source project will still have to override the configuration file or build settings for the projects they check out. As far as I know there is no way to define PROVISIONING_PROFILE_SPECIFIER generically, the way you can specify code signing identities as e.g. “Mac Developer”. It would be great if open source projects could declare a PROVISIONING_PROFILE_SPECIFIER (when needed) as:

PROVISIONING_PROFILE_SPECIFIER = Automatic/

And allow the host project’s prevailing team identity automatically apply. But, I am satisfied to know this at least doesn’t need to be manually set everywhere in order to continue building my projects successfully.

Nullable Edge Cases

I’ve recently embarked on the tedious process of annotating my source code for nullability. This is a good idea in theory, because it adds information about assumptions in your code, which may have been previously held in arcane comments, or worse yet your mind, in a format that can be readily understood by the compiler. If you’re bridging your Objective C to Swift, the idea goes from good to essential, as the nullability information is critical to Swift working safely with your existing classes.

Does nil make sense here or not? If you decide to take on this task, you’ll ask yourself that question again, and again, and again. It sounds like such an easy thing to answer, and it certainly is in many cases, but the edge cases are deeply tied to one of several impedance mismatches between Objective C and Swift.

The easiest answers apply to methods where you can state with absolute certainty that programmers should never pass nil as a method. For example, I’m confident in annotating this method’s single parameter nonnull because the functionality of the method would be inherently undefined were a nil value supplied:

- (void) presentMessageToUser:(NSString*)theMessage;

The implications of marking a method parameter as nonnull are less worrisome than marking a return value, at least with respect to bridging the gap between Swift and Objective C. Because a method parameter will almost always represent the mapping of a Swift variable into Objective C, where handling of nil is traditionally safer, it doesn’t matter as much if a Swift variable passed to a nonnull Objective C parameter actually an optional. Marking method parameters optimistically nonnull is a fairly safe move.

Marking return values accurately is more important, because it maps possibly null values into Swift variable classes that won’t allow it. The annotation of a return value can be very straightforward, if inspection of the method indicates a 100% likelihood of a null response:

- (NSString*) importantString
{
	NSLog(@"Not implemented! Subclass responsibility.");
	return nil;
}

Or a nonnull one:

- (NSString*) errorMessage
{
	return @"Welp!";
}

For this method to return nil, the statically allocated string that the compiler creates and compiles into your binary would have to somehow turn to nil at runtime. If this were to somehow occur, you’d be facing much worse problems than the nonnull method managing to return nil.

It’s only slightly more complicated when the code path for a method can be audited to a degree that a nil result seems exceedingly unlikely:

- (NSString*) localizedErrorMessage
{
	return NSLocalizedString(@"Welp!", @"Error message of last resort");
}

For this method to return nil, NSLocalizedString, which is a macro that maps to -[NSBundle localizedStringForKey:value:table:], would have to return nil. This method is not only annotated by Apple as nonnull, but supported in its documentation as returning sensible, nonnull values in error cases:

This method returns the following when key is nil or not found in table:

  • If key is nil and value is nil, returns an empty string.
  • If key is nil and value is non-nil, returns value.
  • If key is not found and value is nil or an empty string, returns key.

So, any method whose return value is derived from a nonnull method, can also be annotated as nonnull. Right? Yes, for the most part.

The question of whether a method should return nil vs. whether it may return nil gets very fuzzy around the edges, owing to Objective C’s traditionally very nil-friendly (some might say nil-happy) programming paradigm. Let’s take a core method, one of the most fundamental in all of Objective C, the NSObject init method. Barring an override in any Objective C class, this method will be used to initialize and return a usable instance of any just-allocated class:

- (instancetype) init;

But is it nullable? In practice, NSObject’s root implementation, at least, should never return nil. It’s safer even than -[NSObject alloc], which should also never return nil, but theoretically could if you for example had completely exhausted virtual memory. Want to absolutely convince yourself that -[NSObject init] cannot return nil? Break in the debugger and examine its disassembly:

libobjc.A.dylib`-[NSObject init]:
    0x7fff8909ebf0 <+0>:  pushq  %rbp
    0x7fff8909ebf1 <+1>:  movq   %rsp, %rbp
    0x7fff8909ebf4 <+4>:  movq   %rdi, %rax
    0x7fff8909ebf7 <+7>:  popq   %rbp
    0x7fff8909ebf8 <+8>:  retq   

To summarize the behavior of this simple method, literally all it does, apart from the probably unnecessary stack-manipulating boilerplate, is to move the parameter in register %rdi (the first parameter to objc_msgSend, the object instnace itself) to register %rax (the return value of the method). It’s just “return self”, and self has to be nonnull or else this method wouldn’t have been reached.

Yet if you examine the objc/NSObject.h header file, where -[NSObject init] is declared, you’ll find something curious: it’s not annotated for nullability at all. And although I showed by disassembly above that it is guaranteed 100% to be nonnull, here’s what Apple’s own documentation says:

Return Value
An initialized object, or nil if an object could not be created for some reason that would not result in an exception.

Although this is declared on the documentation for NSObject, it’s no doubt based on the Cocoa convention that any class’s own particular -init method may and in fact should return nil if it cannot be initialized:

In a custom implementation of this method, you must invoke super’s designated initializer then initialize and return the new object. If the new object can’t be initialized, the method should return nil.

NSObject’s init method is one example of many situations in Objective C where nil is an expected return value in edge cases, but where as a general rule, a nonnull value will be returned. This is at huge odds with Swift’s emphasis on variable values being either nonnull or nullable by design.

In recent Xcode releases, Apple has added a useful clang analysis option, off by default, called CLANG_ANALYZER_NONNULL. When enabled, the analyzer goes the extra mile identifying situations where your code contains paths that will violate the spirit of your nonnull annotations. This has been very useful to me in identifying some spots where I missed a nuanced behavior of a method when adding nullability annotations. However, it also identifies a lot of defensive coding techniques that I’m frankly not prepared to abandon. The Objective C, it is strong in my family. Here is an example of a category class method I’ve defined on NSImage (explicit nonnull annotations added for emphasis):

+ (nonnull NSImage*) rsImageWithCGImage:(nonnull CGImageRef)srcImage
{
	NSImage *newImage = nil;
	if (srcImage != NULL)
	{
		NSBitmapImageRep *bitmapImageRep = [[NSBitmapImageRep alloc] initWithCGImage:srcImage];
		if (bitmapImageRep != nil)
		{
			newImage = [[NSImage alloc] init];
			[newImage addRepresentation:bitmapImageRep];
		}
	}

	return newImage;
}

Clang analysis rightly reveals that although I’ve marked this method as having both a nonnull parameter and a nonnull result, the implementation of the method behaves otherwise. But this method is exceedingly unlikely to return nil. The srcImage parameter is marked nonnull, so that first line of defense should always be breached, and the -[NSBitmapImageRep initWithCGImage:] is so likely to return a nonnull value, it is annotated as nonnull (even though the documentation claims it may return nil, Radar #26621826).

I struggle with methods like these: do I mark them optimistically as nonnull, or do I suck it up and concede that they may in fact return nil? Do I remove the defensive checks for nil that I’m so accustomed to, and migrate to the Swift-style assumption that nonnull values will be as they say they will? I clearly have trust issues.

The consequences of vending a null value to a Swift non-optional are dire: the app crashes immediately either unwrapping an implicitly unwrapped optional, or directly accessing a nonnull value. Because of this, I think that in annotating Objective C code for nullability, one must err heavily towards marking return values as nullable, even if such a result is unlikely. How heavily will I err? Will I assume that every -init method may return nil, as indeed Apple has documented? I … well, no. That way lies madness. But wherever I felt compelled to put up overt safeguards in Objective C to handle possible nils from Apple’s frameworks, there is probably still cause for skepticism, even if the method has been marked as nonnull.

The down side of course is that my Swift code will have to jump through mostly unnecessary optional chaining, or other protections, to work safely with what is 99.9% guaranteed to be safe. But in the rare instances where nil is returned, especially when the circumstances that lead to it are out of my control in Apple’s own frameworks, I’d rather my Swift code behave a little bit more like my beloved old Objective C code, and fail gracefully on null.

Implicitly Crashing Optionals

If you’re an old-time Objective C programmer like me, your first effort to add a Swift file to your project will be met with a cheerful offer from Xcode to “add a bridging header.” This header accommodates the Swift compiler generating Swift interfaces for all the existing Objective C classes you’ll no doubt want to interface with.

At first, I didn’t think too much of this bridging header. Sure, I want to access my Objective C files from Swift. Which ones? Why not all of them?

Life goes on, and as you proceed to write more and more Swift files, you’ll inevitably come to a point where your app crashes because an “implicitly unwrapped optional” turns out to be nil. What happened?

If you’re like me and you haven’t gotten around to annotating all your Objective C headers with nullability tags, Swift imports almost every pointer type as an implicitly unwrapped optional. This means it will be treated as a value that can be nil but is never expected to be nil by the time you access it.

This stuff is all well-covered in Apple’s documentation, but I have never been one to jump straight to RTFM. :)

My takeaway is to accept that it is fundamentally unsafe to interface with Objective C classes whose headers have not been audited for nullability. So back to that bridging header… 

I now impose a rule for my Objective C -> Swift bridging header that any import added to the file must first be confirmed as audited for nullability. If it’s not audited yet, I commence with the somewhat tedious task of annotating (with the help of NS_ASSUME_NONNULL_BEGIN and NS_ASSUME_NONNULL_END, of course) the entire header file, sometimes searching the correlated source files to confirm assumptions about nullability of parameters. When I’m done, I can (relatively) safely import the header into Swift and count on appropriate optional type checking. Except…

In addition to the imported header, of course, I need to audit any headers that the header itself imports. So if there’s some secondary class that works in conjunction with the main class, and whose header is also incorporated into the header, I have to go annotate that class for nullability, too. Here is an opportunity to take stock of whether you really need to import such a class. For example, it might be that a forward declaration will do, and the implementation (.m) file can import the header for internal use only.

Failing to recognize the importance of nullability annotation was probably my biggest mistake yet in my young Swift career. If you’re just getting started tackling Swift from a legacy Objective C source base, here’s hoping you won’t make the same mistake!

Brent’s Swift Tension

Brent Simmons has been writing new code exclusively in Swift for a while now, and he recognizes omissions from the language that he still anticipates will be hard to overcome as Apple presumably moves toward incorporating Swift into its developer-facing frameworks. The Tension of Swift:

The Objective-C runtime allows AppKit and UIKit to provide some powerful features that I’d hate to do without: the Responder Chain, xib and storyboard loading, KVC, and the undo manager.

A key point he gets at is that however great Swift is, it’s only possible to use it to develop functional iOS and Mac apps because of the Objective C runtime still operating behind the scenes. Both AppKit and UIKit not only lean on the functionality of the runtime, but are designed with the runtime in mind.

Some major design priorities of the Swift language, namely type safety and compile time dependency binding, are at odds with the design priorities of 20 years of evolution in Apple’s frameworks. How and if that disparity will be reckoned by Apple remains to be seen.

I’m optimistic, because the Swift team has already made many concessions to make the language more compatible with the Objective C runtime. It strikes me as possibly non-optimal that a language that strikes the right compromise between Swift’s priorities and Objective C’s would start at the opposite extreme and work its way backwards, but that is what Apple seems to be doing.

Let’s hope they continue in that direction, and surprise us all with how well it all works out in the end.

Scripted Swift Warnings

Since I complained yesterday about Swift’s lack of a counterpart to Objective C’s #warning directive, folks have been in touch to encourage another solution to this problem. I could take advantage of Xcode’s standardized, built-in support for text-based comments such as “// FIXME” and “// TODO”. Using this notation causes the affected source lines to show up in Xcode’s per-file function popup, and other folks affected by the issue I describe above have added scripted build phases to emit text that Xcode will treat as a warning for each of these lines.

I initially dismissed the solution out of hand, because I anticipated that searching all of my sources with every build would be excessive. I’m kind of a stickler for fast build times because I like to do quick iterations when writing and testing code. My largest source base, MarsEdit, is big but not huge. But, appreciating that computers and SSDs are fast, I decided to test the straightforward script linked above to see how much time it would add to each of my build iterations. On average, it’s about 1.2s per build. It doesn’t sound like much, but personally that would nag at me. I’ve made a fuss over build phase speeds in the past, and reported bugs against unnecessary slowdowns in Apple’s standard build phases.

Zev Eisenberg suggested on Twitter that the use of specialized search tools such as the_silver_searcher might help. It’s true, it might, but then I’d be going down a path of installing custom software, writing custom scripts, all to accommodate what is a very useful feature that I personally still believe should be built in to the language. Or, at the very least, the language’s feature set should accommodate efficiently extending it to support something like #warning.

We’ll see where things go. I may yet end up adopting a convoluted (to my mind) approach involving expensive, redundant build phase script that scans my entire source base with every build, but more likely I’ll keep scratching my head while using the one-line, inline hack that I previously described.

Artificial Swift Warnings

I’ve been tackling more and more of my coding challenges in Swift, recently. I’ve run into a number of hangups. Some attributable to the learning curve, some to bugs in Swift or Xcode, and some to features I’ve grown to love in clang and Objective C which are simply not present in Swift.

For years, I’ve been in the habit of tagging my in-progress code with “artificial warnings.” While working in code, if a concern crosses my mind, the easiest way to make sure I won’t ship the software without addressing that concern is to add it directly to the code:

#warning Step through this in a debugger and confirm it still works...

Or:

#warning This isn't implemented yet, need to handle XYZ...

These warnings serve as an active reminder of things to fix while I’m working in Xcode, since they show up in the build navigator, and are illuminated in the source code while stepping through with the debugger. And because of my strict “no warnings” policy for shipping code, they are guaranteed to show up as errors in any Release build, thus thwarting an accidental shipment of code that is known to need further refinement.

I don’t think Swift supports anything like the “#warning” preprocessor directive.

The closest I’ve come to matching this behavior is a trick that employs Swift’s willingness to emit a warning for unreachable code:

if false { "in lieu of #warning, this will do" }

Unfortunately, it only shows up in the issue navigator as “Will never be executed,” and doesn’t show the specific warning text contained in the string. But at least as soon as click on the warning, I am reminded of the concern at hand.

I don’t know if the Swift team is philosophically opposed to implementing support for #warning, or something like it. I filed an enhancement request with the Swift project. In the meantime, if false { “life goes on.” }.

Update: Many folks have suggested a build-phase script to tag warnings. I wrote more about that option in a followup post.

A Tale Of Two Optimizers

I have quipped a few times that my biggest problems with Swift so far have to do with struggles in the debugger. It seems slow, inaccurate, harder to use than in Objective C. Some of this is just a learning curve, but other aspects seemed fundamentally broken. I whined on Twitter about a scenario in which lldb seemed utterly unaware of one of my variables:

A kind Apple employee, Kate Stone, followed up with me and ultimately encouraged me to file a bug:

I obliged, filing Radar #26032843. Today, Apple got back to me with a followup, suggesting rather gently that I may have neglected to disable optimization in my target. Rookie move! The kind of behavior I was seeing in the debugger is exactly what happens when lldb can’t make as much sense of your code because of inlined functions, loops that have been restructured, etc.

In fact, I had correlated the symptoms with such a problem, but when I went to check on the status of my optimization settings, everything looked fine. Why? Because I was looking, by habit, at the Clang LLVM “Code Generation” settings for optimization:

MarsEdit xcodeproj

See? Optimization disabled. Just as it is for all my projects, and all my targets, because I define it once in my centralized Debug “.xcconfig” file, to be sure I never screw it up:

// We only specify an optimization setting for Debug builds.
// We rely upon Apple's default settings to produce reasonable
// choices for Release builds
GCC_OPTIMIZATION_LEVEL = 0

So why does debugging Swift fail so hard for me? Because Swift doesn’t use that optimization setting. Scrolling down a little farther, I find the culprit in Swift’s own compiler settings section:

MarsEdit xcodeproj

So the lesson is that new Swift developers coming from a legacy of Objective C, C++, or C development need to take stock of Swift compiler settings because they are liable to be rooted in completely different build settings. On the one hand, I’m glad Apple is finally able to get away from a build setting like “GCC_OPTIMIZATION_LEVEL” (though keeping the name in the GCC -> LLVM transition prevented problems like this back then), but on the other hand, it’s kind of annoying to have to express high level directives that affect whether my code will be debuggable or not using multiple build settings.

At least, because I am not an animal, this will also only ever need to be done once, with an edit to the pertinent “.xcconfig” file:

// We only specify an optimization setting for Debug builds, we rely upon
// Apple's default settings to produce reasonable choices for Release builds
GCC_OPTIMIZATION_LEVEL = 0
SWIFT_OPTIMIZATION_LEVEL = -Onone

Now if you’ll excuse me I’m going to take a tour of other Swift-specific compiler settings to make sure I’m not shooting myself in the foot in some other way!