All posts by Daniel Jalkut

A Tale Of Two Optimizers

Swift, Xcode

I have quipped a few times that my biggest problems with Swift so far have to do with struggles in the debugger. It seems slow, inaccurate, harder to use than in Objective C. Some of this is just a learning curve, but other aspects seemed fundamentally broken. I whined on Twitter about a scenario in which lldb seemed utterly unaware of one of my variables:

A kind Apple employee, Kate Stone, followed up with me and ultimately encouraged me to file a bug:

I obliged, filing Radar #26032843. Today, Apple got back to me with a followup, suggesting rather gently that I may have neglected to disable optimization in my target. Rookie move! The kind of behavior I was seeing in the debugger is exactly what happens when lldb can’t make as much sense of your code because of inlined functions, loops that have been restructured, etc.

In fact, I had correlated the symptoms with such a problem, but when I went to check on the status of my optimization settings, everything looked fine. Why? Because I was looking, by habit, at the Clang LLVM “Code Generation” settings for optimization:

MarsEdit xcodeproj

See? Optimization disabled. Just as it is for all my projects, and all my targets, because I define it once in my centralized Debug “.xcconfig” file, to be sure I never screw it up:

// We only specify an optimization setting for Debug builds.
// We rely upon Apple's default settings to produce reasonable
// choices for Release builds
GCC_OPTIMIZATION_LEVEL = 0

So why does debugging Swift fail so hard for me? Because Swift doesn’t use that optimization setting. Scrolling down a little farther, I find the culprit in Swift’s own compiler settings section:

MarsEdit xcodeproj

So the lesson is that new Swift developers coming from a legacy of Objective C, C++, or C development need to take stock of Swift compiler settings because they are liable to be rooted in completely different build settings. On the one hand, I’m glad Apple is finally able to get away from a build setting like “GCC_OPTIMIZATION_LEVEL” (though keeping the name in the GCC -> LLVM transition prevented problems like this back then), but on the other hand, it’s kind of annoying to have to express high level directives that affect whether my code will be debuggable or not using multiple build settings.

At least, because I am not an animal, this will also only ever need to be done once, with an edit to the pertinent “.xcconfig” file:

// We only specify an optimization setting for Debug builds, we rely upon
// Apple's default settings to produce reasonable choices for Release builds
GCC_OPTIMIZATION_LEVEL = 0
SWIFT_OPTIMIZATION_LEVEL = -Onone

Now if you’ll excuse me I’m going to take a tour of other Swift-specific compiler settings to make sure I’m not shooting myself in the foot in some other way!

Pasteboard Priority

Debugging, Mac, OS X

A weird bug cropped up in MarsEdit, in which a URL copied and pasted from Safari, for example, was pasting into the plain text editor as the text from the link instead of the link itself. Daring Fireball’s star glyph permalinks to entries presented a most dramatic example. Right-clicking a star glyph and copying the link to paste into MarsEdit was supposed to yield:

Image of pasted link shown as expected with full text content of URL

But instead gave:

Image of pasted URL showing the text of the link instead of the URL content

How strange. What could have possibly changed something so fundamental as the manner in which a pasted link is processed by my text editor? I leaned on Mercurial’s “bisect” command which led me to the specific source code commit where the behavior had changed, in my text view’s helper method for building a list of acceptable paste types. My color emphasis is on the changed part:

-	return [[NSArray arrayWithObject:NSFilenamesPboardType] arrayByAddingObjectsFromArray:[NSImage imageFileTypes]];
+	return [[NSArray arrayWithObject:NSFilenamesPboardType] arrayByAddingObjectsFromArray:[NSImage imageTypes]];

That’s it? One little tweak to the construction of a list of image types affects the behavior of pasting a URL copied from Safari? Programming is hard.

I had made the change above because imageFileTypes is deprecated. The deprecation warning specifically says: “use imageTypes instead.” Okay. Functionally, everything related to the image types should work the same. Instead of a list of file extensions from -imageFileTypes, I’m now getting a list of UTIs. I scrutinized the lists a bit to satisfy myself that all of the major image types were present in the new list, and I trust that Apple had covered the bases when they made this migration themselves.

It turns out the change above, the one word diff that causes everything to work either as expected or otherwise, is fine. It’s outside of this method where the real problem lies: in my override of NSTextView’s -readablePasteboardTypes method. In it, I endeavor to combine my own list of pasteable types with NSTextView’s own list. To do this, I create a mutable set to combine -[super readablePasteboardTypes] and my own list, and then return an array of the result. The idea is to avoid listing any items redundantly from the built in list and my own:

- (NSArray*) readablePasteboardTypes
{
	NSMutableSet* allReadableTypes = [NSMutableSet setWithArray:[super readablePasteboardTypes]];
	[allReadableTypes addObjectsFromArray:[self acceptableDragTypes]];
	return [allReadableTypes allObjects];
}

Ah, my spidey sense is finally starting to tingle. While it’s not mentioned in the NSTextView reference documentation, the NSTextView.h header file includes comments about this method that are pertinent here:

Returns an array of types that could be read currently in order of preference.  Subclassers should take care to consider the "preferred" part of the semantics of this method.

Ah, so order matters. Of course. And what does the -[NSSet allObjects] say about order? 

The order of the objects in the array is undefined.

So all this time, I’ve been playing fast and loose with NSSet, lucking out with the coincidence that types I prefer would show up higher in the list than types I don’t prefer. It turns out that “public.url” is among the types included in NSTextView’s own, built-in readablePasteboardTypes method implementation, but previous to this change, it always showed up lower in resulting list than NSStringPboardType. Thus, when faced with an opportunity to paste a URL from Safari, rich with information including the original text from the link, MarsEdit always favored the plain string representation.

Changing from -[NSImage imageFileTypes] to -[NSImage imageTypes] effectively changed the roll of the dice, causing the resulting array from NSSet, documented as being “undefined” in its order, to place the URL type above the string type in the list. Thus it tries to paste as a rich URL with text linking to a URL, but since my plain text HTML editor doesn’t support rich text, all you see is the star.

The fix will be to arrange that my resulting array from readablePasteboardTypes does impose some predictable prioritization. Probably by taking the code that I have now and, after generating the list of all unique types, carefully moving a few types to the top of the list in the order that I’d prefer.

Test With Swift

Cocoa, Swift, Testing

I have recently passed a sort of tipping point where I’m indulging more and more in Swift for new code that I add to my projects. There are some instances where I will still create a new class in Objective C, primarily where I anticipate the need for dynamic runtime hijinx that might be more complicated in Swift. In general though, I’m opting for Swift. Finally.

There are many reasons to remain gun-shy about Swift, and I don’t fault anybody too much for choosing to continue forestalling the transition. I’ve spoken with many people who are as tentative as I was or moreso. Some of our collective reasons for waiting may sound familiar to you:

Swift …

  • … is not mature.
  • … requires adding bloated libraries to the app.
  • … presents an impedance mismatch with existing Cocoa design patterns.
  • … is still too risky for production code.

I don’t agree with all of these rationale, especially now that I’ve decided to dive in myself. However, they make a good basis for the argument I’d like you to consider: you should write all new unit tests in Swift.

For many of us who spent years developing a vast collection of Objective-C based classes, it does seem daunting to transition to a new language. But unit tests are different from “regular code” in a number of ways that make them a suitable place to start delving into Swift:

Unit tests …

  • Don’t ship to customers.
  • Can be as bloated as you like.
  • Test the exposed interfaces of classes more than the internal design.
  • Are not technically production code.

I’m sure somebody will argue that tests are so vital to the development process, that they are the last place one should invest in risky technology. I guess what I’m urging you to believe is that Swift is no longer risky technology. It’s not longer coming, it’s here. We will serve ourselves well to adopt it as quickly as practical. And those of us who are daunted by the challenge incorporating it into our existing, Objective-C heavy source bases, have a perfect opportunity in unit testing to get our feet wet while establishing a Swift source base that will live on well into the future. After all, your unit tests should, in theory, outlive any specific implementations of your shipping code.

Principally Unclassy

iOS, Mac, OS X

I had a real head-scratcher of a bug with MarsEdit, manifesting as a subtle failure of specific AppleScript commands to work properly in the release builds of the app. It seemed as though AppleScript were unable to reckon any properties of my app stemming directly from the application object itself.

I was so perplexed that I found myself digging around inside AppleScript’s own internal handling of the command. Setting a breakpoint on -[NSScriptCommand executeCommand], I could trace the evaluation of the request from AppleScript, watching as it ultimately failed to handle the command and returned an error.

It made sense to discover that, in the course of determining how to handle the command, it would consult the application’s “top level object.” That is, the application object itself. I witnessed that AppleScript internally determines the object by calling something like this:

(lldb) po [[NSScriptExecutionContext sharedScriptExecutionContext] topLevelObject]
<NSApplication: 0x618000160600>

Well, that all looks pretty ordinary. Nothing to be alarmed by, right? Luckily, it jumped out at me as all wrong, even though it’s the subtlest of things. The instance returned above is of class NSApplication, which would normally be fine, except that I’ve subclassed NSApplication in my app. It should be of class MEApplication.

Apple doesn’t encourage subclassing NSApplication, but it’s well-documented and appropriate in some circumstances. I happen to implement some of MarsEdit’s high level scripting support in the NSApplication subclass.

I double-checked the target’s Info.plist file. Sure enough the value for NSPrincipalClass is “MEApplication”, so why am I getting an NSApplication instance instead? And why did this suddenly start happening in recent beta test releases of the app?

The answer is that I added an innocuous looking call to -[NSApplication sharedApplication] in MarsEdit’s main.m, before the call to NSApplicationMain. And this code path doesn’t get reached on Debug builds, only on Release ones. Sigh.

It’s in NSApplicationMain that the documented behavior occurs of looking up NSPrincipalClass from the Info.plist, and using that value to call -[WhateverClass sharedApplication]. By calling -[NSApplication sharedApplication] before that, I preemptively force the shared application instance to be of class NSApplication.

So remember, kids: don’t subclass NSApplication. But if you do, then don’t reference -sharedApplication before NSApplicationMain. And if you simply must, then make darned sure you explicitly target your custom subclass by name. For you iOS developers, this wisdom applies to UIApplication, as well!

Nasty. Yeah.

Careful What You Wish For

Bug Reports, Xcode

I am a well-known proponent of bug filing. In fact, I filed a bug last year that argued Xcode should apply the custom Code Signing Flags for a project when code signing the bundled Swift libraries that are installed in in an app’s bundle.

I noticed that at some point along the way, Apple did fix the bug! Exciting! Time to rejoice, no? Well, no. Because while the fix for the bug addressed my original complaint, it brought a new headache which I hadn’t anticipated.

For my app MarsEdit, I supply a custom designated requirement for the code signing phase. The long and short of this approach is that through careful specification of a designated requirement, you can arrange it so that if the user stores an item in the keychain from one version of your app (say the Mac App Store version), then the same app can access the data later without prompting the user, even if it has a different bundle ID, or is signed by a different identity (say, because it’s a direct sale version).

The problem I’m running into now, as I dabble with adding Swift-based code to MarsEdit, is the bundled Swift libraries are being signed with my app’s custom designated requirement. Oops. This never would have happened if Apple hadn’t … fixed the bug I reported.

I’m not sure what the right solution is, or even how to best approach a workaround. The copying of Swift libraries and their subsequent signing is not only opaque to developers, but it happens after the entire user-configurable build process is done. It seems as though to work around the problem in such a way that I can still supply a designated requirement for my app, I may need to avoid adding the flag to OTHER_CODE_SIGNING_FLAGS, and amend the designated requirement on my own sometime after the Xcode build process is complete.

I find the code signing configuration options to be pretty troubling for this reason, among others. The highest level code signing identity (and now flags) are used implicitly by Xcode for some code signing operations that are opaque to developers. It puts us in a bind when something subtle turns out to be not quite right with the resulting built product.

I filed a new bug, Radar #25579693, to raise the issue with Apple about the can of worms that fixing my previous report has caused. In the mean time, I’m on the hunt for workarounds! I’ll update this post when I feel I’ve got something good.

NSDebugScrolling

Cocoa, Debugging, Hacking, OS X, Xcode

I’m working on some heavy NSTextView, NSScrollView, NSClipView type stuff in MarsEdit. This stuff is fraught with peril because of the intricate contract between the three classes to get everything in a text view, including its margins, scrolling offset, scroll bars, etc., all working and looking just right.

When faced with a problem I can’t solve by reading the documentation or Googling, I often find myself digging in at times, scratching my head, to Apple’s internal AppKit methods, to try to determine what I’m doing wrong. Or, just to learn with some certainty whether a specific method really does what I think the documentation says it does. Yeah, I’m weird like this.

I was cruising through -[NSClipView scrollToPoint:] today and I came across an enticing little test (actually in the internal _immediateScrollToPoint: support method):

0x7fff82d1e246 <+246>:  callq  0x7fff82d20562            ; _NSDebugScrolling

0x7fff82d1e24b <+251>:  testb  %al, %al

0x7fff82d1e24d <+253>:  je     0x7fff82d20130            ; <+8160>

0x7fff82d1e253 <+259>:  movq   -0x468(%rbp), %rdi

0x7fff82d1e25a <+266>:  callq  0x7fff8361635e            ; symbol stub for: NSStringFromSelector

0x7fff82d1e25f <+271>:  movq   %rax, %rcx

0x7fff82d1e262 <+274>:  xorl   %ebx, %ebx

0x7fff82d1e264 <+276>:  leaq   -0x118d54fb(%rip), %rdi   ; @“Exiting %@ scrollHoriz == scrollVert == 0”

0x7fff82d1e26b <+283>:  xorl   %eax, %eax

0x7fff82d1e26d <+285>:  movq   %rcx, %rsi

0x7fff82d1e270 <+288>:  callq  0x7fff83616274            ; symbol stub for: NSLog

 

Hey, _NSDebugScrolling? That sounds like something I could use right about now. It looks like AppKit is prepared to spit out some number of logging messages to benefit debugging this stuff, under some circumstances. So how do I get in on the party? Let’s step into _NSDebugScrolling:

AppKit`_NSDebugScrolling:

0x7fff82d20562 <+0>:   pushq  %rbp

0x7fff82d20563 <+1>:   movq   %rsp, %rbp

0x7fff82d20566 <+4>:   pushq  %r14

0x7fff82d20568 <+6>:   pushq  %rbx

0x7fff82d20569 <+7>:   movq   -0x11677e80(%rip), %rax   ; _NSDebugScrolling.cachedValue

0x7fff82d20570 <+14>:  cmpq   $-0x2, %rax

0x7fff82d20574 <+18>:  jne    0x7fff82d20615            ; <+179>

0x7fff82d2057a <+24>:  movq   -0x116a7ad9(%rip), %rdi   ; (void *)0x00007fff751a9b78: NSUserDefaults

0x7fff82d20581 <+31>:  movq   -0x116d5df8(%rip), %rsi   ; “standardUserDefaults”

0x7fff82d20588 <+38>:  movq   -0x1192263f(%rip), %rbx   ; (void *)0x00007fff882ed4c0: objc_msgSend

0x7fff82d2058f <+45>:  callq  *%rbx

0x7fff82d20591 <+47>:  movq   -0x116d5fa0(%rip), %rsi   ; “objectForKey:”

0x7fff82d20598 <+54>:  leaq   -0x118ab0cf(%rip), %rdx   ; @“NSDebugScrolling”

0x7fff82d2059f <+61>:  movq   %rax, %rdi

0x7fff82d205a2 <+64>:  callq  *%rbx

 

Aha! So all i have to do is set NSDebugScrolling to YES in my app’s preferences, and re-launch to get the benefit of this surely amazing mechanism. Open the Scheme Editor for the active scheme, and add the user defaults key to the arguments passed on launch:

Screenshot 3 29 16 3 50 PM

You can see a few other options in there that I sometimes run with. But unlike those, NSDebugScrolling appears to be undocumented. Googling for it yields only one result, where it’s mentioned offhand in a Macworld user forum as something “you could try.”

I re-launched my app, excited to see the plethora of debugging information that would stream across my console, undoubtedly providing the clues to solve whatever vexing little problem led me to stepping through AppKit assembly code in the first place. The results after running and scrolling the content in my app?

Exiting _immediateScrollToPoint: without attempting scroll copy ([self _isPixelAlignedInWindow]=1)

I was a little underwhelmed. To be fair, that might be interesting, if I had any idea what it meant. Given that I’m on a Retina-based Mac, it might indicate that a scrollToPoint: was attempted that would have amounted to a no-op because it was only scrolling, say, one pixel, on a display where scrolling must move by two pixels or more in order to be visible. I’m hoping it’s nothing to worry about.

But what else can I epect to be notified about by this flag? Judging from the assembly language at the top of this post, the way Apple imposes these messages in their code seems to be based on a compile-time macro that expands to always call that internal _NSDebugScrolling method, and then NSLog if it returns true. Based on the assumption that they use the same or similar macro everywhere these debugging logs are injected, I can resort to binary analysis from the Terminal:

cd /System/Library/Frameworks/AppKit.framework
otool -tvV AppKit | grep -C 20 _NSDebugScrolling

This dumps the disssembly of the AppKit framework binary, greps for _NSDebugScrolling, and asks that 20 lines of context before and after every match be provided. This gives me a pretty concise little summary of all the calls to _NSDebugScrolling in AppKit. It’s pretty darned concise. In all there are only 7 calls to _NSDebugScrolling, and given the context, you can see the types of NSLog strings would be printed in each case. None of it seems particularly suitable to the type of debugging I’m doing at the moment. It’s more like plumbing feedback from within the framework that would probably mainly be interesting from an internal implementor’s point of view. Which probably explain why this debugging key is not publicized, and is only available to folks who go sticking their nose in assembly code where it doesn’t belong.

Constraint Activation

iOS, OS X, Xcode

I got started with Auto Layout a few years ago, and on the whole I’m very happy with the framework. It can be exceedingly frustrating at times, especially when some nuanced constraint priority or other is imposing a layout that just doesn’t make sense. But I measure its value by the degree to which I shudder in imagining going back to the old springs and struts approach.

Although some interfaces work perfectly with a fixed set of constraints, other interfaces require dynamic manipulation at runtime in order to achieve the desired result. For example, if a change in a preferences panel brings in some new element to the UI, it might make sense to adjust constraints at runtime to accommodate it.

Prior to OS X 10.10 and iOS 8.0, this could be achieved in a general case by removing and adding constraints as needed to the view in question:

  1. Remove constraints and save them somewhere, e.g. in an array, for later.
  2. Add or remove elements to the view.
  3. Add constraints, e.g. by fetching them from a saved array.

It is important to remove constraints and save them before removing an affected element, because removing the element will cause the constraint to be implicitly removed before you can save it.

Starting in OS X 10.10 and iOS 8.0, I was intrigued by the announcement that NSLayoutConstraint now supports a property called “active,” which can be used to, you guessed it, activate or deactivate a constraint. I assumed this would be an answer to my prayers: a constraint could now be left installed on a view for safe-keeping, but its impact on layout would be negated by setting it to be “inactive.” I envisioned setting up competing groups of constraints on a view and simply activating or deactiving them en masse when the need arose.

I assumed wrong.

Looking at the documentation more closely, I see the description of what the method actually does:

Activating or deactivating the constraint calls addConstraint: and removeConstraint: on the view that is the closest common ancestor of the items managed by this constraint. Use this property instead of calling addConstraint: or removeConstraint: directly.

The rub is that an NSLayoutConstraint whose “active” is set to false will be removed from view it is installed on. So if you have any hopes or dreams of reapplying that constraint later, you’ll need to save it somewhere, just as before. If you don’t keep a strong reference to the constraint, it may be deallocated. If you tried to go back and set “active” a constraint that you referenced as a weak IBOutlet, for example, it would be nil by the time you tried to do so.

The OS X 10.10 AppKit Release Notes makes a clearer emphasis on the intended utility of the “active” property:

Under Mac OS X 10.10, it is now possible to directly activate and deactivate NSLayoutConstraint objects, without having to worry about adding them to an appropriate ancestor view. This is accomplished by manipulating NSLayoutConstraint’s new boolean property ‘active’. Class methods are available for operating on multiple constraints simultaneously, which can be much faster. The legacy API on NSView for adding & removing constraints is now deprecated.

So “active” is not a convenience for easily toggling whether an installed layout constraint has an effect or not, but a convenience for the plumbing of installing and removing them. The emphasis on adding and removal API being deprecated especially underscores that.

Having written this all out, it suddenly occurs to me that the “active” flag I was dreaming of is actually sort of available, and has been all along. Because NSLayoutConstraint supports a mutable “priority” property, you can effectively disable it by setting its priority lower than any other constraints that affect the same view. One caveat though is you can’t change a constraint’s priority to or from “required” at runtime, so you have to choose a priority lower than 1000. Something like this should work:

NSLayoutPriority newPriority = activate ? 999 : 1;
[dynamicConstraint setPriority:newPriority];

So long as a set of counterpart constraints affecting similar views is always prioritize to the inverse when toggling state, something like this should workâ„¢. Of course, it requires knowing the “active” priority in code. If a given disabled constraint should actually have priority 501 or 250, or whatever, then you’d have to save that priority somewhere. In which case you may as well go back to saving the whole constraint.

Remote Codesign Trust

iOS, Mac, Worfklow, Xcode

It’s common to use a remote, headless build server to perform iOS or Mac builds for which the “codesign” tool must be run to sign either the resulting binary, the installer package, or both.

My friend Mitch Cohen raised a concern on our local CocoaHeads that, since 10.11.1 was released, there is a problem with such setups. The dialog that pops up when using a code signing identity for the first time ignores clicks to the “Allow” and “Always Allow” buttons:

Panel prompting to allow permission to use codesign tool with private key.

If you look in the system Console, you see a line along the lines of this, correlated to each attempted click of an “Allow” button:

3/15/16 8:51:55.303 AM SecurityAgent[65532]: Ignoring user action since the dialog has received events from an untrusted source

The changes are apparently rooted in a legitimate security update made by Apple, but the end result for us developers is pretty bleak. It’s seemingly impossible to authorize use of a code signing identity on a remote server. As my friend Mitch put it, he has to call IT and “get someone to go into the data closet” every time this happens. What a drag!

I’m also affected by this issue, since I use a headless Mac Mini as a build server. Lucky for me the server is in my house, but it’s still a drag to have to go log in and control the computer with an attached keyboard.

I poked around for a solution to this problem, and found it lurking in the answers and comments of a Stack Overflow question. The basic idea is you can convince OS X to trust codesign to use the tool, just as if you had clicked the “Allow” button in that UI prompt. Here is a recipe for doing just that, logged in as a remote user over say Screen Sharing:

  1. Open Keychain Access and locate the pertinent private key. You may have to click on the “Certificates” section and then type in part of the name of the identity in question, e.g. “iPhone Distribution”:

    Screenshot of Keychain Access showing the private key

  2. Click the private key and then select File -> Export Items from the menu bar. Save the item anywhere on your server, for example the Desktop. Give it a password e.g. “hardpassword”.
  3. Now stop. Think real hard before you continue. You should have a backup of that private key somewhere. If you don’t? Don’t continue. Did you just make a backup to the Desktop? Yes, in theory. But if these are the only two copies of this private key you need to think long and hard before you continue with the next step, which I absolutely do not advise you to do.
  4. Delete the private key from your keychain. This is the scary (see above), but necessary step to get the Keychain to respect the new permissions you’re going to grant the key in a minute.
  5. Switch to the Terminal and issue a command like the following: 
    security import ~/Desktop/Certificates.p12 -k ~/Library/Keychains/login.keychain -P hardpassword -t priv -T /usr/bin/codesign -T /Applications/Utilities/Keychain\ Access.app

    It may be important to specify the path to the specific keychain, and to belabor the point you’re importing a private key with the “-t priv” parameter. It seemed like maybe things weren’t working for me until I deigned to include these. In any case, the goal here after running this command is that the private key shown in the screenshot above should now be back in your keychain, but if you double-click it to examine its permissions, you’ll find that codesign is listed among the approved apps. If you need to provide access to other tools than codesign, list them as shown above with multiple -T parameters to the “security import” command.

In my tests that seems to address the issue. After following the recipe above carefully, I can remotely “codesign” with the specified identity to my heart’s content, without a UI prompt of any kind.

It strikes me that this trick probably shouldn’t work, at least in light of Apple’s clamping down on the ability to change authorizations through the UI of Keychain Access. So I won’t be surprised if this series of tricks doesn’t stand the test of time. For now though? No need to call IT and have them go into the data closet.

Update: Erik Schwiebert reports that Apple has addressed this problem in 10.11.4 betas:

That’s great news!

Supercharged Search Scopes

Xcode

Xcode’s search facility is pretty robust, offering the ability to add custom scopes that can restrict with some precision the focus of your search:

Xcode custom search scope editor

However, if you browse the list of options in the popup menu for some of these items, there are frustrating omissions. For example, in the screenshot above you can see that I’m searching files that are within a folder “RSCommon” but whose path is not “YuckySources”. Unfortunately this search is not very useful, because none of my files has a path exactly equal to “YuckySources”. What I really want is an option to specify that the path does not contain “YuckySources”, but the option is not made available to me. Similarly, Xcode offers no choice for “is not within folder.” Generally, there are glaring omissions for negating conditions. (Radar #24510002).

Frustrated by this limitation, I went poking around in Xcode’s configuration files, and was pleased to discover that search scopes are saved in plain text in a file called SearchScopes.xcsclist, for example in Xcode’s configuration folder:

[Home] -> Library -> Xcode -> UserData -> SearchScopes.xcsclist

Update: As of Xcode 9 the search scopes are stored in a file called IDEFindNavigatorScopes.plist.

If you open this up in a text editor, you’ll see that in fact each search scope definition is substantially defined by a standard NSPredicate style expression:

<Scope
  name = "Yummy Sources"
  predicate = "path != &quot;YuckySources&quot; AND location_filesystem == &quot;/Users/daniel/Sources/RSCommon&quot;"
  uuid = "F5110DAA-5DE7-4DD8-A4B1-6D7CB4A2DF1E">
</Scope>

Do I dare to dream that fine-tuning this predicate in a text editor will lead to the desired search results in Xcode? I do! I quit Xcode and edited the predicate in the file so it reads:

(NOT path CONTAINS "YuckySources") AND location_filesystem == "/Volumes/Data/daniel/Sources-Modern/RSCommon"

This expresses the predicate for what I really wanted my search scope to be, but which Xcode wouldn’t allow me to configure. Opening Xcode and selecting the “Yummy Sources” search now allows me to search for results in RSCommon, but NOT in YuckySources. We’re golden!

So I think the limitation here is in Xcode’s predicate editor, which is apparently not robust enough to specify the variety of options afforded by predicate syntax. In fact a caveat here is if I now attempt to edit the hand-tuned predicate above, Xcode crashes with an exception:

comparisonPredicate should be an instance inheriting from NSComparisonPredicate, but it is <NSCompoundPredicate: 0x7fedcff19ff0>

Lucky for us, the code that applies the predicate seems perfectly happy with the hand-tuned, robust predicate. So I’ll be contented to continue using my fancy, supercharged search scopes, but I’ll remember to edit them in BBEdit when necessary.

By the way, the search scopes you create in Xcode are saved by default to the home folder relative path described above, but Xcode will also search for and load any search scopes configured within a workspace’s xcuserdata folder. For example, if you want to configure search scopes that are only meaningful to your “Delicious” project, you could store the configurations in this file:

/Delicious.xcworkspace/xcuserdata/[yourname].xcuserdatad/SearchScopes.xcsclist

I hope this understanding of Xcode’s search scopes helps you transform its already powerful search capability into an even better, finer-tuned resource for navigating your source files.

Mac App Store Sandbox Testing

Apple, Mac, OS X

For months, many of us Mac developers have noticed that apps built for Mac App Store submission could no longer be tested using iTunes Connect “sandbox” users. Previously, a sandbox user account could be used to authenticate and download a _MASReceipt for a Mac app, to ensure that in-app receipt validation is working as expected. After updating to 10.11.2, many of us noticed that this functionality suddenly stopped working:

At first, we assumed it was a bug. But as time went on, it started to seem as though it could be related to Apple’s announcement that one of its key certificates was expiring.

Still, the communication from Apple about this issue was poor enough that it wasn’t obvious what exactly we needed to do. Even though the page linked above has a section explicitly listing what Mac developers are expected to do:

You can verify your receipt validation code is compatible with the renewed certificate in the test and production environments. A pre-release version of the Mac App Store Update for OS X Snow Leopard beta is now available.

The linked “pre-release version” was no-doubt once a valid link, but at least through my account, it now leads to a permission-denied type failure page.

So what do we do? Fortunately, after chatting through the problem with some friends, Chris Liscio deduced the key, somewhat-obvious in retrospect steps to test your Mac app for compliance with the new certificate, while getting sandbox testing working again at the same time:

  1. Install the new certificate from Apple. In my case, I opened it in Keychain access and added it to the System keychain, where the older, expiring certificate currently resides.
  2. Reboot.

The second step is the important one. If you just install the certificate and expect everything to work, you’ll be sadly rebuffed with continued failures. Reboot. Let the system soak in the new certificate, then try re-launching your Mac app built for submission to the Mac App Store. It will prompt you, as you had previously expected it to, for your sandbox credentials. When you enter them, instead of insisting you set up a new iTunes customer profile, it will just launch. Or, if it doesn’t, maybe you’ve got some problems to work though in your receipt validation code.