I had a real head-scratcher of a bug with MarsEdit, manifesting as a subtle failure of specific AppleScript commands to work properly in the release builds of the app. It seemed as though AppleScript were unable to reckon any properties of my app stemming directly from the application object itself.
I was so perplexed that I found myself digging around inside AppleScript’s own internal handling of the command. Setting a breakpoint on -[NSScriptCommand executeCommand], I could trace the evaluation of the request from AppleScript, watching as it ultimately failed to handle the command and returned an error.
It made sense to discover that, in the course of determining how to handle the command, it would consult the application’s “top level object.” That is, the application object itself. I witnessed that AppleScript internally determines the object by calling something like this:
(lldb) po [[NSScriptExecutionContext sharedScriptExecutionContext] topLevelObject] <NSApplication: 0x618000160600>
Well, that all looks pretty ordinary. Nothing to be alarmed by, right? Luckily, it jumped out at me as all wrong, even though it’s the subtlest of things. The instance returned above is of class NSApplication, which would normally be fine, except that I’ve subclassed NSApplication in my app. It should be of class MEApplication.
Apple doesn’t encourage subclassing NSApplication, but it’s well-documented and appropriate in some circumstances. I happen to implement some of MarsEdit’s high level scripting support in the NSApplication subclass.
I double-checked the target’s Info.plist file. Sure enough the value for NSPrincipalClass is “MEApplication”, so why am I getting an NSApplication instance instead? And why did this suddenly start happening in recent beta test releases of the app?
The answer is that I added an innocuous looking call to -[NSApplication sharedApplication] in MarsEdit’s main.m, before the call to NSApplicationMain. And this code path doesn’t get reached on Debug builds, only on Release ones. Sigh.
It’s in NSApplicationMain that the documented behavior occurs of looking up NSPrincipalClass from the Info.plist, and using that value to call -[WhateverClass sharedApplication]. By calling -[NSApplication sharedApplication] before that, I preemptively force the shared application instance to be of class NSApplication.
So remember, kids: don’t subclass NSApplication. But if you do, then don’t reference -sharedApplication before NSApplicationMain. And if you simply must, then make darned sure you explicitly target your custom subclass by name. For you iOS developers, this wisdom applies to UIApplication, as well!