Matt Rajca

Designing Shared Services for the Mac App Sandbox

September 12, 2016

Unlike iOS apps, Mac apps running in the Mac App Sandbox can delegate their work to separate processes with XPC Services. What you may not know is that processes created with NSTask or POSIX API work as well, though with a few limitations. For one, the child process inherits the sandbox of its parent process, which is usually the app itself. This means that child processes are no more powerful than their parent. However, the point of separating your app into subprocesses is not to escape the App Sandbox, but rather it is isolate critical components such that if a crash does occur, it won’t bring down the rest of your app with it. All popular web browsers (including Firefox and Microsoft Edge) do this today with separate web processes.

Robotary, a Swift IDE for robotics programming, is architected around a sea of subprocesses. For one, SourceKit, which provides code completions and drives syntax highlighting, runs in an XPC Service. Hardware communication over Bluetooth, HID, and USB transports all happens in isolated XPC Services, and the debug server runs in a separate process as well. What’s more, the hardware transport processes are shared; both Robotary.app and the debug server (one of its child processes) connect to them. This allows us to do all device connection management in one central place; without this, we’d have multiple processes competing for the same hardware resources!

Visually, this looks something like this:

On macOS 10.11 and 10.10, we were able to make our hardware transport XPC Services shared by setting their ServiceType property to User. This ensures we only ever get one instance of each process per-user, so both the debug server and Robotary.app end up connecting to the same instance.

On macOS Sierra, however, this no longer works. While Robotary.app can still connect to our hardware transport services just fine, the debug server (which is a child process of Robotary.app) no longer can. I’m not sure if this is a bug or a change in security policy, but it means we can no longer execute Robotary programs on macOS Sierra. More visually, the connections in red fail:

You can find a barebones demo app that mimics Robotary’s architecture and demonstrates this issue here. If you run the example as is and look in the console, you’ll notice only the app receives a response from the embedded XPC Service:

SharedXPC[...] Obtained result: 1

The child process does not:

process[...] Obtained result: 0

If you turn off the App Sandbox in both *.entitlements files, both the app and its child process receive a response of value 1 from the embedded XPC Service.

I tried numerous workarounds, including granting a com.apple.security.inherit attribute to the XPC Service and setting the JoinExistingSession Info.plist option to YES, but nothing helped. I also never received a response to rdar://problem/27739880 tracking this issue.

To work around this, I first considered sticking with XPC as a form of IPC but doing the process management myself. I was hoping we could use anonymous listeners for this, which would allow us to use XPC without proper XPC Services. To use anonymous listeners, we first create one with:

NSXPCListener *listener = [NSXPCListener anonymousListener];

We then create an endpoint for the listener:

NSXPCListenerEndpoint *endpoint = listener.endpoint;

Since NSXPCListenerEndpoint conforms to NSSecureCoding, I figured we could just archive it using standard Cocoa archiving API such as NSKeyedArchiver and send it over to the client, which would then initiate a new XPC connection with it. It turns out, however, that only NSXPCCoders can be used to encode endpoints. Since NSXPCCoder is not a public class, this implies we can only use anonymous listeners with existing XPC connections. After doing some research, it seems as if anonymous listeners are mostly useful for Mach Services advertised with launchd, which we cannot use in a Sandboxed Mac app. Since I felt like we were going in circles trying to avoid proper XPC Services, it was time for a new IPC mechanism.

While Mach Ports generally don’t work in the App Sandbox, they can be used in apps and processes that belong to the same App Group. So the next plan was to assign Robotary.app an App Group that would allow all of its subprocesses to communicate via Mach Ports. We could then use traditional NSMachPort and NSMachBootstrapServer APIs to register Mach services in the hardware transport processes and connect to them from Robotary.app and the debug server.

As a first step, I removed the XPC Services from the project and replaced them with standard Command Line Tool targets.

In terms of implementation, we now create a Mach Port for the server end and register it via NSMachBootstrapServer:

@interface Server () <NSMachPortDelegate>
@end

@implementation Server

- (void)run {
	NSMachPort *serverPort = (NSMachPort *)[NSMachPort port];
	serverPort.delegate = self;
	[serverPort scheduleInRunLoop:NSRunLoop.currentRunLoop forMode:NSDefaultRunLoopMode];
	[NSMachBootstrapServer.sharedInstance registerPort:serverPort name:@"RJKYY38TY2.com.MR.server"];
	
	[NSRunLoop.currentRunLoop run];
}

@end

If you wrote client-server software before, this should feel fairly straight-forward. We create a Mach Port for the server and register it with NSMachBootstrapServer. By registering the Mach Port with the bootstrap server, we will be able to look it up by name from clients. Note that the name used must be prefixed with the name of the App Group (RJKYY38TY2.com.MR), which itself is prefixed by my Team ID (RJKYY38TY2). You’ll have to change this value if you try running the example yourself. Finally, we keep the server alive by spinning the run loop, otherwise it would terminate immediately as the APIs we are using are non-blocking.

In fact, the service as is will stick around even after your app quits, so be sure to register for NSPortDidBecomeInvalidNotification notifications and exit gracefully once all clients disconnect. If you’re using Swift, another gotcha is NSMachBootstrapServer is unavailable for policy (but not technical) reasons. You can, of course, easily write a wrapper for it that works in Swift. Here’s Robotary’s implementation verbatim:

// Header

#import <Foundation/Foundation.h>

NS_ASSUME_NONNULL_BEGIN

@interface RBMachBootstrapServer : NSObject

+ (nullable NSMachPort *)portWithName:(NSString *)name;
+ (BOOL)registerPort:(NSMachPort *)port withName:(NSString *)name;

@end

NS_ASSUME_NONNULL_END


// Implementation

#import "RBMachBootstrapServer.h"

@implementation RBMachBootstrapServer

+ (NSMachPort *)portWithName:(NSString *)name {
	NSPort *__nullable port = [[NSMachBootstrapServer sharedInstance] portForName:name];
	if ([port isKindOfClass:[NSMachPort class]]) {
		return (NSMachPort *)port;
	}
	
	return nil;
}

+ (BOOL)registerPort:(NSMachPort *)port withName:(NSString *)name {
	return [[NSMachBootstrapServer sharedInstance] registerPort:port name:name];
}

@end

Now we just need our server to respond to requests from clients. We can do this by implementing the -handlePortMessage: delegate method:

- (void)handlePortMessage:(NSPortMessage *)message {
	NSError *error;
	NSDictionary *request = [NSJSONSerialization JSONObjectWithData:message.components[0] options:0 error:&error];
	
	if (request == nil) {
		NSLog(@"Could not parse the request: %@", error);
		return;
	}
	
	NSLog(@"Request: %@", request);
	
	NSDictionary *response = @{
		@"hi": @1
	};
	NSData *responseData = [NSJSONSerialization dataWithJSONObject:response options:0 error:&error];
	
	if (responseData == nil) {
		NSLog(@"Could not form the response: %@", error);
		return;
	}
	
	NSPortMessage *reply = [[NSPortMessage alloc] initWithSendPort:message.sendPort receivePort:nil components:@[responseData]];
	[reply sendBeforeDate:[NSDate distantFuture]];
}

We simply log all requests to the console and respond with a dictionary containing the key hi: 1. Note that -sendBeforeDate: actually returns a boolean value indicating success, which you might want to check in production code, though I have never seen port message sends fail.

Requests are currently serialized as JSON, but [binary] property lists or any other data serialization mechanism would also work. Binary plists have the advantage of being more space efficient and supporting more types than JSON, and they’re what we ended up going with in the Robotary 1.1 update. For more complex needs, I recommend taking a look at Protocol Buffers, which Apple uses for its iWork file formats and Pixen 4 will use for its new file format.

That’s it for the server side! The more tricky part is getting the Server process set up such that we can successfully connect to it. For one, our new Command Line Tool target must have App Sandbox entitlements. Unfortunately, Xcode doesn’t display its usual Capabilities pane for Command Line Tool targets, so we have to do this with a new Build Phase. In the app target’s Build Phases pane, we have to add a new ‘Run Shell Script’ Build Phase. To sign the Service with entitlements, we can use the codesign tool:

Service="$BUILT_PRODUCTS_DIR/SharedMach.app/Contents/MacOS/Service"
cd $SRCROOT
export CODESIGN_ALLOCATE="$DT_TOOLCHAIN_DIR/usr/bin/codesign_allocate"
/usr/bin/codesign --force --sign \"${CODE_SIGN_IDENTITY}\" --entitlements ./Inherit.entitlements $Service

We’ll also have to create the Inherit.entitlements plist and place it in our project’s root directory. Mine currently looks like:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "https://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>com.apple.security.app-sandbox</key>
	<true/>
	<key>com.apple.security.inherit</key>
	<true/>
</dict>
</plist>

We simply enable the App Sandbox and tell it to inherit the sandbox of our parent process. I never got simply setting the App Group to match that of the parent app to work, but this works nicely.

While we’re in the Build Phases pane, our Service target should also be made a target dependency of the main app, and we should ensure it gets copied to the app’s Executables directory. Be sure to turn code signing off here, as we’re code signing the service manually with the script shown above.

On the client side, we first need to look up the Mach service. If it exists, we can connect to our existing instance, thus making our service shared. If it doesn’t exist, we launch the service with NSTask and then wait for its Mach Port to become available. Once it becomes available, we can start sending data in both directions.

- (NSURL *)_serviceURL {
	return [NSBundle.mainBundle.executableURL.URLByDeletingLastPathComponent URLByAppendingPathComponent:@"Service"];
}

...

NSMachPort *clientPort = (NSMachPort *)[NSMachPort port];
clientPort.delegate = self;
[clientPort scheduleInRunLoop:NSRunLoop.currentRunLoop forMode:NSDefaultRunLoopMode];

// Wait for the server to be available.

NSMachPort *sendPort = (NSMachPort *)[NSMachBootstrapServer.sharedInstance portForName:@"RJKYY38TY2.com.MR.server"];

if (sendPort == nil) {
	NSTask *task = [[NSTask alloc] init];
	task.launchPath = self._serviceURL.path;
	[task launch];
	
	// Wait for the server to be available.
	
	while (sendPort == nil) {
		sendPort = (NSMachPort *)[[NSMachBootstrapServer sharedInstance] portForName:@"RJKYY38TY2.com.MR.server"];
		[NSThread sleepForTimeInterval:0.1];
	}
}

NSError *error;
NSData *data = [NSJSONSerialization dataWithJSONObject:@{
	@"process": @1
} options:0 error:&error];

if (data == nil) {
	NSLog(@"Could not form request: %@", error);
	return;
}

NSPortMessage *message = [[NSPortMessage alloc] initWithSendPort:sendPort receivePort:clientPort components:@[data]];
[message sendBeforeDate:[NSDate distantFuture]];

Note that the call to -[NSThread sleepForTimeInterval:] will block the current thread, so be careful when making connections from the main thread. As before, we can receive responses by implementing -handlePortMessage::

- (void)handlePortMessage:(NSPortMessage *)message {
	NSError *error;
	NSDictionary *response = [NSJSONSerialization JSONObjectWithData:message.components[0] options:0 error:&error];

	if (response == nil) {
		NSLog(@"Could not parse response: %@", error);
		return;
	}

	NSLog(@"Response: %@", response);
}

As with the server, the example client currently doesn’t listen for NSPortDidBecomeInvalidNotification notifications. In practice, you might want to store a reference to your service’s Mach Port and invalidate it upon receiving the notification. Depending on your application, you may choose to try reconnecting to the service or launch a brand new instance of it.

This summarizes how we moved Robotary’s shared services from XPC to Mach Ports and got Robotary programs executing on macOS Sierra again.

The full sample code is available on GitHub. If you have any questions, I’m on Twitter @mattrajca.