Things to remember: files in test bundles.

For a program I’m working on, I want to store a file in the test bundle of an app for unit testing, then load that file to test a parser.

Specifically I want to create a class of XCTestCase, and create a file which is included in the unit test object. I then want to be able to load a URL from the file using NSBundle’s URLForResource:withExtension: method, so that I can test an internal parser using the test file, but without the test file being built as a part of the final application.

To do this, you need to load the URL from the test class’s bundle. So in the end, I wound up doing the following:

(1) Including the file as a target of the unit tests app.

(2) Loading the file with:

- (void)testZip
{
    NSBundle *b = [NSBundle bundleForClass:self.class];
    NSURL *url = [b URLForResource:@"MyFile" withExtension:@"ext"];
    XCTAssert(url != nil);
    ... Do my other tests with the file here...
}

Quiet Insanity and YACC.

One of the things I wanted to do involves having a parser generated from a grammar, similar to YACC.

But I need the code generated in Objective C. And I need a parser that is re-entrant, so it can be run in a separate thread.

Now there are a number of solutions out there. But what I want is an LR(1) or GLR based parser built via a state machine which can be incorporated into Xcode and which generates Objective C code that can be used in an iPhone or iPad.

And let’s be honest, a lot of advise out there is really fucking stupid. Like this:

Code generation is not the “true way” in dynamic languages like Objective-C. Anything that can be achieved by a parser generator can be achieved at runtime. So, I’d suggest you try something like ParseKit, which will take a BNF-like grammar, and give you various delegate hooks you can implement to construct your parser.

That sound you just heard was my eyes rolling.

The reason, by the way, why you may wish to precompile a grammar rather than compile it at runtime is because generally (a) your grammar won’t change, and (b) the more computational time you can use evaluating the grammar, the more compact the grammar can be that you generate.

So without any real good solutions I thought how hard can it be to roll my own?

Well, fairly hard, mostly because the documentation out there for constructing LR(1) grammars sorta sucks.

So I started writing a document that attempts to describe the LR parsing algorithms out there in sufficient detail to roll my own YACC-like parser.


I haven’t quite figured out the GLR grammar parsing techniques in “Efficient Parsing for Natural Language, so it hasn’t been included yet.

But the rest should be there.

And sadly the whole thing grew to 66 pages in length, even without the GLR stuff.

This is a preliminary version, of course. Eventually I plan to upload all this to my GitHub account.

Announcing a new version of SecureChat

I’ve just checked in a new version of SecureChat on the main branch at GitHub.

New features include:

  • A working Android client.
  • Various notification bug fixes.
  • Various iOS bug fixes.

Why am I doing this?

Even after all these months, since the Apple v FBI fight began, I’ve been hearing way too much stupidity about encryption. The core complaint I have is the idea that somehow encrypted messaging is the province of large corporations and large government entities, entities that must somehow cooperate in order to assure our security.

And it’s such a broken way to think about encryption.

This is a demonstration of a client for iOS, a client for Android and a server which allows real-time encrypted chatting between clients. What makes chatting secure is the fact that each device generates its own public/private key, and all communications are encrypted against the device’s public key. The private key never leaves the device, and is encoded in a secure keychain with a weak checksum that would corrupt the private key if someone attempts a brute-force attack against the device’s secure keychain.

Meaning there is no way to decrypt the messages if you have access to the server. Messages are only stored on each device, encrypted using the device’s private key–meaning a data dump of the device won’t get you the decrypted messages. And a brute force attempt to decode the device’s keychain is more likely to corrupt the keychain than it is to reveal the private key.


Security is a matter of architecture, not just salt that is sprinkled on top to enhance the flavor. Which is why there are so many security breaches out there: because most software architects are terrible at their job: they simply do not consider the security implications of what they’re doing. Worse: many of the current “fads” on designing client/server protocols are inherently insecure.

This is an example of what one person can do in his spare time to create a secure end-to-end chat system which cannot be easily compromised. And unlike other end-to-end security systems (where a communications key is generated by the server rather than on the device), it is a protocol that cannot be easily compromised by compromising the code on the server.

Things to remember: broken singletons and XCTests

Ran into a bizarre problem this morning where a singleton (yeah, I know…) was being created twice during the execution of an XCTestCase.

That is, with code similar to:

+ (MyClass *)shared
{
	static MyClass *instance;
	static dispatch_once_t onceToken;
	dispatch_once(&onceToken, ^{
		instance = [[MyClass alloc] init];
	});
	return instance;
}

During testing, if you set a breakpoint at the alloc/init line inside the dispatch_once block, you would see instance being created twice.

Which caused me all sorts of hair pulling this morning.


The solution? Well, the unit test code was including the main application during linking.

And the MyClass class was also explicitly included (through Target Membership, on the right hand side when selecting the MyClass.m file) in our unit tests as well.

What this means is that two instances of the MyClass class is being included. That means two sets of global variables, two sets of ‘onceToken’, two sets of ‘instance’. And two separate calls to initialize two separate instances, causing all sorts of confusion.

The answer?

Remove the MyClass.m class from Target Membership.


Well, I guess the real solution is to design the application without singletons, but that’s an exercise for another day.

Besides, there are times when you really want a singleton: you really want only one instance of a particular class to exist because it represents a common object shared across the entire application–and the semantics creates confusion if multiple objects are created. (This is also the case with NSNotificationCenter, for example.)

Things to remember: opening an SSL socket using CFStream

This was a pain in the neck, but I finally figured out the answer.

So I was attempting to open a connection to a server from iOS which may have a self-signed certificate installed.

The specific steps I used to open the connection (from my test harness) was:

1. Set up the variables

	NSString *host = @"127.0.0.1";
	NSInteger port = 12345;
	CFReadStreamRef readStream;
	CFWriteStreamRef writeStream;

2. Set debugging diagnostics. Noted here so I can remember this trick.

	setenv("CFNETWORK_DIAGNOSTICS","3",1);

3. Open the sockets.

	CFStreamCreatePairWithSocketToHost(kCFAllocatorDefault, (__bridge CFStringRef)host, (uint32_t)port, &readStream, &writeStream);

	NSInputStream *inStream = (__bridge_transfer NSInputStream *)readStream;
	NSOutputStream *outStream = (__bridge_transfer NSOutputStream *)writeStream;

4. Set up SSL. The part that tripped me up: you must set the properties for kCFStreamPropertySSLSettings after setting NSStreamSocketSecurityLevelKey. It appears the NSStreamSocketSecurityLevelKey setting overwrites the kCFStreampropertySSLSettings parameter.

	// Note: inStream and outStream are linked by an underlying object, so
	// parameters only need to be set on one of the two streams.

	NSDictionary *d = @{ (NSString *)kCFStreamSSLValidatesCertificateChain: (id)kCFBooleanFalse };
	[inStream setProperty:NSStreamSocketSecurityLevelNegotiatedSSL forKey:NSStreamSocketSecurityLevelKey];
	[inStream setProperty:d forKey:(id)kCFStreamPropertySSLSettings];

5. Open and initalize as needed. Here I’m just opening (because I don’t care if this blocks; this is test code). If using blocking API, use threads. Otherwise use the runloop and delegate APIs.

	[inStream open];
	[outStream open];

On the Java (server) side, the way I set up my server socket for listening to incoming connections was:

1. Set up the variables. Note that my Config class is an internal class that reads properties, and is beyond the scope of this exercise.

	Properties p = Config.get();
	String keystore = p.getProperty("keystorefile");
	String password = p.getProperty("keystorepassword");

	int port = 12345;

2. Load the keystore and key manager. This can be a signed or (in my case) self-signed certificate.

	FileInputStream keyFile = new FileInputStream(keystore); 
	KeyStore keyStore = KeyStore.getInstance(KeyStore.getDefaultType());
	keyStore.load(keyFile, password.toCharArray());

	KeyManagerFactory keyManagerFactory = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
	keyManagerFactory.init(keyStore, password.toCharArray());

	KeyManager keyManagers[] = keyManagerFactory.getKeyManagers();

3. Create an SSLContext. Note that you cannot use “Default” for getInstance below, because that returns an already initialized context, and we want to initialize it with our parameters above. Also note that iOS 9 prefers TLS 1.2.

	SSLContext sslContext = SSLContext.getInstance("TLSv1.2");
	sslContext.init(keyManagers, null, new SecureRandom());

4. Open a ServerSocket class to listen for incoming connections. Note the constant 50 below is arbitrary.

	SSLServerSocketFactory socketFactory = sslContext.getServerSocketFactory();

	socket = socketFactory.createServerSocket(port, 50);

Note that the way I loaded the keystore is sort of the “hard way” to do this; my eventual goal is to have the Java startup code generate a self-signed certificate internally if a keystore is not provided, but I haven’t figured out how to do that yet. (There are plenty of pages out there that show how, but most of them rely on internal Java APIs, and I’m sort of allergic to using undocumented stuff.)

Thinking about mobile, tablets, desktops and TVs

So I got an Apple TV and the necessary cables in order to sideload software to it. It’s a very interesting product.

But it’s a product which I’m having a hard time wrapping my head around, so here are my thoughts.

Think, for a moment, about how you interact with your mobile device. You may be waiting for a bus or you may be waiting at an airport–so you pull your mobile device out and maybe kill 5 minutes surfing the web or playing a game. (Thus, games that are easy to learn and which have a short play cycle–meaning a game you can play a level in 30 seconds or so–are quite popular. What makes games like Candy Crush or Angry Birds popular is that they combine a short play cycle with something that is easy to learn, and something which provides a lot of pretty animation and sounds–almost like a slot machine.)

A tablet is not something you’re going to pull out of your pocket like your cell phone. It’s something you pull out of your bag. But because its a tablet, you don’t need a desk; you can hold it in your hand. Like a paperback book.

And so a tablet is a great device if you want to spend an hour reading, or watching a movie. It’s also a great device for browsing your e-mails, surfing the web and reading content. It’s also a good device for playing games which have a deeper level of engagement: remember, it’s not something you can whip out in 5 seconds, tinker with it, and stuff back in your pocket. So I would think games that would work well on the iPad would be ones like world building games (like Caesar III).

Now a tablet combined with a keyboard would make a good device for creating some content–and in some ways it occupies the same space as a small laptop computer, which is also equally hard to pull out, and set up. So a tablet with a keyboard is like a laptop computer: you’re not pulling it out of your pocket like a cell phone. You’re not pulling it out of a backpack and holding it like a paperback book. Instead, you’re pulling it out, putting it together (a tablet with a keyboard) or opening it up, and you’re setting it on a desk.

At which point it’s time to start creating content–even if that’s just a blog post or a long response to an e-mail from work.

Desktop computers, of course, sit on your desk; they’re ideal for creating content, and since they are not mobile, they can be far more powerful since there are fewer constraints on power consumption and size. And being the most powerful, they are ideal for high powered games–games which require far more computational power than can run on a laptop computer. (Though today most processor manufacturers are concentrating on energy efficiency over raw performance, so the gap between desktop and laptop computers are not as wide as they used to be.)

Desktop computers are ideal for software developers, for running video and photo editing, and for sophisticated music editing. (I have a MacPro with 64gb RAM as my primary development computer, and it can compile a product like JDate’s mobile app in moments, where my 13 inch Mac Air takes several minutes to do the same task. I also have a 21″ monitor and a 27″ monitor attached to my MacPro–which means I can easily open Xcode on one monitor, have the app I’m debugging on the second, and have documentation open while I’m debugging the code.)


The Apple TV is not something you pull out of your pocket and fiddle with for 5 minutes. It’s not something you pull out of your backpack or purse and open up like a paperback book. It’s not even a laptop or tablet with a keyboard that you pull out of your backpack and set up on a convenient desk. It isn’t even a desktop computer, since the monitor is across the room and being watched by several people rather than sitting a couple of feet from your face on your desk.

And that makes the use case of the Apple TV quite different than the device you pull out and fiddle with for 5 minutes while waiting for a train, or pull out of your backpack and read like a paperback book.


Think of how you use your TV. You may pop some popcorn, or grab something to eat (my wife and I routinely eat dinner in front of the TV), sit down and eat while watching the TV. You may play a video game in front of the TV–but ideally the best video games for a TV can be a social experience.

But sitting down in front of the TV is not as trivial a process for many of us than even sitting down in front of a desktop computer. (A desktop computer you may sit down in front of in order to check your e-mail, but chances are you’re not sitting down for the long haul. So you’re not relaxing as you would on a couch, settling in and leaning back, sometimes with pillows or a blanket. Your desktop chair is probably far more utilitarian than your couch.)

That means you’ve sat down for the long haul, and you’re seeing some degree of entertainment. Even if it is interactive–a video game–you’re not sitting down on a couch for 5 minutes to check your e-mail.


So I would contend that the Apple TV is ideal for the following types of things:

(1) Watching video content. (Duh.) And it’s clear the primary use case Apple has with the Apple TV is to permit individual content providers help Millennials “cut the cord” by allowing content providers build their own content apps.

(2) Playing interactive games with high production value and deep and involving storylines. (Think Battlefront or Fallout 4.)

On this front I’m concerned Apple’s limits on the size of shipping apps may hinder this, since a lot of modern games have memory requirements larger than the current Apple TV app size constraints.

People are complaining that Apple is also hobbling app developers by requiring all games to also work in some limited mode with the Apple remote–but realize that the serious gamer that is your target market will have quickly upgraded to a better input device, so think of using the Apple control as a sort of “demo mode” for your game. Yeah, you can play with the Apple remote, but to really enjoy the game you need a joystick controller.

(3) Browsing highly interactive content that may also work on other form factors. (I could envision, for example, a shopping app that runs on your TV that is strongly tied with video content–such as an online clothing web site with lots of video of models modeling the clothing, or a hardware store web site tied with a lot of home improvement videos. Imagine, for example, a video showing how to install a garbage disposal combined with online ordering for various garbage disposals from the site.)

I think this is a real opportunity for a company with an on-line shopping presence to provide engaging content which helps advertise their products, though it does increase the cost of reaching users in an era where margins are getting increasingly thinner.

(4) Other social content which may involve multiple people watching or flipping through content. Imagine, for example, a Domino’s Pizza Ordering app for your Apple TV, or a version of Tinder that runs on the TV.