Elaborating on that bug from yesterday.

The code which caused me problems which I traced to a rather popular third-party networking library–a networking library that was (in my opinion) rendered obsolete with the introduction of Grand Central Dispatch and NSURLConnection’s sendSynchronousRequest:returningResponse:error: method–which, by the way, is two generations deprecated from today’s NSURLSession class–was a retain cycle that the developer of that library choose to ignore rather than fix in the most obvious way.

The offending code was something like this:

self.callbackBlock = ^{
    [self doSomething];
};

In order to understand why this is a problem we need to look into how Objective C blocks work.

Generally blocks are created on the stack. In order to allow a block (essentially a form of lambda expression for Objective C) to exist beyond the current execution scope, you must make a copy of the block. In creating a copy we create an Objective-C object in memory which must strongly retain any values outside the scope of the block for use inside the scope.

That is, at the end of the assignment to callbackBlock we get in memory something like this:

The retain cycle should be obvious here:

Basically the block holds a pointer to self, and self holds a point to the block. If we then free self, the reference count from the reference holding self is decremented by 1, but the self object and the block object–because they hold references to each other–are never freed:

Memory is leaked–and if self is something complex (like a UIViewController), a lot of memory is leaked.

Oh, but I break the retain cycle in my code by expressly deleting the reference to my block when I’m done with it!

Yeah, because your code is bug free. (Sheesh)


Look, “I deal with it in code so it’s no big deal” is a fucking excuse, not a reason.

And it’s an excuse that is dependent on a misunderstanding of the nature of ARC and the retain process. So if you say “but I break the retain cycle in code”, you are a fucking idiot. Well, not a complete fucking idiot–just smart enough to shoot yourself in the foot.

It’s simple, really. The assumption behind ARC is that all allocations are in a tree-like structure of memory references. Things hold other things in a tree-like hierarchy: your UIViewController holds the root view it manages. The root view holds the other views in the hierarchy. The NSDocument object holds the root of the data structures representing your document. And so forth.

By holding references in a strict tree-like structure, it is clear what objects need to be freed when the retain count for a particular object hits zero: everything below it needs to be released. When a view goes away the children views also need to go away. When a block goes away the objects held by that block can be released.

Now of course sometimes we need back references up the tree. And those back references should be handled with weak references; references which do not increment the reference count, but are intelligent enough to be set to nil if the contents are deallocated. (I imagine a linked list of weak references associated with each object, though I’m not entirely sure how weak references are handled.)

This way we don’t get retain cycles.


So what do you do instead of writing

#pragma clang diagnostic ignored "-Warc-retain-cycles"

above your block where the compiler is complaining about a retain cycle?

Simple. Don’t break the cycle in code; that’s just an invitation to a memory leak, and depending on how you try to break the cycle, the cycle may never be broken. (I’m looking at you, AFNetworking.) Break the cycle using a weak reference.

In the example above, this means writing:

__weak MyObject *this = self;
self.callbackBlock = ^{
    [this doSomething];
};

When we create a weak reference cycle what we wind up constructing in memory is this:

That is, while self holds the block, the block holds a weak reference to self. This breaks the retain cycle by preserving the strict tree-like ordering to the hierarchy.

Then, as we free the reference to self:

And as the retain cycle reaches zero in self, it gets freed:

and then the block goes away itself.


So what happens if we have a separate reference to the block?

Well, again, notice the weak reference to self in block, which gets zeroed out.

So if you ever want to be extra cautious you could write:

__weak MyObject *this = self;
self.callbackBlock = ^{
    if (this) [this doSomething];
};

But remember a method invocation on a nil reference is not illegal in Objective C; it simply invokes a nil method which returns nil. So the

if (this)...

is not strictly necessary.


Today’s tl;dr: always break retain cycles when writing block code.

Never use “-Warc-retain-cycles”–even if you think you know what you’re doing. Because the very fact that you’re contemplating breaking a retain cycle in code clearly demonstrates you do not know what you’re doing.


Addendum: One thing I’ve heard developers say is that sometimes you have to break the rules in exceptional cases.

Sure. I’ll buy that. I’ve even done that.

However, your lack of understanding is not an exceptional case. It’s just stupidity.

Retain cycles, or why your iOS app sucks.

One of the reasons why I hate the over-use of third party libraries is that, while in theory it’s “the more eyeballs the better the code”, in practice none of us who use a third party library ever bother to trace bugs into the third party library. It’s because we think that third party library (especially one that has been around for a long time) is somehow “bug free”–when, as we all know, “too many cooks spoil the broth”.

(That is, we’ve seen in practice the more people who work on a software project in a corporate setting, the more likely there are bugs in the source kit, as different developers with different levels of aptitude work on the same source kit to varying results, often unchecked by his peers as most software projects are just too big for everyone to understand. So why is this different with third party libraries?)

In my case I managed to trace down a bug.

Let’s put it this way:

If, in your source kit, you have something like this:

#pragma clang diagnostic ignored "-Warc-retain-cycles"

You have a really bad memory leak bug which affects Every. Single. Fucking. App. that uses your code.

Now I’d forgive this if we’re talking about a small project and the person working on it wasn’t all that established in the community.

But this is a big honking library which is used by a lot of companies, where there are tutorials floating around on sites like NSHipster and Ray Wenderlich, and is often the very first library everyone wants to include that may engage in any sort of networking operations.


The good news is that 3.0 of AFNetworking did away with the problem.

The bad news is that 2.0 of AFNetworking has the problem in spades.

Things to remember: opening an SSL socket using CFStream

This was a pain in the neck, but I finally figured out the answer.

So I was attempting to open a connection to a server from iOS which may have a self-signed certificate installed.

The specific steps I used to open the connection (from my test harness) was:

1. Set up the variables

	NSString *host = @"127.0.0.1";
	NSInteger port = 12345;
	CFReadStreamRef readStream;
	CFWriteStreamRef writeStream;

2. Set debugging diagnostics. Noted here so I can remember this trick.

	setenv("CFNETWORK_DIAGNOSTICS","3",1);

3. Open the sockets.

	CFStreamCreatePairWithSocketToHost(kCFAllocatorDefault, (__bridge CFStringRef)host, (uint32_t)port, &readStream, &writeStream);

	NSInputStream *inStream = (__bridge_transfer NSInputStream *)readStream;
	NSOutputStream *outStream = (__bridge_transfer NSOutputStream *)writeStream;

4. Set up SSL. The part that tripped me up: you must set the properties for kCFStreamPropertySSLSettings after setting NSStreamSocketSecurityLevelKey. It appears the NSStreamSocketSecurityLevelKey setting overwrites the kCFStreampropertySSLSettings parameter.

	// Note: inStream and outStream are linked by an underlying object, so
	// parameters only need to be set on one of the two streams.

	NSDictionary *d = @{ (NSString *)kCFStreamSSLValidatesCertificateChain: (id)kCFBooleanFalse };
	[inStream setProperty:NSStreamSocketSecurityLevelNegotiatedSSL forKey:NSStreamSocketSecurityLevelKey];
	[inStream setProperty:d forKey:(id)kCFStreamPropertySSLSettings];

5. Open and initalize as needed. Here I’m just opening (because I don’t care if this blocks; this is test code). If using blocking API, use threads. Otherwise use the runloop and delegate APIs.

	[inStream open];
	[outStream open];

On the Java (server) side, the way I set up my server socket for listening to incoming connections was:

1. Set up the variables. Note that my Config class is an internal class that reads properties, and is beyond the scope of this exercise.

	Properties p = Config.get();
	String keystore = p.getProperty("keystorefile");
	String password = p.getProperty("keystorepassword");

	int port = 12345;

2. Load the keystore and key manager. This can be a signed or (in my case) self-signed certificate.

	FileInputStream keyFile = new FileInputStream(keystore); 
	KeyStore keyStore = KeyStore.getInstance(KeyStore.getDefaultType());
	keyStore.load(keyFile, password.toCharArray());

	KeyManagerFactory keyManagerFactory = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
	keyManagerFactory.init(keyStore, password.toCharArray());

	KeyManager keyManagers[] = keyManagerFactory.getKeyManagers();

3. Create an SSLContext. Note that you cannot use “Default” for getInstance below, because that returns an already initialized context, and we want to initialize it with our parameters above. Also note that iOS 9 prefers TLS 1.2.

	SSLContext sslContext = SSLContext.getInstance("TLSv1.2");
	sslContext.init(keyManagers, null, new SecureRandom());

4. Open a ServerSocket class to listen for incoming connections. Note the constant 50 below is arbitrary.

	SSLServerSocketFactory socketFactory = sslContext.getServerSocketFactory();

	socket = socketFactory.createServerSocket(port, 50);

Note that the way I loaded the keystore is sort of the “hard way” to do this; my eventual goal is to have the Java startup code generate a self-signed certificate internally if a keystore is not provided, but I haven’t figured out how to do that yet. (There are plenty of pages out there that show how, but most of them rely on internal Java APIs, and I’m sort of allergic to using undocumented stuff.)

Creating a custom Key in Objective C

When creating a custom key in Objective C for NSDictionary or NSCache or the like, you need to create an object which does the following:

Implements <NSCopying> protocol.

If your key is invariant, you can implement the method copyWithZone: as follows:

- (id)copyWithZone:(NSZone *)zone
{
    return self;
}

Of course if your key is invariant, ideally you would create the key entirely using a custom init function, and mark all the properties (readonly).

Implement the isEqual: method.

This is part of the NSObject protocol. Note that any class (or nil) could be passed in as the argument to isEqual: method, so you may want to use the method isKindOfClass: to verify that you got what you expected as the parameter.

Implement the hash method.

This is also part of the NSObject protocol, and returns a NSUInteger value which provides a hash function of the data you passed in.

The hash function doesn’t need to be complicated. For example, if your key is three integers, your hash function could be as simple as:

- (NSUInteger)hash
{
    return (self.a << 8) ^ (self.b << 4) ^ self.c;
}

What is important is that two keys passed into your system are unlikely to have a similar value.

Also note that many of the classes that you see used routinely as keys (such as NSString or NSNumber) also follow this protocol. Meaning if your custom key has a string in it, you can use the NSString’s hash function as one of the inputs to your own hashing function:

- (NSUInteger)hash
{
    return (self.intVal << 16) ^ [self.stringVal hash];
}

Objective C subscripting operators

I keep forgetting where this link is, so I’m putting it here: CLang Reference: Objective C, Object Subscripting.

This documents the methods we need to define if we are declaring our own Objective C class and we want to implement array-style subscripting or dictionary-style subscripting.

Extracting from the text:

… Moreover, because the method names are selected by the type of the subscript, an object can be subscripted using both array and dictionary styles.

Meaning if the compiler detects the subscript is an integral type, it uses the array-style subscript method calls, and when the subscript is an Objective C pointer type, it uses a dictionary-style subscript method call.

For array subscripting, use either or both of the methods

- (id<NSObject>)objectAtIndexedSubscript:(NSInteger)index;
- (void)setObject:(id<NSObject>)value atIndexedSubscript:(NSInteger)index;

For dictionary-style subscripting, use either or both of the methods

- (id<NSObject>)objectForKeyedSubscript:(id<NSObject>)key;
- (void)setObject:(id<NSObject>)value forKeyedSubscript:(id<NSObject>)key;

This goes hand-in-hand with my earlier post Objective C declaration shortcuts, and has been a public service announcement.

First pass at a more formal language for JSON.

So the single most common thing I run into, which is a source of all sorts of headaches when writing custom software for clients, is hooking into their back end system.

A very common pattern for me is to create a single interface which can perform a HTTP ‘get’ or ‘post’ call in order to obtain the contents, run everything through a JSON parser, and then handing the resulting NSDictionary or NSArray to an object which converts the results into a set of Objective C classes.

Up until now I’ve been using JSON Accelerator, which is a really nice little tool for converting JSON into a set of classes. But this runs into a couple of problems.

(1) A number of sites I integrate with have multiple JSON endpoints, each which return subtly different JSON results. Using JSON Accelerator and I wind up generating a lot of duplicate classes which represent more or less the same thing.

(2) Often those sites will change; after all, the back end is under development as well as the front end. I often have a hard time seeing the structure from the JSON; sometimes buried in a few hundred lines is a field that contains a null pointer or which was changed from a string to a JSON field–and tracking those bugs down can be a pain in the ass.

It seemed to me the best way to handle this is to have an intermediate representational language which allows me to see what it is that I’m working with, and to allow allow me to ‘tweak’ the results, so I can point out that the ‘Person’ record in call A is the exact same thing as the ‘Person’ record in call B, except for one of the fields being omitted.

So I built a simple analysis app and a simple compiler app to resolve this problem.

You can download the compiled tools and read the documentation (such as it is) from here.

The representational language is fairly simple: a set of objects, which can be compiled into Objective C and (when I have time) into Java. Each field in an object can be a primitive, an object or an array of objects. So, for example:

/*  Feed 
 *
 *      Top level of the feed
 */

Feed {
    id: integer,
    name: string,
    date: string,
    active: boolean,
    addressList: arrayof Address,
    phoneList: arrayof Phone,
}

/*  Address
 *
 *      The user's address
 */

Address {
    id: integer,
    name: string,
    address: string,
    address2: (optional) string, // optional in the data stream
    city: string,
    state: string,
    zip: string
}

/*  Phone
 *
 *      The user's phone
 */

Phone {
    id: integer,
    name: string,
    phone: string
}

Note that fields can also be marked as ‘nullable’:

Feed {
    id: integer,
    name: string,
    value: (nullable) real
}

This will translate into an NSNumber * field rather than into a double.

There are also a couple of tools: one that generates the Objective C code, and one which reads in a bunch of JSON (in fact, it will read multiple JSON objects all in a row), and makes a best guess at the underlying structure, collapsing common objects as needed, and even noting when the same field appears to contain ambiguous content.


At some point I will need to clean this up, add Java support, and push this out to GIT. But for now, there you go.

Let me know if this seems useful.