Why I hate Cocoapods.

(1) As a general rule I dislike any build system which requires a list of third party libraries to be maintained separate from the source base you’re building.

This violates the rule that you should be able to rebuild your source kit at any time so long as you have the compiler tools and the contents of your source repository, and get the exact same executable every time–as part of your source kit is stored elsewhere, in off-line databases stored across the Internet.

(2) Because your libraries are stored elsewhere, unless you are very careful with your configuration settings you cannot know if, in the future when you rebuild your project, you won’t get the exact same application. This makes final testing impossible because you cannot know the code you compile when you build for shipping your product is the exact same as the code you compiled when entering the final QA phase.

(3) Cocoapods in particular works by manipulating the settings file and project files of your project in invisible ways. I currently have a project where Cocoapods broke my ability to view inspectable views, and it required some fairly obscure setting hacking.

Unless a tool like Cocoapods is built into Xcode and is made an integral element of the Xcode ecosystem as it ships from Apple (as Gradle is a part of Android Studio), it is inevitable that future releases of Xcode will be broken by Cocoapods, and broken in ways which are nearly impossible for all but the most skilled hacker (or user of Google Search) to resolve.

(4) There is an entire mentality that has evolved around Cocoapods and Maven and other such library management tools that if you don’t have at least a dozen different libraries included in your project, you’re not engaged in software engineering.

I’ve seen projects which didn’t need a single third-party library, and once I handed them off to someone else (or worse, in one project, another developer started working on the code I was working on without management telling me he immediately gutted a bunch of my working code and replaced it with a handful of third party libraries that added nothing to the party.

Now it’s not to say there aren’t a lot of very good third party libraries. In the past I’ve used SLRevealViewController and iCarousel with great success. But in both cases I’ve simply downloaded the class which provides the implementation and included the sources directly into my application.

But I’ve also seen people include libraries which may have been useful during the iOS 4 era but which provide little value over the existing iOS 7/8 API. For example, there are plenty of network libraries which provide threaded networking access–nearly a necessity when the only APIs available to users to do HTTP requests was the event-driven version of NSURLConnect class, which required an intelligent implementation of NSURLConnectionDataDelegate, or rolling your own calls using Berkeley Sockets and POSIX Threads.

However, today we have Grand Central Dispatch and, when combined with the synchronous version of NSURLConnect, can make downloading a request from a remote server as simple as:

	dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
		NSURLRequest *req = [[NSURLRequest requestWithURL:[NSURL URLWithString:@"http://www.google.com"]];
		NSData *data = [NSURLConnection sendSynchronousRequest:req returningResponse:nil error:nil];
		dispatch_async(dispatch_get_main_queue(), ^{
			[self processResult:data];
		});
	});

Do we need CocoaAsyncSocket anymore? FXBlurView given that the current iOS tools do not build before iOS 7? Do we need the other various iOS networking solutions given that it now takes 6 lines of code to perform an asynchronous network connection?

And really, do we need SLRevealViewController? The problem with the UI design that SLRevealViewController allows you to implement makes the problem of “discoverability” hard for users: by hiding a screen in a non-obvious place, it makes it hard for users to know what he can do with his app–which is why Facebook’s mobile app, which first promoted the side bar model that SLRevealUserController implements, has moved away from that design in favor of a standard UITabBarController. (Yes, they’ve kept the same UI element with the list of users on-line hidden on a right-side reveal bar–but frankly that could have also been handled by a simple navigation controller push.)

By the way, Maven is worse, as is the entire Java ecosystem: I’ve seen projects that rely on no less than 50 or so third party libraries–which creates an inherently fragile application in that all of those libraries must be compatible with the other libraries in the collection. And all it takes are two of those 50 or so which require completely different and incompatible versions of a third party library–which I’ve seen, when two libraries suddenly required completely different versions of the same JSON parser library.

(Ironically, the incompatible third party library which triggered this issue was a replacement of Java’s built in logging facility. So we had a completely broken and somewhat unstable build simply because some developer working on the project wanted a few more bells and whistles with an internal logging tool that was not actually used in production.)


Most programmers out there that I’ve encountered today are “plumbers”; rather than build an application with the features requested they immediately go searching for libraries that implement the features they want and glue them together.

While there is something to be said about that when it comes to doing something that is either tricky (such as what SLRevealViewController does), or which requires integration to a back-end system that you don’t control (such as Google Analytics), it has been my experience that programmers who reduce the problem of programming to finding and snapping together pre-fabricated blocks they barely understand using tools they hardly understand and the most cursory understanding of how components work does not result in the best quality applications.

Things to remember: Apple Store and TestFlight Edition.

  1. The ID you sign in on developer.apple.com/ios and the ID you sign in using itunesconnect.apple.com must be the same ID. If they are different, you cannot submit through Xcode.
  2. You cannot get around this by doing an ad-hoc export of your .IPA file and using Apple’s Application Loader to get around this. For some reason your .IPA will not be signed with the correct endorsements to enable TestFlight.
  3. Until you get all the stars to correctly align:
    • Use the same Apple ID for your developer.apple.com and itunesconnect.apple.com accounts,
    • Set up a new application in itunesconnect.apple.com for your application you wish to test,
    • Correctly build your .IPA with a distribution certificate and a provisioning profile which is set up correctly,
    • Correctly submit your .IPA through Xcode (rather than exporting the .IPA and submitting with Apple’s Application loader),
    • Correctly update your build numbers as you submit incremental builds,

    then the Apple TestFlight UI makes absolutely no fucking sense.

    There is no hint how to get through the happy path, or why things go wrong. You just see weird errors that make no sense, and instead of seeing a line in your pre-release software list saying why something cannot be tested, you instead just see a blank line and errors which make little sense and give zero hint as to what you did wrong.

Apple sure can suck the wind out of the fun of writing iOS software.

Update: Even if you do manage to finally work the ‘happy path’ correctly, (a) Apple still does a review on the application before they allow it to go out to “external testing” (really?!? isn’t the point of ad-hoc distribution for your own people to test features for incomplete applications???), and (b) it’s not entirely clear why some of your members do not have the ability to become “internal testers.”

Combine this with the fact that each Apple device generally is held by a single owner with a single Apple ID, and that makes even accepting invites (assuming invites are sent out) for a consultant who may have multiple Apple IDs a bit problematic. Especially if I want to test for a client on my own private iPhone or iPad.

Why design patterns from web development makes crappy iOS or Android apps.

Increasingly I’ve been encountering software developed for iOS which was written by someone who cut their eye-teeth on writing UI software for the web in Javascript. And I’ve noticed a common pattern which arises from that code which makes their code overly complex and hard to maintain on iOS.

Basically, they treat UIView-based objects as passive objects.

And if you started live writing software on the web using Javascript, this makes sense. After all, the view structure represented in the DOM hierarchy is by and large a passive participant in the user interface. When you want to present a list of items in a scrolling region, for example, you would create the scrolling region, then populate the items in that region, setting the attributes of each of the <div> sections and <li> sections with the appropriate CSS styles for overflow and font information.

(In something like GWT it’s not hard to create your own custom view objects, just as you can create custom view objects in Javascript. But at the bottom of the stack, the Menu class or the Table class you built simply declares a <div> and manipulates the subset of the DOM within that <div> tag. The view hierarchy is still a passive participant; you simply wrap part of that passive content into something with some logic.)


The problem is when you move all this onto iOS or onto Android or onto Windows or onto MacOS X, you lose one of the most powerful elements of those platforms–and that is Views are active objects rather than passive participants.

Meaning that a child class of a UITableViewCell in iOS can actively know how to present a data record passed to it; you do not need to put the code to populate the table view cell within the UITableViewController class. You can build an Image View which knows how make a network call and load and cache an image without putting image loading and caching code into the UIViewController. You can create a single view which has complex behavior–without having to put the behavior code inside the view controller.

And this allows you to create very complex user interface elements and reuse them throughout your code.

Of course this also needs to be balanced with the available tools. For example, it makes no sense to create a custom UIButton which presents a button with a different font, when you can set the font and appearance of a default button within the NIB.

But it does indicate that in many cases the proper thing to do is to push functionality down into the view, rather than make the view controller responsible for that logic.


You can’t do this in Javascript. You cannot create a new <myTableTag> which takes special arguments and presents the contents in a unique way, which can be reused throughout your site.

And there is nothing inherently “wrong” with putting all the logic for your button’s behavior, the way table view cells are laid out, and the responsibility for loading images in the background into your view controller.

It just makes life far more complex than it has to be, because you’re leaving one of the most important tools in your arsenal on the floor.

Things to remember: resize icon script for iOS

On iOS there seems to be about a billion different icon sizes, depending on if you’re on iOS 7 or iOS 6, on iPhone or iPad, or for search icons or whatever.

This is a script which I created to help automatically resize all the icons that are used.

if [ $# -gt 0 ]; then
    sips $1 -Z 29 --out icon-29.png
    sips $1 -Z 58 --out icon-29@2x.png
    sips $1 -Z 40 --out icon-40.png
    sips $1 -Z 80 --out icon-40@2x.png
    sips $1 -Z 50 --out icon-50.png
    sips $1 -Z 100 --out icon-50@2x.png
    sips $1 -Z 57 --out icon-57.png
    sips $1 -Z 114 --out icon-57@2x.png
    sips $1 -Z 60 --out icon-60.png
    sips $1 -Z 120 --out icon-60@2x.png
    sips $1 -Z 72 --out icon-72.png
    sips $1 -Z 144 --out icon-72@2x.png
    sips $1 -Z 76 --out icon-76.png
    sips $1 -Z 152 --out icon-76@2x.png
    echo "Done."
else
    echo "You must provide the name of an image file to process."
fi

To use, in the terminal execute this script with the name of the .png file to be resized, and this will generate all the different icon sizes used.

How to suck down iOS memory without even trying.

If you build custom UIView or UIControl objects, it’s important to remember that all of your views are backed by a CALayer object, which is essentially a bitmap image of the rendered view object. (This is how iOS is able to very quickly composite and animate views and images and the secret behind iOS’s UI responsiveness.)

However, this has some consequences on memory usage.

For example, if you create a view which is 2048×2048 pixels in size (say you are scrolling through an image), and you don’t switch the backing layer to a CATiledLayer object, then for that view iOS will allocate 2048x2048x4 bytes per pixel = 16 megabytes for the CALayer backing store. Given that the memory budget for the backing store on a small device (such as the previous generation iPod Touch) is only about 8 megabytes, your application will die an ignoble death pretty quickly.

Now not all views are the same. If you create a view that is 2048×2048 pixels in size, you set the background color to a flat color (like ‘gray’), but then you don’t respond to the drawRect method to draw your view’s contents, then the backing store in CALayer is very small. (Internally I believe iOS is allocating odd-sized texture maps in the graphics processor–and if the view is a flat color the graphics processor allocates a single pixel texture and stretches it to fit the rectangle. Also if you’re setting the background to one of the predefined textures, I believe it’s not allocating any memory at all, but instead setting up a texture map from a predefined texture and setting the (u,v) coordinates to make the texture fit.)

(An empty view, by the way, can be useful for laying out the contents inside that view; just respond to layoutSubviews, grab the bounding rectangle, and do some rectangle math to lay the contents out.)

So if you need to create a large container view with a border around it, you may be better off allocating two UIViews: one with the background color inset a pixel from the other view which is the border color. Otherwise, the moment you invoke drawRect to draw the frame, on an iPad with a retina display your container view–if it fills the screen–will require 2048 x 1536 x 4 bytes per pixels = 12 megabytes, just to represent a bordered container.

(Footnote: I have a feeling CAGradientLayer is doing something behind the scenes to cut down on memory usage; otherwise why have this class at all? And if that’s the case, in the event where you need to populate a bunch of controls over a background with a subtle gradient effect, you may want to use this as your view’s backing store rather than overriding the drawRect method and filling the entire view with a gradient.)

Targeted broadcasting of multithreaded results.

Okay, so here’s a basic problem. You’re building an iOS application (or an Android application) which needs to download an image from a remote site for display in a view.

So you write code similar to the following:

- (void)setImageUrlTest:(NSString *)url
{
	/*
	 *	Request the download on a background thread. Once we've downloaded the
	 *	results, we kick off a new block in the main thread to update the
	 *	image for this image object, and then animate a fade-in
	 */
	
	dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
		NSURLResponse *resp;
		NSURLRequest *req = [NSURLRequest requestWithURL:[NSURL URLWithString:url]];
		NSData *data = [NSURLConnection sendSynchronousRequest:req returningResponse:&resp error:nil];
		if (data) {
			/*
			 *	We have the data from the remote server. Kick off into the main
			 *	thread the UI update
			 */
			
			dispatch_async(dispatch_get_main_queue(), ^{
				/*
				 *	Be a little tricky: fade in from transparent with a half
				 *	second delay
				 */
				
				UIImage *image = [UIImage imageWithData:data];
				self.alpha = 0;
				self.image = image;
				[UIView animateWithDuration:0.5 animations:^{
					self.alpha = 1;
				}];
			});
		} else {
			NSLog(@"An error occurred downloading image");
		}
	});
}

If we were doing this on Android, we could do this using a Java thread. (Ideally we’d want to create a thread queue, but for illustration purposes we will just create a new thread.)

    void setImageUrlTest(final String url)
    {
        final Handler h = new Handler();
        
        /*
         *  Request the download on a background thread. Once we've downloaded
         *  the results, we kick off a runnable in the main thread using the
         *  handler created above, animating a fade-in
         */
        new Thread() {
            @Override
            public void run()
            {
                try {
                    URL u = new URL(url);
                    URLConnection conn = u.openConnection();
                    InputStream is = conn.getInputStream();
                    final Bitmap bmap = BitmapFactory.decodeStream(is);
                    is.close();
                    
                    /*
                     * If we get here we have a bitmap. Put a runnable which
                     * will set the image in the main evnet loop
                     */
                    setVisibility(View.INVISIBLE);
                    setImageBitmap(bmap);
                    h.post(new Runnable() {
                        @Override
                        public void run()
                        {
                            AlphaAnimation a = new AlphaAnimation(0.0f, 1.0f);
                            a.setDuration(500);
                            a.setAnimationListener(new AnimationListener() {
                                @Override
                                public void onAnimationEnd(Animation animation)
                                {
                                }

                                @Override
                                public void onAnimationRepeat(Animation animation)
                                {
                                }

                                @Override
                                public void onAnimationStart(Animation animation)
                                {
                                    setVisibility(View.VISIBLE);
                                }
                            });
                            startAnimation(a);
                        }
                    });
                }
                catch (Throwable th) {
                    Log.d("DownloadImageView","Failed to download image " + url, th);
                }
            }
        }.start();
    }

In both cases what we do is kick off a background task which downloads the image, then using a reference to the original image view, we then load the image into the image view (doing so on the main thread where all UI tasks need to take place), and finally we trigger an animation which fades in the view.

Question: What happens if the image view goes away?

On Android and on iOS, it’s fairly routine to have a slow internet connection, and the user may dismiss your view controller or activity before the view finishes downloading.

But there is a problem with that.

On iOS, the block object that you create has an implicit ‘retain’ on the UIImageView object. This means that, until the network operation completes, all the resources associated with the UIImageView cannot be released.

Things get worse on Android, which (ironically enough) has a much smaller memory footprint for typical applications: not only can’t the ImageView object go away, but the ImageView object contains a reference to the activity which launched the image. Meaning not only is the ImageView object retained by the anonymous thread declaration, but so is the activity that the image is contained in–along with the entire rest of the view hierarchy and all other activity resources associated with the containing activity.

Given that a network timeout can be up to 30 seconds, this means if the user is browsing in and out of different screens, you can very quickly run out of memory as memory is filled up with defunct views whose sole purpose is to exist as a target for a network activity that is no longer really necessary.

What to do?

Okay, the following is not a viable solution, despite my seeing it in multiple places:

	__weak UIImageView *weakSelf = self;
	dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
		NSURLResponse *resp;
		NSURLRequest *req = [NSURLRequest requestWithURL:[NSURL URLWithString:url]];
		NSData *data = [NSURLConnection sendSynchronousRequest:req returningResponse:&resp error:nil];
		if (data) {
			/*
			 *	We have the data from the remote server. Kick off into the main
			 *	thread the UI update
			 */
			
			dispatch_async(dispatch_get_main_queue(), ^{
				/*
				 *	Be a little tricky: fade in from transparent with a half
				 *	second delay
				 */
				
				UIImage *image = [UIImage imageWithData:data];
				weakSelf.alpha = 0;
				weakSelf.image = image;
				[UIView animateWithDuration:0.5 animations:^{
					weakSelf.alpha = 1;
				}];
			});
		} else {
			NSLog(@"An error occurred downloading image");
		}
	});

This doesn’t work because if the image is delayed during download and the image view goes away, the weakSelf reference now points to garbage, causing a crash.

Broadcast to the rescue

One possible solution on iOS is to use NSNotificationCenter to send notifications when a network operation completes successfully. The advantage of using NSNotificationCenter (or an equivalent broadcast/receive mechanism on Android) is that the receiver is effectively detached from the broadcaster: a receiver can detach itself on release, and if the broadcast message has no receivers, the message is dropped on the floor.

And in our case this is the right answer: once the image view has gone away we don’t care what the resulting image was supposed to be.

There is a problem with this, though: the code which handles the incoming message is also separated logically from the code that makes the request. And unless you insert code into the broadcast receiver which explicitly detaches the image view from the NSNotificationCenter once a response is received, you can have dozens (or even hundreds) of broadcast receivers listening to each incoming image.

Another way: CMBroadcastThreadPool

By combining the semantics of a block notification system with a broadcast/receiver pair we can circumvent these problems.

Internally we maintain a map between a request and the block receiving the response. Each block also is associated with a ‘handler’ object; the object which effectively ‘owns’ the response. So, in the case of our image view, the ‘handler’ object is our image view itself.

When our image view goes away, we can notify our thread pool object using the removeHandler: method; this walks through the table of requests and response blocks, deleting those response blocks associated with the handle being deleted:

- (void)dealloc
{
	[[CMNetworkRequest shared] removeHandler:self];
}

Note: CMNetworkRequest inherits CMBroadcastThreadPool to implement network semantics. More information later.

We can then submit a request using the request:handler:response: method; this takes in a request (in our case, the NSURLRequest for an image), the handler (that is, the object which will be receiving the response) and the block to invoke when the response is received.

- (void)setImageUrl:(NSString *)url
{
	CMNetworkRequest *req = [CMNetworkRequest shared];
	[req removeHandler:self];	/* Remove old handler */
	NSURLRequest *rurl = [NSURLRequest requestWithURL:[NSURL URLWithString:url]];
	[req request:rurl handler:self response:^(NSData *data) {
		UIImage *image = [UIImage imageWithData:data];
		self.alpha = 0;
		self.image = image;
		[UIView animateWithDuration:0.5 animations:^{
			self.alpha = 1;
		}];
	}];
}

The call to request:handler:response: stores the handler and response in association with the request, then executes the request in a background thread. Once the response is received, the CMBroadcastThreadPool object looks up the handler and block to invoke, and if present, invokes the block.

However, if the UIImageView has gone away, there are no blocks to invoke–and the network response is dropped on the floor.


Internally our CMBroadcastThreadPool class on iOS invokes an internal method responseForRequest: to process the request. This method is invoked on a block passed into a background thread via Grand Central Dispatch.

The class itself is presented in full here:

CMBroadcastThreadPool.h

//
//  CMBroadcastThreadPool.h
//  TestThreadPool
//
//  Created by William Woody on 7/20/13.
//  Copyright (c) 2013 William Woody. All rights reserved.
//

#import <Foundation/Foundation.h>

@interface CMBroadcastThreadPool : NSObject
{
	@private
		NSMutableSet *inProcess;
		NSMutableDictionary *receivers;
}

- (void)request:(id<NSObject, NSCopying>)request handler:(id<NSObject>)h response:(void (^)(id<NSObject>))resp;
- (void)removeHandler:(id<NSObject>)h;

/* Override this method for processing requests */
- (id<NSObject>)responseForRequest:(id<NSObject, NSCopying>)request;

@end

CMBroadcastThreadPool.m

//
//  CMBroadcastThreadPool.m
//  TestThreadPool
//
//  Created by William Woody on 7/20/13.
//  Copyright (c) 2013 William Woody. All rights reserved.
//

#import "CMBroadcastThreadPool.h"

/************************************************************************/
/*																		*/
/*	Internal Storage													*/
/*																		*/
/************************************************************************/

@interface CMBroadcastStore : NSObject
@property (retain) id<NSObject> handler;
@property (copy) void (^response)(id<NSObject>);
@end

@implementation CMBroadcastStore

#if !__has_feature(objc_arc)
- (void)dealloc
{
	[handler release];
	[response release];
	[super dealloc];
}
#endif

@end

/************************************************************************/
/*																		*/
/*	Thread pool															*/
/*																		*/
/************************************************************************/

@implementation CMBroadcastThreadPool

- (id)init
{
	if (nil != (self = [super init])) {
		inProcess = [[NSMutableSet alloc] initWithCapacity:10];
		receivers = [[NSMutableDictionary alloc] initWithCapacity:10];
	}
	return self;
}

#if !__has_feature(objc_arc)
- (void)dealloc
{
	[inProcess release];
	[receivers release];
	[super dealloc];
}
#endif

/*	request:handler:response:
 *
 *		Submit a request that will be sent to the specified hanlder, via the
 *	response block
 */

- (void)request:(id<NSObject, NSCopying>)request handler:(id<NSObject>)h response:(void (^)(id<NSObject>))resp
{
	@synchronized(self) {
		/*
		 *	Add this to the map of handlers for this request
		 */
		
		NSMutableArray *recarray = [receivers objectForKey:request];
		if (!recarray) {
			recarray = [[NSMutableArray alloc] initWithCapacity:10];
			[receivers setObject:recarray forKey:request];
#if !__has_feature(objc_arc)
			[recarray release];
#endif
		}
		
		CMBroadcastStore *store = [[CMBroadcastStore alloc] init];
		store.handler = h;
		store.response = resp;
		[recarray addObject:store];
#if !__has_feature(objc_arc)
		[store release];
#endif

		/*
		 *	Now enqueue a request in GCD. This only enqueues the item if the
		 *	item is not presently in the queue.
		 */
		 
		if (![inProcess containsObject:request]) {
			[inProcess addObject:request];
			
			dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
				
				/* Process for response */
				id<NSObject> response = [self responseForRequest:request];
				
				/* Get the list of receivers; if any exist, trigger response */
				@synchronized(self) {
					NSMutableArray *a = [receivers objectForKey:request];
					if (a) {
						dispatch_async(dispatch_get_main_queue(), ^{
							for (CMBroadcastStore *s in a) {
								s.response(response);
							}
						});
						[receivers removeObjectForKey:request];
					} else {
						NSLog(@"No receivers");
					}
					
					/* Remove from list of items in queue */
					[inProcess removeObject:request];
				}
			});
		}
		
	}
}

/*	removeHandler:
 *
 *		Remove the handler, removing all the receivers for this incoming
 *	message
 */

- (void)removeHandler:(id<NSObject>)h
{
	@synchronized(self) {
		NSArray *keys = [receivers allKeys];
		for (id request in keys) {
			/*
			 *	Remove selected handlers associated with this receiver
			 */
			
			NSMutableArray *a = [receivers objectForKey:request];
			int i,len = [a count];
			for (i = len-1; i >= 0; --i) {
				CMBroadcastStore *store = [a objectAtIndex:i];
				if (store.handler == h) {
					[a removeObjectAtIndex:i];
				}
			}
			if ([a count] <= 0) {
				/*
				 *	If empty, remove the array of responses.
				 */
				
				[receivers removeObjectForKey:request];
			}
		}
	}
}

/*	responseForRequest:
 *
 *		This returns a response for the specified request. This should be
 *	overridden if possible
 */

- (id<NSObject>)responseForRequest:(id<NSObject, NSCopying>)request
{
	return nil;
}

@end

In Java

For Java I’ve done the same sort of thing, except I’ve also included a thread pool mechanism which manages a finite number of background threads. I’ve also added code which causes a request to be dropped entirely if it is not being processed. For example, if you’re attempting to download an image, but the image view goes away, and the request to download the image hasn’t started being processed by a background thread, then we drop the request entirely.

To use in Android you need to override runOnMainThread to put the runnable into the main thread. (This can be done using Android’s ‘Handler’ class.) You also need to provide the processRequest method.

BroadcastThreadPool.java

/*  BroadcastThreadPool.java
 *
 *  Created on Jul 20, 2013 by William Edward Woody
 */

package com.glenviewsoftware.bthreadpool;

import java.util.ArrayList;
import java.util.HashMap;
import java.util.HashSet;
import java.util.Iterator;
import java.util.LinkedList;
import java.util.Map;

/**
 * A thread pool which sends the response to a request to a broadcast list,
 * which can be disposed of at any time.
 */
public abstract class BroadcastThreadPool<Response,Request,Handler>
{
    /// The number of threads that can simultaneously run
    private int fMaxThreads;
    
    /// The number of threads currently operating
    private int fCurThreads;
    
    /// The number of waiting threads
    private int fWaitingThreads;
    
    /// The request queue; a queue of requests to be processed
    private LinkedList<Request> fRequestQueue;
    
    /// The list of requests tha are being processed
    private HashSet<Request> fInProcess;
    
    /// The request response mapping; this maps requests to their responses.
    private HashMap<Request,HashMap<Handler,ArrayList<Receiver<Response>>>> fReceivers;
    
    
    private class BackgroundThread implements Runnable
    {

        @Override
        public void run()
        {
            ++fCurThreads;
            ++fWaitingThreads;
            
            for (;;) {
                /*
                 * Get the current request
                 */
                
                Request req;
                
                synchronized(BroadcastThreadPool.this) {
                    while (fRequestQueue.isEmpty()) {
                        try {
                            BroadcastThreadPool.this.wait();
                        }
                        catch (InterruptedException e) {
                        }
                    }
                    
                    /* When we get a request, note it's in process and mark thread in use */
                    req = fRequestQueue.removeFirst();
                    fInProcess.add(req);
                    --fWaitingThreads;
                }
                
                /*
                 * Process the response. If an exception happens, set the response
                 * to null. (Is this right?)
                 */
                
                Response resp;
                try {
                    resp = processRequest(req);
                }
                catch (Throwable th) {
                    /* Unexpected failure. */
                    resp = null;
                }
                
                /*
                 * Now send the response
                 */
                
                synchronized(BroadcastThreadPool.this) {
                    /* Remove request from list of in-process requests */
                    fInProcess.remove(req);
                    
                    /* Get map of receivers for this request and remove from list */
                    final HashMap<Handler,ArrayList<Receiver<Response>>> rmap = fReceivers.get(req);
                    fReceivers.remove(req);
                    
                    /* Send response to all receivers */
                    final Response respFinal = resp;
                    runOnMainThread(new Runnable() {
                        @Override
                        public void run()
                        {
                            for (ArrayList<Receiver<Response>> l: rmap.values()) {
                                for (Receiver<Response> rec: l) {
                                    rec.processResponse(respFinal);
                                }
                            }
                        }
                    });
                    
                    ++fWaitingThreads;
                }
            }
        }
        
    }
    
    /**
     *	Broadcast receiver. This is the interface to the abstract class which
     *  will receive the response once the message has been processed.
     */
    public interface Receiver<Response>
    {
        void processResponse(Response r);
    }
    
    /**
     * Create a new thread pool which uses a broadcast/receiver pair to handle
     * requests.
     * @param maxThreads
     */
    public BroadcastThreadPool(int maxThreads)
    {
        fMaxThreads = maxThreads;
        
        fReceivers = new HashMap<Request,HashMap<Handler,ArrayList<Receiver<Response>>>>();
        fRequestQueue = new LinkedList<Request>();
        fInProcess = new HashSet<Request>();
    }
    
    /**
     * This enqueues the request, adds the response ot the list of listeners
     * @param r
     * @param h
     * @param resp
     */
    public synchronized void request(Request r, Handler h, Receiver<Response> resp)
    {
        boolean enqueue = false;
        
        /*
         *  Step 1: Add this to the hash map of handlers. If we need to
         *  create a new entry, we probably have to enqueue the request
         */
        
        HashMap<Handler,ArrayList<Receiver<Response>>> rm = fReceivers.get(h);
        if (rm == null) {
            /* Enqueue if we're not currently processing the request */
            enqueue = !fInProcess.contains(r);
            
            /* Add this receiver */
            rm = new HashMap<Handler,ArrayList<Receiver<Response>>>();
            fReceivers.put(r,rm);
        }
        
        /*
         * Step 2: Add this receiver to the list of receivers associated with
         * the specified handler
         */
        ArrayList<Receiver<Response>> list = rm.get(h);
        if (list == null) {
            list = new ArrayList<Receiver<Response>>();
            rm.put(h, list);
        }
        list.add(resp);
        
        /*
         * Step 3: If we need to enqueue the request, then enqueue it and wake
         * up a thread to process. Note we only enqueue a request if the same
         * request hasn't already been enqueued.
         */
        
        if (enqueue) {
            fRequestQueue.addLast(r);
            
            if ((fWaitingThreads <= 0) && (fCurThreads < fMaxThreads)) {
                /*
                 * Create new processing thread
                 */
                
                Thread th = new Thread(new BackgroundThread(),"BThread Proc");
                th.setDaemon(true);
                th.start();
            }
            
            notify();
        }
    }
    
    /**
     * This is called when my handler is being disposed or going inactive and
     * is no longer interested in the response. The handler will be removed
     * and all of the responses will go away
     * @param h
     */
    public synchronized void remove(Handler h)
    {
        Iterator<Map.Entry<Request,HashMap<Handler,ArrayList<Receiver<Response>>>>> iter;
        
        /*
         * Iterate through all requests, removing handlers. If the request
         * goes empty, remove
         */
        
        iter = fReceivers.entrySet().iterator();
        while (iter.hasNext()) {
            Map.Entry<Request,HashMap<Handler,ArrayList<Receiver<Response>>>> e = iter.next();
            
            HashMap<Handler,ArrayList<Receiver<Response>>> pm = e.getValue();
            pm.remove(h);
            if (pm.isEmpty()) {
                /*
                 * The request is no longer needed. If it is not in process,
                 * remove from the queue, as it doesn't need to be processed.
                 */
                
                Request req = e.getKey();
                if (!fInProcess.contains(req)) {
                    fRequestQueue.remove(req);
                }
            }
        }
    }
    
    /**
     * runOnMainThread: override on an OS which requires responses on the
     * main thread. For now, this just executes directly
     * @param r
     */
    protected void runOnMainThread(Runnable r)
    {
        r.run();
    }

    /**
     * This is the abstract method executed in the thread queue which responds
     * with a specified response.
     * @param req
     * @return
     */
    protected abstract Response processRequest(Request req);
}

Here is an example of using this class to receive data from a remote network connection. We would pass in a string and receive a byte array of the data from the remote server.

NetworkThreadPool.java

/*  NetworkThreadPool.java
 *
 *  Created on Jul 22, 2013 by William Edward Woody
 */

package com.glenviewsoftware.bthreadpool;

import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.net.URL;
import java.net.URLConnection;

public class NetworkThreadPool extends BroadcastThreadPool<byte[], String, Object>
{
    public NetworkThreadPool()
    {
        /* Max of 5 background threads */
        super(5);
    }
    
    private byte[] readFromInput(InputStream is) throws IOException
    {
        byte[] buffer = new byte[512];
        ByteArrayOutputStream baos = new ByteArrayOutputStream();
        int rlen;
        
        while (0 < (rlen = is.read(buffer))) {
            baos.write(buffer, 0, rlen);
        }
        
        return baos.toByteArray();
    }

    @Override
    protected byte[] processRequest(String req)
    {
        try {
            URL url = new URL(req);
            URLConnection conn = url.openConnection();
            InputStream is = conn.getInputStream();
            byte[] buffer = readFromInput(is);
            is.close();
            return buffer;
        }
        catch (Throwable th) {
            // Handle exception
            return null;
        }
    }

    public static NetworkThreadPool shared()
    {
        if (gNetworkThreadPool == null) gNetworkThreadPool = new NetworkThreadPool();
        return gNetworkThreadPool;
    }
}

We can then invoke this by writing:

        NetworkThreadPool.shared().request(imageUrl, this, new NetworkThreadPool.Receiver() {
            @Override
            public void processResponse(byte[] r)
            {
                /* Convert to image bitmap and load */
                ...
            }
        });

and of course when the image view goes away, we can (on onDetachedFromWindow) write:

void onDetachedFromWindow()
{
    super.onDetachedFromWindow();
    NetworkThreadPool.shared().remove(this);
}

Disclaimer:

This code is loosely sketched together in order to illustrate the concept of using a thread pool married with a broadcast/receiver model to allow us to handle the case where the receiver no longer exists prior to a request completing in the background.

I believe the code works but it is not throughly tested. There may be better ways to do this.

Feel free to use any of the above code in your own projects.

And if you need a dynamite Android or iOS developer on a contract basis, drop me a line. 🙂

OpenGL ES for iOS

I’m working on an application that needs to use OpenGL ES v2.0 on iOS and Android. And one problem I’m running into is getting a good in-depth discussion of OpenGL ES shaders on iOS: one book I have covers shaders well but sample code was written for Microsoft Windows. Another book was rated highly on Amazon–but the discussion seems to be geared to people who have never worked with OpenGL or with iOS before.

The basic problem I’m running into is getting a basic OpenGL ES app running. I finally have something, so I’m posting it here for future reference, and in case it works well for other people.

This basically marries the OpenGL ES sample code from both books; incorporating shaders from one with the iOS base of the other.

This relies on GLKit; at this point, with iOS 6 on most devices and iOS 5 on most of the rest, there is no reason not to use GLKit. I’m only using GLKView, however; the types of applications I’m working on do not require constant rendering (like a OpenGL game), so I’m not using GLKViewController, which provides a timer loop which constantly renders frames for continuous smooth animation. (To plug in GLKViewController you just change GSViewController’s parent to GLKViewController, and remove the delegate assignment to self.view in viewDidLoad.

Also note I’m releasing resources on viewDidDisappear rather than on viewDidUnload; iOS 6 deprecates viewDidUnload.

GSViewController nib

This is actually very simple: the GSViewController nib contains one view: a GLKView. Not posted here because it’s so simple.

Note if you have other views and you want to move the GLKView to a different location in the hierarchy, modify the GSViewController.m/h class to provide an outlet to the view.

GSViewController.h

//
//  GSViewController.h
//  TestOpenGL
//
//  Created by William Woody on 6/12/13.
//  Copyright (c) 2013 Glenview Software. All rights reserved.
//

#import 
#import 

@interface GSViewController : UIViewController 
{
	EAGLContext *context;
	GLuint vertexBufferID;
	
	GLuint programObject;
}

@end

This implements the basic example out of the book OpenGL ES 2.0 Programming Guide. Note, however, that instead of creating a ‘UserData’ object and storing that in an ‘ESContext’ (which isn’t on iOS AFAIK), instead, I keep the contents of the ‘UserData’ record (the programObject field), along with a reference to the EAGLContext (the ‘ESContext’ of iOS), and a reference to the vertex buffer I’m using.

GSViewController.m

//
//  GSViewController.m
//  TestOpenGL
//
//  Created by William Woody on 6/12/13.
//  Copyright (c) 2013 Glenview Software. All rights reserved.
//

#import "GSViewController.h"

typedef struct {
	GLKVector3 postiionCoords;
} SceneVertex;

static const SceneVertex vertices[] = {
	{ {  0.0f,  0.5f, 0.0f } },
	{ { -0.5f, -0.5f, 0.0f } },
	{ {  0.5f, -0.5f, 0.0f } }
};

@implementation GSViewController

GLuint LoadShader(GLenum type, const char *shaderSrc)
{
	GLuint shader;
	GLint compiled;
	
	shader = glCreateShader(type);
	if (shader == 0) return 0;
	
	glShaderSource(shader, 1, &shaderSrc, NULL);
	glCompileShader(shader);
	
	glGetShaderiv(shader, GL_COMPILE_STATUS, &compiled);
	if (!compiled) {
		GLint infoLen = 0;
		glGetShaderiv(shader, GL_INFO_LOG_LENGTH, &infoLen);
		if (infoLen > 1) {
			char *infoLog = malloc(sizeof(char) * infoLen);
			glGetShaderInfoLog(shader, infoLen, NULL, infoLog);
			NSLog(@"Error compiling shader: %s",infoLog);
			free(infoLog);
		}
		glDeleteShader(shader);
		return 0;
	}
	return shader;
}

- (BOOL)internalInit
{
	const char vShaderStr[] =
		"attribute vec4 vPosition;                                           n"
		"void main()                                                         n"
		"{                                                                   n"
		"    gl_Position = vPosition;                                        n"
		"}                                                                   n";
	const char fShaderStr[] =
		"precision mediump float;                                            n"
		"void main()                                                         n"
		"{                                                                   n"
		"    gl_FragColor = vec4(1.0,0.0,0.0,1.0);                           n"
		"}                                                                   n";
		
	GLuint vertexShader;
	GLuint fragmentShader;
	GLint linked;
	
	vertexShader = LoadShader(GL_VERTEX_SHADER,vShaderStr);
	fragmentShader = LoadShader(GL_FRAGMENT_SHADER, fShaderStr);
	
	programObject = glCreateProgram();
	if (programObject == 0) return NO;
	
	glAttachShader(programObject, vertexShader);
	glAttachShader(programObject, fragmentShader);
	glBindAttribLocation(programObject, 0, "vPosition");
	glLinkProgram(programObject);
	
	glGetProgramiv(programObject, GL_COMPILE_STATUS, &linked);
	if (!linked) {
		GLint infoLen = 0;
		glGetProgramiv(programObject, GL_INFO_LOG_LENGTH, &infoLen);
		if (infoLen > 1) {
			char *infoLog = malloc(sizeof(char) * infoLen);
			glGetProgramInfoLog(programObject, infoLen, NULL, infoLog);
			NSLog(@"Error linking shader: %s",infoLog);
			free(infoLog);
		}
		glDeleteProgram(programObject);
		programObject = 0;
		return NO;
	}
	return YES;
}

- (void)viewDidLoad
{
	[super viewDidLoad];
	
	GLKView *view = (GLKView *)self.view;
	NSAssert([view isKindOfClass:[GLKView class]],@"View controller's view is not a GLKView");
	context = [[EAGLContext alloc] initWithAPI:kEAGLRenderingAPIOpenGLES2];
	view.context = context;
	view.delegate = self;
	[EAGLContext setCurrentContext:context];
	
	glClearColor(0.0f,0.0f,0.0f,1.0f);
	
	[self internalInit];
	
	// Generate, bind and initialize contents of a buffer to be used in GLU memory
	glGenBuffers(1, &vertexBufferID);
	glBindBuffer(GL_ARRAY_BUFFER, vertexBufferID);
	glBufferData(GL_ARRAY_BUFFER, sizeof(vertices), vertices, GL_STATIC_DRAW);
}

- (void)viewDidDisappear:(BOOL)animated
{
	[super viewDidDisappear:animated];
	
	GLKView *view = (GLKView *)self.view;
	[EAGLContext setCurrentContext:view.context];
	
	if (0 != vertexBufferID) {
		glDeleteBuffers(1, &vertexBufferID);
		vertexBufferID = 0;
	}
	
	view.context = nil;
	[EAGLContext setCurrentContext:nil];
	
	glDeleteProgram(programObject);
	programObject = 0;
	[context release];
	context = nil;
}

- (void)glkView:(GLKView *)view drawInRect:(CGRect)rect
{
	glClear(GL_COLOR_BUFFER_BIT);
	
	glUseProgram(programObject);
	
	glEnableVertexAttribArray(GLKVertexAttribPosition);
	glVertexAttribPointer(GLKVertexAttribPosition, 3, GL_FLOAT, GL_FALSE, sizeof(SceneVertex), NULL + offsetof(SceneVertex, postiionCoords));
	glDrawArrays(GL_TRIANGLES, 0, 3);
}

@end

Note a few things. First, I’m setting up a GLKView for rendering; this is all handled in -viewDidLoad. I’m also setting up a vertex buffer in viewDidLoad; the OpenGL ES 2.0 Programming Guide example puts that initialization in Init() instead. The -viewDidLoad method also replaces some of the setup in the example’s main() method.

Also note that -(BOOL)internalInit replaces most of the rest of Init()’s functionality. Specifically we handle compiling the shaders and creating a program there.

I handle cleanup in -viewDidDisappear; keep in mind the example OpenGL ES application doesn’t do any cleanup. We do it here because our application may continue to run even after our view controller disappears, so we need to be a good citizen.

And our draw routine (glkView:drawInRect:) delegate doesn’t set up the viewport, nor does it need to call to swap buffers.


Yes, there are a lot of problems with this code. It’s a quick and dirty application that I’m using to understand shaders in OpenGL ES 2.0.

But I do get a triangle.