And then Apple changes the Rules. Again.

So I uploaded J2OC, and had lost interest in it. After all, who needs a second “let’s recompile Java into Objective C” in order to build iPhone and Android applications, if Apple isn’t going to allow it?

Then Apple does this: Statement by Apple on App Store REview Guidelines

In particular, we are relaxing all restrictions on the development tools used to create iOS apps, as long as the resulting apps do not download any code. This should give developers the flexibility they want, while preserving the security we need.

What I would ideally want is a Java VM kernel that can be linked into an iPhone application, one capable of running a jar file. Because ideally I’d like to write model code in Java–so I can port that model code to Android. Yet I don’t want UI bindings into the Apple API–I’d rather just build the UI twice, while the (more complicated) model code remains the same.

Thank you Apple. Maybe I’ll document J2OC better and provide some sample programs. It really is a cool little bit of technology. 🙂

Something Funny Happened To Me On The Way To Release.

So I started playing with parsing Java class files, creating a cross compiler capable of converting Java class files into Objective C files. I even had a sufficient amount of Apache Harmony running so I could use a good part of the java.lang and java.util classes; roughly in parity with the GWT cross compiler that can compile Java class files into Javascript.

Then Apple dropped the “no cross compiling” bombshell.

Now, keep in mind that I’m just me, tinkering on my spare time during weekends. I don’t have the desire or the time to go up against Apple. I’d rather allow the XMLVM project (which has a well established ecosystem, or so it seems) to decide to go (or not go) against Apple’s wishes.

Then time went by, and I sort of lost interest in this thing.

So I’ve taken the liberty to post the source code here: the Java to Objective C Compiler sources, and the J2OC RTL, which contains a subset of the Apache Harmony project, and implementing the java.lang and java.util classes.

It’s been an interesting project, and hopefully in the next few weeks I’ll document how this all works–including the wierdnesses and pitfalls I came across with the Java VM to get Apache Harmony to work. (Nothing like working through a very large collection of class files to find all the fringe cases.) The output code was intended to be human readable–but it really isn’t for some expressions.

But I’ll describe that in the next few weeks.

And at some point I’ll post an example iPhone application which includes Java code.

Note that my approach was different than the XMLVM project. Instead of providing Java bindings of the iOS libraries, my intent was to only allow the compilation of a computational kernel, then have the user provide the UI elements separately for Android, the iPhone, the iPad, and whatever other target the code was to compile for.

So you won’t find a turn-key solution for recompiling Android code and have it run on the iPhone. You should really check out the XMLVM project instead.

All this code, by the way, is being published under a BSD style license: go ahead and use the code, but leave me out of it and don’t blame me if it goes haywire.

separator.png

While I don’t intend to get into the functioning of the compiler, I will give a taste of how the code works. The bulk of the .class file parser, which reads and loads the .class file data into memory, is contained in the class ClassFile in com.chaosinmotion.j2oc.vm. This class takes in its constructor an input stream opened to the first byte of a .class file, and loads the entire class file into memory.

Once read, the entire class file can be accessed using the getters associated with that class. The bulk of the code contained inside the .vm (and subpackages within .vm) are used to represent the contents of the class file. The .vm.data classes contain the various data types used to store the meta data within a class file (such as the method names, the attributes fields, and the like), and the .vm.code classes contain a code parser to convert the code within the .class files into an array of processed instructions.

Once the instructions are parsed (by the vm.code.Code class), the code in a method is represented as an array of code segments; a run of instructions that starts with an instruction first jumped into by another instruction, and terminates with either the end of the method or with a jump instruction. In other words, a CodeSeg (Code.CodeSeg class) is a section of instructions that always enters at the first instruction and executes sequentially to the last instruction in the segment. Additional information, such as the list of variables that are used when the segment is entered are noted; this is the current state of the Java operator stack as this segment is entered.

Ultimately the code parser and class file reader represents the code in a .class file in memory in an intermediate state that can then be used to write Objective C with the WriteOCMethod class (com.chaosinmotion.j2oc.oc). A class, CodeOptimize (.oc package) provides utilities that determine if code preambles must be written for memory management or for exception handling: memory management preamble does not need to be written if I never invoke another method. (This is the case for simple functions which return a field or does simple math.)

The theory is that in practice, it should be possible to replace the code writer method with a writer method capable of writing a different language, such as C++ or C.

separator.png

In the future, when I have more time, I’ll write more about the J2OC project. But for now, if there are any segments or parts you want to use or play with, be my guest.

Four features from Java which are really cool.

There are, in my opinion, four basic language features in Java which make the language extremely cool–in that they allow IDE developers and software developers to quickly and easily deploy tools which make working with Java very easy.

I wanted to point out these features in the hope that future languages (or future extensions to other existing languages such as C++ or Objective C) could incorporate these features, in order to make those language ecosystems better overall.

Now there are a lot of things that are cool about Java. The standard RTL, so there is a standard way to create multiple threads, do file I/O, networking or other similar things is quite cool. (I’m used to the days of early versions of Pascal or C where it was anybody’s guess how to do these things.) But in my estimation the following four features have made the overall Java ecosystem quite friendly compared to other languages.

1. Reflection.

Java’s reflection system is quite complete, allowing full introspection for any given object of the class of that object, the interfaces it implements, the base class it extends, the methods it implements and the fields that are contained in that class. With methods you can introspect the parameters and the return values. You can even dynamically, using the reflection system, construct objects on the fly and invoke methods on the fly.

Java’s reflection system is far more powerful than similar systems in Objective C (which, by design, cannot really allow introspection of the methods implemented by a class, since methods are just messages and there is no strong association of implemented messages), or in C++ (which really only allows you to discover the class of an object, and that’s it.)

Later versions of Java even provide an annotation mechanism by which classes can be annotated or marked with special flags, so during introspection different classes can be handled in different ways.

This level of reflection allows one to construct a complete dynamic serialization library, serializing and deserializing objects without ever needing the object to know what is being done to it.

And because the reflection information is contained (as a first class, well documented data structures) inside a compiled .class file, it is not all that difficult to compile a Java file, then introspect the .class file and provide information about that class. By using this it is possible for an IDE to automatically determine, for example, which methods in an abstract class need to be implemented by a child class, and automatically write the missing method calls for you. I suspect in fact this is how the Eclipse functions to create a new class, populating the methods that need to be implemented, is performed.

I use that all the time.

2. Proxying.

Java also has the ability to, given an interface class declaration, create a proxy class which provides an implementation of that interface, redirecting the method calls to another object.

Proxying is a very powerful mechanism which allows remote procedure calls to be implemented with just a few lines of code. Proxying also allows other very powerful techniques to be implemented, such as using interface methods to define the constant values in a property file, and automatically associate variables in a dynamically loaded configuration file.

While proxying is a bit of a one-trick horse, it provides a tremendous amount of power that, combined with reflection, allows you to do RPC without having to create a RMI language specification and compiler to build the client and server components. You can infer the parameters with reflection and redirect with proxying.

3. Exception Stack Introspection.

Of course most languages have exceptions. What makes Java really cool is the ability to dump the function call stack when an exception is thrown. This is an extremely powerful mechanism because it allows you to know precisely what was thrown and where it was thrown from. And most of the time, knowing what happened and where gives you a huge clue as to why something went wrong.

In fact, I’d say the biggest mistake I’ve ever seen a Java programmer make is to re-throw an exception without setting the cause parameter of the re-thrown exception. For example, I’ve seen the following:

try {
    // A whole bunch of complicated crap with calls
    // into thousands of lines of error-prone methods.
    ...
}
catch (Exception someException) {
    throw new StandardException("All hell broke loose.");
}

Then I see the following:

com.chaosinmotion.mything.StandardException: All hell broke loose.
	at com.chaosinmotion.mything.Test.doSomething(Test.java:19)
	at com.chaosinmotion.mything.Test.main(Test.java:32)

And I want to murder the developer who couldn’t be bothered to add an ‘initCause’ call to the rethrown exception.

4. Inner classes and anonymous classes

To me, these both have the same net effect: they give the developer the ability to define a class inside another class, which has access to the parameters of the outer class, which can then be used as a form of closure. And closures are cool because they allow a callback function or method invocation to have access to the immediate lexical environment where the functioning requiring the closure to be invoked.

For example, suppose we have a method call that takes an interface, where the method in that interface is called for each list of objects in a system:

    private interface Callback
    {
        void object(Collection l);
    }

    private void getAllObjects(Callback callback)
    {
        ....
    }

We can gather up all of the objects in a single list using an anonymous inner class in a function call:

    private List gatherAll()
    {
        final ArrayList n = new ArrayList();
        
        getAllObjects(new Callback() {
            public void object(Collection l)
            {
                n.addAll(l);
            }
        });
        
        return n;
    }

While Java’s implementation of anonymous functions may not meet the strict criteria of closures, in that you do not get full read/write access to all the stack variables within the function where the anonymous class is declared (thanks to the Funarg Problem), read-only access to mutable objects gets us close enough that it doesn’t really make much difference.

Closure-like functionality is such a cool language feature that Apple added it to the latest version of Objective C. Something like the above could be done by declaring a method -getAllObjects: which takes a block–a reference to a function with closure:

	- (void)getAllObjects:(void (^)(NSArray *))callback;

Then calling it with a block that accumulates the results:

    - (NSMutableArray *)gatherAll
    {
        NSMutableArray *n = [[[NSMutableArray alloc] init] autorelease];

        [self gatherAllObjects: ^(NSArray *param) {
            [n addObjectsFromArray:param];
        }];

        return n;
    }

The reason for me picking these four features is simple: I believe if other languages were extended to provide these features, then it would not only make the languages themselves more useful, but if they were provided, they would also allow the ecosystem of IDEs and compilers to become more useful as well. Reflection is extremely important to the ecosystem of IDEs: by being able to parse some sort of data structure during development that gives full inspection of the various components of a class or collection of function calls, it would make automatic code generation far easier to create. Better yet, if compilers for a given language could standardize on an intermediate “story of your compile” like structure that also gives line numbers for compiled objects (as is provided in Java class files with debugging turned on), it would make it extremely easy to automatically generate code on the fly during development.

And every language has an undue amount of bookkeeping stuff that needs to be written which make those languages cumbersome to use. C++, for example, requires all methods to be described in two separate places: once in the header file, and again with the method code. Imagine an IDE which could automatically generate the code declarations based on the header declarations, leaving you to only fill in the blanks. Or an IDE which could automatically add the required methods from the abstract class method declarations in an abstract C++ class.

Time Zones are a pain in the ass.

Some interesting links I found that discuss time zones:

Complete timezone information for all countries. – Just as the label says

The official US time (NIST & USNO) – Displays the current time in the 10 major time zones in the United States.

Time Zone Converter – Allows a search of the various time zones in the world.

What makes time zone conversions a pain in the ass is that each jurisdiction has control over its own timezone. In some places (like California) we’re on Pacific Standard Time/Pacific Daylight Time, period, thankyouverymuch, haveaniceday.

But in places like Indiana, where every blasted county has it’s own ideas as to if it is in Central Standard Time, or Eastern Standard Time, and if the cows there can tolerate the shift between daylight savings time–well, the tz database becomes a hot mess, with things like “America/Indiana/some-random-place” littered throughout. On the other hand, most of that hot mess boils down to “this week, Knox County Indiana has decided to switch time zones from central to eastern”–which, for a UI timezone picker, doesn’t really matter. The fine folks of Knox County can just pick “Central” or “Eastern” and be done with it.

Sometimes the easiest sounding thing, like “pick a time zone”, becomes a royal pain in the neck. And even easier things like “what time is it in GMT at my location” become royally hard. It’s like that, when local politics gets involved.

GWT Weirdness Of The Day.

On the project I’m working on, I noticed that, for whatever reason, it was now taking up to 3 minutes to start a web site running under GWT. That is, from hitting the “Debug” button and opening the web site in the browser, to seeing the web site actually up and running, was taking just shy of three minutes, with most of that time with the Spinning Beach Ball Of Death™.

I finally figured out what was happening. The first clue came from hitting ‘pause’ on the Daemon Thread labeled “Code server for myProject from Safari DMP…”, and noticing a lot of time being spent walking around inside the source kit for resources.

And noticing a lot of that time was spent in a bunch of .svn folders.

I think what was going on is that, for some reason or another, a lot of crud had accumulated in a number of random hidden folders associated with subversion. I don’t know if this is normal or abnormal behavior for Subversion; I just know there was a bunch of crap floating around there–and GWT was spending several minutes walking around the directory structure.

Blowing away the entire source kit and checking it all out fresh from Subversion reduced the startup time to less than 5 seconds. And it greatly improved the performance of the browser in the debugger as well: rollovers which were taking a while to get detected now are fast and responsive.

I suspect all the crud in my source directory was creating a lot of crap for the debugger to deal with, creating a substantial slowdown that got cleared right up.

GWT, ScrollPanel and the iPad.

GWT makes it easy to create a scrolling panel within an HTML driven web site, by using the ScrollPanel class. ScrollPanel itself expands into a div tag containing the overflow attribute; specifying which axes the scroll bars land on translate into overflow-x or overflow-y attributes in the generated HTML DOM object.

And the iPad (and iPhone and iPod Touch) do not allow you to scroll within the overflow tags; the touch-drag gesture only scrolls the entire page.

What this means is if you are putting together a GWT application that you want to work on the iPad, the natural instinct to create a root panel the full size of the screen and tossing in a ScrollPanel area in the lower right that resizes with the window is incorrect. Instead, you are better off generating your GWT code so that it generates the properly sized HTML, and scroll around the GWT application using the window’s scroll bars.

Building a Java RMI system.

Yeah, this is old technology from the stone age. But I wanted to set up a simple Java RMI system.

And it turns out every tutorial I encountered starts with:

        if (System.getSecurityManager() == null) {
            System.setSecurityManager(new SecurityManager());
        }

Then there is a ton of pages showing how we then customize the security manager settings through all sorts of security property files, or extending the SecurityManager class to give us explicit permissions or all of that stuff.

Of course all of this is nice. However, for just testing something it’s unnecessary: the SecurityManager (once set) prohibits access to various things in the JVM unless explicitly opened–and the suggestion to then grant all permissions just seems silly to me: I’m locking the door, then I’m holding it wide open.

Just don’t lock the damned door.

The following seems to work for me; YMMV.

TestInterface.java

import java.rmi.Remote;
import java.rmi.RemoteException;

public interface TestInterface extends Remote
{
    String sayHello() throws RemoteException;
}

TestServer.java

import java.rmi.Naming;
import java.rmi.RemoteException;
import java.rmi.registry.LocateRegistry;
import java.rmi.registry.Registry;
import java.rmi.server.UnicastRemoteObject;

import com.atti.addel.test.springrpc.common.TestInterface;

public class TestServer implements TestInterface
{
    protected TestServer() throws RemoteException
    {
        super();
    }

    private static final long serialVersionUID = 1L;

    public String sayHello() throws RemoteException
    {
        return "Hello World.";
    }
    
    /**
     * Run this as an RMI object
     * @param args
     */
    public static void main(String[] args) 
    {
//        if (System.getSecurityManager() == null) {
//            System.setSecurityManager(new SecurityManager());
//        }
        
        try {
            LocateRegistry.createRegistry(1099);
        }
        catch (RemoteException e) {
            System.out.println("Registry exists.");
        }
        
        try {
            TestInterface engine = new TestServer();
            TestInterface stub = (TestInterface)UnicastRemoteObject.exportObject(engine,0);
            
            Registry registry = LocateRegistry.getRegistry();
            registry.rebind("TestServer",stub);
        }
        catch (Exception ex) {
            System.out.println("Unable to start up.");
            ex.printStackTrace();
        }
    }
}

TestClient.java

import java.rmi.Naming;

import com.atti.addel.test.springrpc.common.TestInterface;

public class TestClient
{
    /**
     * @param args
     */
    public static void main(String[] args)
    {
        try {
            TestInterface test = (TestInterface)Naming.lookup("rmi://localhost/TestServer");
            System.out.println(test.sayHello());
        }
        catch (Exception ex) {
            ex.printStackTrace();
        }
    }
}

Once you run the server, a service “TestServer” is created in the local RMI registry. The client will then invoke the server ‘sayHello’ method, and return the answer.

Naturally more complex examples and interfaces and stuff can be bolted onto this. But this seems to be the very bare bones system–and it avoids stuff like rmic and security permissions and stuff like that.

I ran this on my Macintosh (System 10.5) running JVM 1.6 in Eclipse 3.5 without any problems, outside of wasting an hour trying to figure out why I was getting an “java.security.AccessControlException” exception–which could directly be traced to setting up a security manager.