In my last blog post I was expressing my hopes for a Write-Once-Run-Anywhere solution for Java mobile. One of the founders of CodeNameOne responded to the post by saying that CodeNameOne offered everything that I was looking for. It provides a full solution for writing mobile apps in Java that will run in iOS, Blackberry, Android, and J2ME. They provide a Java class library that encapsulates most useful functionality required by a mobile app (e.g. GUI toolkit, Contacts, Camera, Location, Database, Network, Video, Web Browser, etc…). Their GUI solution is akin to Swing in that it is light-weight and pluggable – making it ideal for cross-platform development.

I was already aware of CodeNameOne and had it at the top of my list to try out. A few months ago, I downloaded their Netbeans plugin and played with it a bit – though my “playing” was limited to just starting a new project from a template and trying out their build server to see what the finished iOS executable looked like. I was actually quite impressed that the app (a simple 4 tab application with a few buttons and a map) came out at a trim 6 megabytes. For java this is good, because this had to include all of the components from the JRE compiled into the binary.

My main stumbling block was the dependence on their build server for actually producing the distributable application. The build server, from my brief interaction with it, works flawlessly, but I don’t like making myself too dependent on factors beyond my control. I have been burned already several times by “the cloud”. I have written software that relies on cloud APIs that are backed by major players like Google. In every case, the API has either been discontinued at some later date, or has been changed to a pay-per-use API that renders the application unusable. As another example, when the iPad first came out I purchased some comics through an app that was based on “the cloud”. I stopped using the app for a year, and when I went to use it again, the company had changed to a 3rd party provider for their purchasing – and all of the comics I had purchased no longer “worked”. Similar things have happened in the Mac App store due to changes in policies.

In any case, I am no longer naive (and may be a bit jaded) when it comes to building a process that relies heavily on “the cloud”.

With these “cloudy” experiences as a backdrop, the ability to build my own applications (at least as an option) is very important to me. In navigating around the CodeNameOne site (when I was first investigating it), I was not able to find any instructions on how to build the applications myself. A few questions had been asked in the forum but the response was that, while it is possible to build it yourself, the weren’t supporting it because it was complex and they didn’t want to clog up the support forum with requests of this nature.

Thankfully, I was finally able to find some instructions in the forum on how to build a CodenameOne application using XMLVM. This thanks to jon…@gmail.com for posting these. They were a lifesaver.

The Avian Port

I also mentioned in my last blog post that Avian had great potential for bringing Java to iOS since it is a lightweight JVM that provides AOT compilation. Its creator, Joel Dice, has already posted a simple “Hello World” application that runs on iOS using Avian. This provides a solid foundation for further development, and importantly a starting point for writing an Avian port for CodeNameOne

Let’s back up a little….

Why would I want to write an Avian port for CodeNameOne? Didn’t I just say that CodeNameOne already provides the ability to deploy Java apps on iOS?

My motivations for writing an Avian port for CodenameOne are three fold:

1. To get intimately acquainted with the CodeNameOne architecture so that I can become a productive developer on the platform.
2. The get familiar with Avian so that I can make use of it in my future projects.
3. To see if I can improve the performance. UPDATE: Check out my later blog post where I discuss my benchmark results

The performance issue is of abstract importance to me at this time. If performance didn’t matter at all, I would just use a stack like PhoneGap to create multi-platform apps. But performance matters, you you *can* tell the difference between a native app and a PhoneGap app when it comes to things like scrolling. Writing games and heavily graphical applications also depend greatly on performance. Especially when we’re talking about low-powered mobile devices like the iPhone.

CodeNameOne’s iOS implementation is currently based on XMLVM, which is a brilliant project developed and maintained by Arno Puder. It works by converting the primitive virtual machine instructions into an intermediate XML representation, which can then be transformed into another language. E.g. The JVM is a stack based virtual machine, so many of the instructions just boil down to stack operations. These can quite easily be represented in any programming language. In XMLVM’s case it is converting the Java VM instructions into C source code.

If you actually go through the build steps for XMLVM, you can take a look at the C code that is generated. Basically a C source file corresponding to each Java library that is used in the application, is produced. It uses name mangling to produce functions that correspond to each Java method, and it provides a small set of runtime functions to be able to interact with the generated code (e.g. convert data types).

I think this solution is particularly clever. It gives us an excellent baseline for porting from one language to another. My instinct tells me, though, that the performance probably takes a bit of a hit due to the increase in number of commands required to perform each function. I haven’t done any benchmarks to test out this theory, but Shai Almog did acknowledge this during the JavaOne demo. (UPDATE: Check out my later blog post where I discuss my benchmark results). He did say that many of the important native iOS functions have been hand-coded, though so performance is pretty good where it counts.

Avian, on the other hand, compiles Java down to machine code so it should be very fast. I suspect, almost as fast or even faster than Objective-C for some things. I would like to be able to compare the performance, side-by-side, with the XMLVM implementation so that I can find out how much of an improvement there is to be gained. Realistically, if we want to treat CodeNameOne as a platform on which we can build a vibrant ecosystem of applications, plugins, and libraries, then the foundation needs to be as *fast* and *solid* as possible. UPDATE: Check out my later blog post where I discuss my benchmark results

Creating the Avian Port

In order to create the Avian port, I just needed three puzzle pieces to fall into place:

1. I needed to see a sample of a project that can be built and run on iOS. The hello-ios sample project posted by Joel Dice, contains enough detail to work with in this area.
2. I needed to see how the existing XMLVM port is built. Thanks to Shai Almog for pointing me to this post that describes the steps..
3. I needed to know a little bit about the CodeNameOne architecture. Specifically where it interfaces with the native environment. Thanks, again, to Shai Almog for giving me some pointers in this area. It turns out that most of the native interface is contained in a single class: com.codename1impl.ios.IOSNative. Its native implementations were likewise contained inside a single IOSNative.m file which can quite easily (though tediously) adapted to use JNI instead of XMLVM.

With these 3 pieces in place, it was just a matter of time, sweat, and carpel tunnel syndrome to make the Avian port a reality.

The CodeNameOne Architecture & The iOSPort Project

If you want to start hacking on the CodeNameOne core, the best place to start, is with the code. You can check out the entire repository, which includes the foundation libraries, the designer project, and all of the port sub-projects. These instructions almost worked perfectly for me, except that there were some problems compiling the bar-code reader libraries, which likely are a new feature since the instructions had been written. I didn’t actually get it to build completely, but I got close enough that I’m sure I could have without too much extra work.

The iOSPort project includes a blueprint that can be quite easily followed to port CodenameOne to any platform you desire. All platform specific functionality is provided by a single class, CodenameOneImplementation, which can be overridden as necessary to implement functionality for the platform. The iOSPort project further factors out most native methods into the iOSNative class. This native class uses XMLVM conventions for native classes rather than the more common JNI conventions, but it was easy enough to generate JNI headers for this class and port all of the functionality over.

With the XMLVM-powered port, all of the source code in CodeNameOne, Apache Harmony, the iOSPort project, and the actual mobile app that is to be built is converted to Objective-C code using the XMLVM compiler. This produces a rather large project since .h and .m files are generated which correspond to each Java class that is used. A set of hand-coded native sources (Objective-C) in the nativeSources directory of the iOSPort project are also included. These one of these Objective-C source files is IOSNative.m, which includes all of the native implementations for the IOSNative class, but there are also several other classes that encapsulate various other components, and are referenced by IOSNative.m.

In producing an Avian version, I primarily just had to go through each of these hand-coded native files and replace all references to XMLVM with corresponding functionality that makes use of the JNI functions. Most of the references to XMLVM involved type conversion and array manipulation, which XMLVM provides some runtime convenience methods for. These replacements were tedious but not terribly difficult. After all of the replacements were made, it was just a matter of adding all of the source files to an Xcode project that is set up to build using Avian, then dealing with build errors one at a time.

The iOSPortAvian Project

Once I replaced all of the XMLVM references with JNI equivalents, I was able to set up an Xcode project scaffold based on the Hello-IOS example. The process for setting up the Xcode project was roughly as follows:

1. Download the OpenJDK Mac port files from SVN. Download Avian from SVN. Download Proguard. Download the Hello-ios Project.
2. Add all files in the iOSPortAvian/nativeSources directory to the Xcode project.
3. Build the JDK.
4. Copy all of the Java source files and resource files that are found in the CodeNameOne project and in the application to be built into the src directory of the Hello-IOS project.
5. Use a modified make file to build the Java source code into native binaries and link them into the Xcode project.
6. Create a custom entry point for the application, main.m, that initializes the JVM using JNI and passes control to the CodenameOne_GLAppDelegate class to enter the application.

Most of the build steps have been included in the build.xml ANT task that is part of the iOSPortAvian project.

I have uploaded the entire project source on GitHub as a point of reference for others who may want to extend it or learn from it.

I will write the specific steps required to build your app with Avian in a future blog post.

The KitchenSink Demo App Running on Avian

My first test app was just the “Hello World” app that is produced from the CodeNameOne template in Netbeans. The next one was the Kitchen Sink demo app. Most of the app started up with no problems. Yay!! Themes worked. Effects and transitions worked. As a proof of concept this feels very close to viable.

>Problems with Garbage Collection

Unfortunately I hit a snag that I suspect is related to Avian’s garbage collection. After performing a few successful transitions and functions, the Kitchen Sink app would crash with an EXC_BAD_ACCESS error (using the iOS simulator). I suspect this problem is related to Avian’s garbage collection but I don’t really have any leads on how to solve this issue. I have posted the issue in the Avian forum but haven’t had any response yet. Until such time as I have a lead in this area, the Avian port will remain 99% complete – but unusable.

UPDATE: Thanks to Joel Dice in suggesting some debugging techniques, I was able to solve the garbage collection issue. It is now working 100%!!

Thoughts in Summary

1. CodenameOne is the real deal. It really does provide cross-platform mobile development in Java. I really hope the community gets behind this project because it is worth of attention.
2. The CodenameOne devs (Shai Almog, and Chen Fishbein) deserve medals for their work on this project. The massive scope of the project across so many platforms surely required a massive effort to get it working. If you look at the CodeNameOne forums you’ll see that Shai responds to pretty much every inquiry that comes in the same day in most cases. If you look at the source code of the project, you’ll see their names in pretty much every javadoc, which tells me that these guys are wearing both the tech support hats and the dev hats for this project. I really hope a community forms that takes some of the burden off of these founders. Being the maintainer of some other open source projects myself, I know how difficult it can be when you have to respond to every question that comes into the forum.
3. Avian is very cool also, but the community doesn’t seem to be there. I have posted 4 questions to the forum and have not received a response to any of them. I answered my own queries in the forum when I found the answers myself, but the last query that is currently the show stopper, hasn’t received a response yet. Since Avian is the only pure AOT solution that I’m aware of that has working examples on the iOS, I think it is still work paying attention to even if the community isn’t really there yet.
4. XMLVM is brilliant. A great way to learn about the possibilities of XMLVM is to look at the iOSPort project of CodeNameOne. It is a wonderful example of the possibilities. I didn’t actually get to the point where I could compare the performance with Avian/Native code, but the fact that it is being used successfully in CodenameOne serves as a compelling proof of concept for it. Great work, Arno and team on this.
5. I still don’t have a good handle on the relative benefits of using CodeNameOne vs using an HTML5 technology (e.g. PhoneGap) vs using Native (other than the fact that Java is a superior language, IMHO). Does CodeNameOne’s performance rival native apps? Does it beat the performance of HTML5 apps? If so, what tangible things can be achieved with CodeNameOne that cannot be done with HTML5?
6. Currently CodeNameOne doesn’t support external Jar files. This is partly because it doesn’t support the full J2SE spec for all platforms. You can work around this problem by just adding the source files for the external libraries into your application’s project. However this kind of nullifies one of the best features of Java: the ability to build up collections of third party libraries that can be used in multiple projects. I suspect that this feature will be added in future releases, but it is certainly a notable omission for now.
7. All of this tinkering with building my CodeNameOne projects on my own are merely academic for now. Even if I did produce a working Avian port, I would probably still opt to use their build server since it is so automatic and they have worked out all of the little issues with optimization. That said, I think without easily accessible public build instructions, it will be difficult for an open source community to build up.
8. I still want JavaFX on mobile. I haven’t really explored the limits of the UI and graphical API capabilities of the CodeNameOne libraries, but I can guarantee that they don’t come close to approaching the capabilities of JavaFX.

Follow Up

Check out my later article where I discuss the results of a benchmark comparing the performance of XMLVM and Avian.