All posts by shannah

Steve Hannah is a 28-year-old software developer currently studying and working at Simon Fraser University in beautiful Vancouver British Columbia. He specializes in web information systems and prefers Java, PHP, and Python as programming languages. He is a Christian and worships at Christ Church of China in Vancouver.
Java, OS X Development, Software Development, Uncategorized

Speaking Cocoa From Java

Java on the Mac recently got a major boost to its profile as a platform for building Mac software when JavaFX ensemble was accepted into the App Store. This was not the first Java app to make it into the app store, but it was the first that used JavaFX. Having recently attended a number of talks on JavaFX at JavaOne, I can tell you that it is really something to get excited about, especially if you are a Java developer. The combination of its fast, rich graphics and media support with a first-rate visual GUI builder (JavaFX Scenebuilder) make for a platform on which you can be both productive and creative.

So let’s get busy writing Mac software in JavaFX. Well….. not so fast. In order to comply with App store guidelines, and to provide a native user-interface experience for Mac users, you still need to be able to talk to some native frameworks.

One major problem is that JFileChooser is not compatible with the Mac App Store because it doesn’t work inside the App Sandbox. The current recommended approach is to use the old java.awt.FileDialog class, but this has some serious limitations. For example, if your user enters a file name with an incorrect extension for your purposes, you are not allowed to programmatically fix it, and the FileDialog doesn’t include any features to force a certain file extension. In cases like this it would be really nice to just be able to use the native NSSavePanel class to show a modal save dialog without having to jump through too many hoops. In a previous post, I described how to use JNI to build a Java wrapper around NSSavePanel, and this is a viable way to go, but this is quite a lot of hassle to just be able to use a simple file dialog.

The NSSavePanel example is only the tip of the iceberg. When you start looking through the Mac OS X frameworks, it becomes clear that there are a lot of fun toys to play with, and a few other essentials that you cannot live without. (E.g. if you want to allow in-app purchase, you’ll need to be able to interact with the StoreKit framework).

Do we really have to battle with JNI, every time we want to make a call to an Objective-C library? Luckily, the answer is no. We do have some easier options. The Rococoa framework is an open source successor to Apple’s discontinued Java-Cocoa bridge. It sits atop JNA and provides access to some core Mac OS X frameworks by way of generated Java class stubs. JDK7 and JDK8 have a little documented library called JObjC included with it, that also provides the core OS X frameworks as generated Java class stubs. Unfortunately, both of these libraries suffer from a serious lack of documentation. In the Rococoa case, there is some documentation, but most of it was written in 2005 and much of it no longer works. JObjC doesn’t seem to have any documentation to speak of aside from the unit tests (It doesn’t even include any javadoc comments).

The biggest pitfall of both of these libraries (in my opinion), is that they rely on you to build class stubs for the frameworks that you require. There are build scripts for this (e.g. JNAerator for Rococoa and JObjC has its own Ruby script that generates class stubs using the OS X bridge support XML files), but they don’t work out of the box. And frankly, after spending 3 days fighting with JNAerator and not getting it to successfully build any stubs, I’m not even sure it the current version even works. The trouble with having to run build scripts is that you need to have your build environment set up just right for them to work. If you reach a point in developing a Java application where you just want to call native method, or access a native framework, you don’t really want to have to go and build an entire mess of class structures from source. Or maybe that’s just me.

So then what? Is there a simpler option? Not that I could find. So I built one myself.

Some background on me: I am primarily a Java developer, but I have read numerous books on Objective-C and Cocoa with the anticipation that, some day, I might actually get around to writing a Cocoa App. I, therefore, prefer to write more code in Java and less in Objective-C, but I’m quite familiar with how the Objective-C runtime works and what the Cocoa frameworks have to offer.

One nice thing about Objective-C is that it has a dynamic runtime that you can easily interact with by way of just a few C functions. It provides a means of passing messages
to objects. This is the entire basis for the “Objective” part of Objective-C. JNA is a fantastic library that allows us to call C functions directly from Java. Therefore, using JNA we can interact with the Objective-C runtime via its C API.

When it comes right down to it, all I really need is a few functions:

  1. objc_msgSend(), to pass messages to objects.
  2. objc_getClass(), to get a pointer to a class.
  3. selector_getUid(), to get the pointer to a selector.

And there are a few other useful functions, but I won’t mention them here. The first thing I needed to find out was how an Objective-C object looked inside of Java (using the JNA API). As it turns out, an Objective-C object can be simply represented by its address as a long variable. JNA provides a useful wrapper class, com.sun.jna.Pointer around an address that can be used interchangeably in your JNA method wrappers.

With just this tiny piece of information, you can start to see what would be required to build a generalized wrapper for Objective-C from Java. By obtaining a pointer to a class via the objc_getClass() function and a selector via the selector_getUid() function, you can create an instance of any class by passing the selector to the class via the objc_msgSend() function.

You can then pass messages to the resulting object using the objc_msgSend() function again.

This is almost enough to form the foundation of a one-way, working bridge. But there are, of course a few caveats.

For example the objc_msgSend() only works for messages that have simple return types (e.g. Pointers, ints, etc..). If it returns a structure, then you need to use the objc_msgSend_sret() function, and messages that return a double or float value need to be sent with the objc_msgSend_fret() function.

But, for the most part, this understanding is sufficient for creating a one-way bridge.

After a little JNA magic, we have a simple, low-level API for sending messages to the Objective-C runtime. The following snippet is from a unit test that is testing out this low-level API:

However exciting it may be to be able to send messages to Objective-C objects, this level of abstraction is still too low to be “fun”. So I built a little bit of abstraction over top of this layer to, for example, allow sending messages without having to first look up the class and selector.

The following is a snippet from another unit test that is sending messages at a slightly higher level of abstraction:

And this can be improved even more:

At this point, we can quite easily interact with Cocoa without too much pain. But we still aren’t quite at the level of abstraction that we like. The next step would be to build an object-oriented wrapper around the Objective-C objects so that we’re not dealing directly with Pointers and long addresses to objects.

For this, I created the Client and Proxy classes. The Client class is an singleton class that allows you to interact with the Objective-C runtime. It allows you to send messages to objects, and it provides automatic type mapping for both inputs and outputs so that the correct message function is used. E.g. using Objective-C Type codings for messages, it determines the return type and input argument types of the messages so that the correct variant of objc_msgSend() is used and that the output is in a format of our liking.

Objective-C objects are automatically converted to and from Proxy objects
(which are just wrappers around the Objective-C object pointers). NSString objects are automatically converted to and from Java Strings. It also introduces a set of
methods for sending messages that expect a certain return type. E.g. The sendInt() method is for sending messages that return ints, and sendString() method is for sending messages that return Strings.

The Proxy class is a thin wrapper around an objective-C object. It uses a Client instance to actually send the messages, and it provides wrappers around all of the relevant send() methods.

The following is some sample code that interacts with the Objective-C Runtime at this slightly higher level of abstraction:

At this level of abstraction, it is pretty easy to add little bits of Cocoa functionality into our Java application. But there is one Major thing missing: Objective-C to Java communication. This is critical if you want to be able to write delegate classes in Java that can be passed to Objective-C objects to handle some callbacks.

Calling Java From Objective-C

One-way communication will get you only so far. If you need to pass a callback to an Objective-C message, or write a delegate class in Java that can be called from Objective-C, we’ll need full two-way communication.

There are a few ways to accomplish this, but I chose to make use of the NSProxy and a little bit of JNI to create an Objective-C object that acts as a proxy to a Java object. Whereas our Proxy Java class that we defined before contained a one-way link to its native peer object, the NSProxy subclass will provide its own one-way link to its Java peer. Combining these two strategies, we end up with a two-way bridge between Java and Objective-C, such that we can call Java methods from Objective-C, and Objective-C methods from Java.

I’ll go into detail on how this was achieved in a future post, but for now I’ll give an example of how the resulting API looks from the Java side of things.

The following is a sample application that opens the NSOpenPanel dialog and registers itself (a Java class, which is a subclass of the NSObject Java class) as a delegate for the dialog.

We introduced a special annoation @Msg to mark Java methods that should be accessible as Objective-C messages. This allows us to specify the selector and signature for the message in a way that the Objective-C runtime understands. Notice that we can even use the Proxy.send() method to send messages to our Java class. This actually sends the message to the Objective-C runtime, which pipes it back to the Java class to be handled — and if the Java class doesn’t define a method to handle the message, it will pipe it back to Objective-C to be handled by the superclass.

To find out more and download the library…

You can download the source or binaries for this project on Github. I have also posted the Javadocs here.

Javascript, Mobile Development, Software Development, Xataface

Scrolling iFrame on iOS

I just finished adding scrolling support to the RecordDialog class in Xataface for the iPad. The RecordDialog basically allows you to open an edit form or new record form in an iframe using a Javascript call. This is useful if you want to be able to pop up a form without the user having to navigate away from the page. It uses an iFrame for legacy reasons, until the forms API can be updated to work 100% through AJAX. This component has worked well for a long time on the desktop, and it works okay on the iPad and iPhone if the form fits inside the iframe without having to scroll. The trouble is that you can’t scroll an iframe on iOS, so if the form is too long, it just gets cut off and the user can’t see the bottom of the form.

After some Google searching I found a few strategies for overcoming this issue. The proposed solutions can be categorized into 3 groups, and all of the solutions have one common element: start by making the iframe unscrollable using the scrolling=”no” HTML attribute. From there, you can either:

  1. Set the iframe height to be as tall as the document body, and wrap the iframe in a scrollable div tag.
  2. Wrap the contents of the iframe in a scrollable div tag. (This requires sizing the DIV’s height to be the same as the iframe’s parent’s inner height).
  3. Use javascript touchStart and touchMove events to do scrolling with Javascript inside the iframe.

After experimenting with options #1 and #3, I settled on solution #2: wrapping the iframe’s contents because it seems to work best for all occasions. Of course this solution won’t work if you don’t own the contents of the iframe.

Option #1 (wrapping the iframe in a scrollable div) is problematic because, by changing the size of the iframe to be taller than the screen, potentially, it screws up calculations in the body of the iframe that rely on window height. I found that dialogs that are created to be displayed in the middle of the page end up displaying way down the page because the window is deemed to be effectively the size of the iframe, which has been artificially resized to be way too big.

Option #3 didn’t look as appealing because of the amount of custom javascript handling. I just have the feeling that I would have been starting down a long road of browser incompatibility glitches.

Resources

  • A good blog post on some of these strategies
  • A stack overflow conversation on the topic.
    • Software Development

    OS X JNI Tutorial

    I have just recently completed my first JNI library for use with one of my applications. Since I have found documentation on JNI (especially on Mac OS X) to be sparse, I decided to write a tutorial about the process so that others can learn from some of the mistakes I made. I wrote the tutorial using Docbook. See the links below to view the tutorial.

    HTML Version

    PDF Version

    Feedback welcome.

    Codename One, Java, Javascript, Mobile Development, Software Development, Xataface

    JavaOne: A Look Back

    I am now home from my Bay area adventure at Java One. This was the first conference I’ve attended since 2005, and is by far the largest conference I’ve ever been a part of. I was blown away by the scale of this one. The downtown core was plastered with Oracle and Java signs. Buses dipped in oracle insignias roamed the streets constantly, and you couldn’t walk a block without seeing at least 5 people brandishing Oracle swag. (I quite liked the back pack that they provided).

    In addition to the sessions (which I’ll get to next), there was a steady stream of entertainment events for the benefit of conference attendees. Union square (and a few other locations) were blocked off for music performances; and then there was the “fan appreciation” event featuring Pearl Jam and Kings of Leon. Doesn’t sound like nerd conference, does it?

    The Sessions

    My reason for attending was the sessions. From Sunday to Thursday my schedule was jam packed with talks about topics ranging from JavaFX to MongoDB and a smattering of Java ME, iOS, and Android development. For any given one hour slot, I had to choose between 2 or 3 talks that I really wanted to see, and another 8 or 9 that would have been interesting. Being able to hear about new technologies directly from the people who are behind it is awesome. It imparts a kind of understanding that cannot be achieved through a book or reading blog posts.

    This conference has sparked a fire in me to seek out more conferences like this, similar to the way that a my first exposure to Disneyland led me to look for other parks that offer the same experience. In fact, the way a child reacts to his first experience at Disneyland is an accurate way to describe my reaction to the conference schedule.

    The Topics

    Currently I have a number of desktop applications that have been developed in Swing and have been deployed as applications on OS X. Therefore, I was most interested in learning about Swing’s successor, JavaFX, and how to deploy applications on OS X. The conference was heavy on JavaFX (yay!) but a little light on OS X. There were two talks on OS X deployment, but sadly I could only attend the first one (which way a fantastic talk, by Scott Kovatch). Nonetheless, I have left the conference with stockpiles of new ammunition for building applications with JavaFX. In fact, I’m really exciting about using JavaFX and Scene Builder (the GUI editor for JavaFX) to start building some business applications at work.

    The demos provided by some of the community served as proof of concept to me that JavaFX is something special. E.g. Gerrit Grunwald’s (@hansolo_) set of gauges in the JFXtras project show what can be achieved with a little time and artistic flair.

    On the OS X front, it looks like we’re all set to go now with the Mac App store. The javafxpackager tool provides the ability to easily deploy applications as native bundles. This is a new direction for Java (up until now, they have been recommending Web Start distribution), and I think it is a good one.

    Another topic that I am keenly interested in is mobile development (iOS and Android). Last year’s java one included a demonstration of JavaFX running on an iPad, but then nothing more was heard. This year they announced that JavaFX will be available for Java7 SE Embedded, and they had a conference scheduler application running on an small, embedded device in a kiosk at various stations in the conference. But there were no announcements about JavaFX for iPad or Android. The announcement of its availability on embedded devices is a step in the right direction, though, as this is a prerequisite for running on the major embedded platforms.

    I did attend one talk (actually 2 talks on the same topic) on Java for iOS and Android. The CodenameOne project (that I’ve blogged about before) is a java library and framework for building cross-platform mobile apps (i.e. deployable on Android, iOS, Blackberry, and J2ME). This looks very promising.

    When I had reached my saturation point on JavaFX, I wandered across to the Parc55 building for a couple of Java Enterprise Edition talks. I attended a talk on the new Project Avatar, which provides a thin-server approach to web applications (i.e. most view logic and layout is handled on the client in javascript, and the server just sends over the data). This project looks quite interesting, both in concept and execution. The concept coincides with much of the more recent development I’ve been doing with Xataface.

    The last talk of the conference that I attended was on NoSQL and scaling strategies. There, Nike’s director of engineering discussed some of their architectures for handling heavy traffic with NoSQL databases and Data grids.

    My head is full. My notebook is full. Now to put all of this new knowledge into practice.

    Software Development

    “Engineers” in the US

    During the course of a dinner last night, a “canadianism” was brought to my attention that I was quite unaware of. A friend of a friend who is working at Google mentioned that the project he is working on is “quite small”. When I asked what “quite small” meant, he responded with “about 100 engineers”. Wow, I thought. “If this project has 100 engineers, how many software developers does it have?”, I asked naively. He responded that, in the US, software developers ARE engineers. It is just the language. If you develop software, you say you’re an engineer. Of course they mean SOFTWARE engineer, but I note that this first part is omitted by default.

    In Canada, the word “Engineer” is a professional term reserved for professional engineers. These are people who have a special P.Eng. designation, and who put their designation on the line whenever they sign off on a project. Engineering is a profession, like doctor or lawyer. Just as you cannot practice law without being a lawyer, you can’t be an engineer without your certification.

    Software Engineering, though it exists, is not a profession in the same sense, and it perhaps never will be. The absence of an official designation appears to have led to a dilution of the term to the point where it has no meaning. All these years, when people have asked me what I do, I have always responded with “Software Developer”. Apparently, when talking to Americans, I can now call myself an engineer.

    If I were a real engineer, I’d take offense. But I’m a software developer, so I’ll take engineer.

    Java, Javascript, Software Development

    Ingredients required to port HTML/Javascript App to JavaFX

    I have been taking in as many JavaFX talks during Java One as I can. I’m keen to move some of my development from LAMP/HTML5 to LAMP/JavaFX (Yes.. still not ready to move into the JavaEE space – though PLAY makes it compelling).

    The Scene Builder application, complete with its powerful CSS support, looks like it will make the UI portion of app development quite easy. In fact, I suspect it may even improve my productivity over my existing HTML development.

    The one piece that I’m concerned about is the HTTP client for consuming the LAMP web service. In Javascript you get quite a bit for free, but the two biggest are:

    1. Sessions/Auth are automatically retained.
    2. AJAX calls and conversion to/from JSON is automatic.

    To replicate the functionality in a JavaFX app, I suppose I need to embed an HTTP Client object of some kind to maintain all of the session information and allow me to load JSON data sets from the server in a simple manner. At the “JavaFX for Business App Developers” talk, the speaker used Jersey in is example application to consume an XML web service. The Jersey website makes it sound like it can also be used to consume JSON web services. Will have to give it a try.

    Another option, which I’ve yet to try, is to actually embed a Web Engine into the app for dealing with AJAX calls. This is either a very good idea, or a very bad one. I would be concerned about such things as performance, and security restrictions that may exist inside the web box.

    I’ll post again with my findings.

    Java, Software Development

    JFXtras: Cool Widgets for JavaFX

    I just attended a session at Java One for JFXtras. I was very impressed by the prolific collection of gauges that were contributed to the project by Gerrit Grunwald (@hansolo_). He has developed incredibly intricate controls to simulate traffic lights, odometers, LCD panels, clocks, and just about every type of gauge you can think of. These were all developed with JavaFX and they are all visually stunning.

    I think that Grunwald and artists like him will be great ambassadors for Java FX and its capabilities. This is still very young technology, but I’m becoming quite convinced that it is here to stay.

    Java, Javascript, Software Development

    PLAY Framework Observations

    Going into James Ward’s talk on the PLAY framework I had no hands-on experience with the framework. I only knew of it from the odd mention in other developers’ Facebook updates.

    Highlights:

    1. A light-weight web application framework with its own integrated web server (Netty).
    2. Stateless. This is a key principle of PLAY that helps encourage scalability.
    3. Encourages non-blocking asynchronous request handling to further enable scalability (e.g. it can handle more than one request per thread).
    4. Applications can be written in either Scala or Java.
    5. Nice MVC separation (Views, Models, Controllers).
    6. Uses Ebean and JPA for ORM.
    7. Integrated test framework.
    8. Asset compiler (to minify javascript automatically etc…).

    The framework reminds me very much of Ruby on Rails, but the fact that it runs on a JVM makes me far more likely to try it.

    I particularly like the light-weighted-ness of PLAY. It really embraces the HTTP protocol, not trying to hide it from the developer. This actually gives me more trust as a developer who would be just diving into a new system. Some of the heavier Java frameworks hide too much, resulting in extra time spent trying to figure out how to adjust the long levers of the framework to achieve something as simple as an HTTP redirect.

    Ward made note that the stateless aspect of PLAY is important. He said that if you require state (as most applications do), you should relegate this to a store like memcached that can be scaled across multiple instances. This makes a lot of sense. I frequently reach the edge of what can be achieved on a single web server. If you depend too much on “state”, it makes it terribly difficult to introduce a 2nd server effectively. Moving to a “stateless” paradigm enables you to fully leverage the HTTP protocol, introducing caching and proxies effectively at many different positions in the request chain.

    Java, Software Development, Uncategorized

    In Depth with JavaFX Scene Builder

    This post is just a collection of my observations after watching a talk at JavaOne by Jean-François Denise about the JavaFX scenebuilder application.

    One word summary: Wow!!
    Two word summary: This rocks!

    Scenebuilder is a GUI application for editing FXML files (an XML format for defining user interfaces in JavaFX). It provides a drag and drop canvas for placing components in a scene. It really is almost as easy as photoshop when it comes creating pretty UI designs. In fact, it may be an effective tool for mocking up UIs. You can drag images directly out of finder onto a scene and then start resizing it, rotating it, etc.. right on the stage.

    I was particularly impressed by how well it uses CSS for customizing the look of an interface. The CSS works just like normal CSS in the web, except it has its own directives prefixed by “-fx”. CSS stylesheets can be added at any level of the DOM. You can attach a stylesheet to the whole stage, or to a single panel, or even a single button. You can also add inline CSS styles to an individual tag. Two differences from HTML CSS is that JavaFX also allows you to define a theme and bean properties to style elements. The order of precedence is:
    1. Inline
    2. CSS Stylesheets
    3. Bean properties (e.g. properties set in the node inspector).
    4. Themes

    Scenebuilder luckily keeps the styles sorted out so that you can don’t have to pull your hair out trying to find out where the “effective” style is defined. If you have overridden a bean property in a stylesheet, the bean inspector will no longer allow you to edit the property, but rather will link to the exact part of the stylesheet where the property is set.

    There is also a powerful CSS analyzer that allows you to see all of the CSS properties for each node, and where they are defined. If you make a change to a CSS file, the preview of your UI will be updated instantly.

    Denise briefly showed a demonstration of how to add a custom type to the scene builder, although apparently there is still work going on in this area to make it easier to add custom components. Currently you can only use the set of built-in components – and there are some notable components still lacking.

    So far I’ve only played around with Scenebuilder and JavaFX briefly and sparsely. However I am looking forward to digging in.

    Java, Javascript, Software Development

    The Metro UI

    I attended a talk yesterday on “Java FX meets Metro” by David Qiao. Going in, I was hoping for some information about deploying JavaFX on Windows 8 in some format that is acceptable by the new windows 8 app store. In fact, this talk was not about that at all. Rather, it discussed strategies for creating a Metro-esque user interface using Java FX and didn’t talk about deployment at all. Even so, I got quite a bit useful information from the talk.

    Some highlights:

    1. Metro is a set of guidelines for the development of user-interfaces that are optimized for touch-based interfaces.
    2. Many user interface paradigms that worked well with desktop applications, don’t work so well for touch devices (e.g. tree navigation, breadcrumb controls).
    3. Java FX makes it relatively simple to design a Metro-esque UI with its animation and CSS support.

    There are basically two kinds of metro-style apps:

    1. Flat – e.g. just one view, maybe some tabs.
    2. Hierarchical – Multiple levels.

    Hierarchical apps should contain only 3 levels:
    1. The hub page. Like the homepage with a view of all of the content – or at least a good summary.
    2. A Section page. This would be a list of individual items for the user to select.
    3. A Detail page. This shows a single item of the application.

    Metro also introduces the notion of a semantic zoom to be able to move to a more or less detailed view of the information. E.g. in a section page you can “zoom” in on an item to show its sub-items. Similarly you can zoom out of the sub-items to see a more succinct view of the items.

    Just as desktop applications of some common components (e.g. menu bars, windows, etc..), Metro apps have their own common components. These include:

    1. A tile (like an icon, but usually presented as a flat rectangle).
    2. Tile pane (a tile that contains a bit more content).
    3. Nav Bar – A menu usually at the top or bottom of the UI providing transient access to other areas of the app.
    4. App Bar – A more contextual menu providing actions that the user can perform in the current context.

    One key design principle that is a departure from traditional UIs is that you almost never see vertical scrolling. Scrolling is almost always horizontal. This, intuitively seems more natural to the user (like turning the page of a book) when on a touch interface like an iPad.

    I’m keen to try to adopt some of these principles to my own apps. Especially html5 apps. It should be relatively easy to build a metro-esque application using HTML and CSS. The rewards could be great to the user as it really seems like Microsoft has gotten a lot right with these principles.

    Now to see what I can find in the way of existing Metro HTML5 stylesheets and Javascript/jQuery libraries.