Category Archives: Java

Javascript Development with the Xataface Javascript Compiler

Javascript is a wonderful language, but I always felt as if it wasn’t terribly scalable. Not so much in the performance sense, but in the component sense. When working on large projects (like Xataface) Javascript can seem a little too “scripty” to build components in. I’m always fighting between the two conflicting goals of library size and library functionality.

If you build a useful library in Javascript it is usually either comprehensive and large, or small and single-purpose. It is possible to build the library in pieces and just include the parts that you want, but then you end up with a cumbersome set of script includes in your HTML page every time you want to use your library’s functionality. This is error-prone as it is easy to forget which scripts you need to load for any particular function to work.

In server-side languages you don’t have this problem. When you write a script in C, PHP, Python, Java (or any other language), you can just include your dependencies at the top of your file using an import, include, or require statement (syntax depends on the language). Then using your library is as simple as including one file – and all of that file’s dependencies will automatically be imported as well.

To solve this problem, I developed a Javascript compiler as part of Xataface. Some features of this compiler include:

  • Compiler directives to process dependencies.
  • Javascript Compression – to minimize the footprint.
  • Duplicate Dependency tracking (i.e. If multiple scripts all require the same script as a dependency, then the dependency will only be included once).
  • Compiler directives to include CSS files.
  • Compiler directives to import files as strings.
  • Configurable source paths. (i.e. Just like with other languages, you can specify multiple paths where source files will be located).

This doesn’t sound like a lot, but these features really improve the Javascript development experience. It allows me to develop reusable components that can be shared and imported into many different projects without having to deal with the complexities of including script tags in an HTML page, or making sure that the proper CSS files are included. Once you have built a component, it is done. All you need to do in order to use it in your application, is include it using the //require directive.

Let’s look at an example…

All of these features are provided by the Dataface_JavscriptTool class in Xataface. So you begin by obtaining an instance of this, and adding the source path where you want it to look for Javascripts.

import('Dataface/JavascriptTool.php');
$jt = Dataface_JavascriptTool::getInstance();
$jt->addPath('/path/to/javascripts', 'http://www.example.com/url/to/javascripts');

At this point the compiler knows to look in the /path/to/javascripts directory for scripts. But you haven’t told it to include any particular script. That’s our next step:

$jt->import('myscript.js');

If you were write this code inside the context of a Xataface application, your myscript.js file would run because Xataface handles the rest of the rendering at the end of the body of any template extending the Dataface_Main_Template.html template (which is pretty much all templates).

Note that by default the following directories are included in your javascript paths:
– DATAFACE_SITE_PATH/js
– DATAFACE_PATH/modules/XataJax/js
– DATAFACE_PATH/js

So you are able to use any script in those directories without needing to call the addPath() method of the JavscriptTool.

A look at the Javascript

myscript.js:

alert('hello world');

This just pops up an alert. Let’s get to some best practices right away though. The general structure of a script usually goes as follows:

//require <myotherscript.js>
(function(){
    alert('hello world');
})();

Note the //require statement on the first line. This tells the compiler to include the script myotherscript.js in its place. The compiler will look first in the /path/to/javascripts directory, then in DATAFACE_SITE_URL/js, then in DATAFACE_URL/modules/XataJax/js, then in DATAFACE_URL/js to see if it can find a script by that name. If it cannot find one it will throw an exception with a stacktrace. If it finds it, it will include the contents of the script in place of the require statement.

Example: Requiring jQuery

Xataface comes with lots of libraries included in its source paths. jQuery is one that is used in almost every script:

//require <jquery.packed.js>
(function(){
    var $ = jQuery;
    alert('Hello world');
   
})();

Note that on the first line, we make a reference $ to jQuery. This is because in Xataface jQuery.noConflict() is set so that it doesn’t conflict with other libraries that want to use the $ variable. By declaring it inside our wrapping function(){}, we are able to still use the $ shorthand without polluting the global namespace.

Using jQuery UI

The following example is meant to show why we need to be able to bind CSS with your javascript libraries. We are going to use jQuery UI’s dialog to display our ‘Hello world’ message, and this requires a CSS file in order to look proper:

//require <jquery.packed.js>
//require <jquery-ui.min.js>
//require-css <jquery-ui/jquery-ui.css>
(function(){
    var $ = jQuery;
    var dlg = $('<div>').append('Hello World').appendTo('body');
    dlg.dialog();
})();

These examples are just the tip of the iceberg of possibilities that a Javascript compiler brings. More cool examples will be forthcoming as Xataface 2.0’s release nears.

Using a “Silent” Java Applet to Render PDFs in HTML

I have read many obituaries for the Java applet. Once upon a time, in the late 1990’s the applet was king of the internet and hope for the future of rich web-based applications. Its decline and (almost) demise can be attributed to a number of factors, not the least of which is the fact that HTML and Javascript have improved to the point where most of the the functions that used to require an applet can now be performed directly in the web browser without requiring any third-party plugins. Nowadays, you scarcely run across any websites that use applets, and when you do, they generally stick out like a sore thumb. They are generally slow to load, and they almost always pop up with a security dialog asking you to approve their execution based on a signed certificate.

Signed Applets – Full Access or No Access

This clumsy security model is the other reason for the growing irrelevance of applets. An applet can only run in two different modes:

  1. Run inside the “sandbox”. The applet cannot access any system information from the client computer, cannot access the file system, and cannot make any network connections to any hosts other than where the applet was loaded from.
  2. Signed Applet – Full access. The applet runs with full permissions of the user. E.g. it can connect to any host, access any client system information, and access the hard drive. This level of access is very risky for the user especially if the applet is just running inside some random webpage that they are trying to open. The applet has the ability to delete the user’s hard drive or copy files from their hard drive without the user even knowing.

Because applet developers are so limited with the sandbox model, most developers end up distributing their applets as signed applets so that they can be run on clients’ computers with full access privileges. This works out great if the client fully trusts the applet developer – but that isn’t the typical client/developer relationship. Most users just click “okay” whenever these types of security dialogs pop up, so it becomes very easy for malicious developers to create trojans for unsuspecting users who browse to their web pages.

Since the proliferation of signed applets across the internet would result in a very dangerous browsing experience for the average internet user, we can be thankful that the web has opted for the applet-less direction that instead makes use of HTML and Javascript – which work inside a secure sandbox. Signed applets are now generally reserved for B2B applications and in-house apps where the client computer has a trusting relationship with the developer.

Unsigned Applets Are Still Safe

All that said, unsigned applets that run inside the applet sandbox are safe for a client to run since they don’t provide any access to the client’s computer. Unsigned applets’ fade from prominence is the natural result of improvements to Javascript and HTML. Javascript/HTML/CSS now have the tools required to create complex user interfaces with rich client interaction. It is no longer necessary to embed a Swing-based user interface inside the browser to give the user a rich experience. Further, it is inconvenient to work with a hybrid of technologies so the developers will often steer clear of applets entirely – opting for pure javascript solutions. For many years, applets were still necessary to accomplish things like drag-and-drop from the desktop (though that required a signed applet), or some more complex user interface widgets, but HTML 5 has introduced most of the tools to do all of these things inside Javascript natively. So the applet has been even further crowded out.

So What Are Applets Useful For Now?

The applet’s relevance may have been eroded down to a few grains of sand, but it turns out that some of those grains still shine like diamonds. Javascript can do many things well, but there are still a few items that Java can bring to the party. For example:

  1. Javascript doesn’t handle processor intensive operations well. It runs in a single thread so any intensive operations will make the browser hang and possibly even crash. Java, on the other hand, is multi-threaded so it can perform parallel processing in the background and pass results back to the browser in way that doesn’t interfere with the user experience.
  2. Java provides a rich, almost limitless set of libraries. – Hence you can find a library to do just about anything you need. The provides an alternative to making server-side requests for processing using AJAX. Instead requests can be passed to a “silent” applet that runs on the client computer – saving both network usage and server usage.

The “Silent” Applet

Observing the strengths of Java vs Javascript yields an interesting model for making use of applets in a web page. I call this the “silent” applet because it is designed to be completely transparent to both the Javascript developer, and the user. A silent applet is one that is loaded silently in the web page and does not manifest itself visually in the web page. Rather is runs invisibly along side the web page as a daemon waiting for requests from the web page which it processes and returns to the web page. This is similar to the AJAX model where the client sends background requests to the server, and the server returns a response which the client then processes. With this model, the request is not sent to the server at all. This saves both network and server resources since much more processing can take place on the client.

The silent applet is useful in situations where a significant amount of processing is required so that performing the “calculation” inside javascript is either impossible or would be disruptive to the user experience. This might include rendering a PDF page as an image, generating a complex chart, performing an image, video, or sound transformation, etc….

Example Silent Applet: Web PDF Renderer

I am developing a PDF reporting module for Xataface. I want the UI to allow the user to arrange database fields and content over top of an existing PDF that they upload as a means of creating report templates that can then be rendered as full reports later on. For this I need to be able to display the PDF as a background image in the user’s work space. I also need the user to be able to navigate through the PDF to different pages and to zoom in and out. The current state of Javascript and HTML is such that the rendering of PDFs needs to be relegated to a different system. Either I need to render the PDF pages as images on the server-side, or I need to find a way to do it in the client.

Rendering the PDFs on the server side would require me to install some server-side extensions or applications that can handle PDFs. There are many, but my goal is to keep the server requirements minimal so that it will work inside a standard LAMP install. So my preference is to find a way to render the PDF on the client side. There are various flash and Java applets already that display a PDF inside the browser, however I’m concerned with mixing flash or applets into the UI of my editor as it will likely result in painful, or intermittent conflicts down the road as the technologies choose not to place nicely together. So I decided to create a silent applet for rendering the PDF.

The Web PDF Renderer applet runs as a silent daemon in the background. There is a thin javascript API that can be used to ask the applet to render pages of a PDF at various sizes. When the processing is done, the applet calls a javascript callback to update an <img> tag in the user interface. This allows me to keep the entire user interface in standard HTML and manipulated via the DOM.

Using the Library

One example use of the library is to render the PDF to an HTML <img> tag. The following example creates a new PDFPage wrapper (which creates an img tag), then appends the image to the body of the document, and finally renders the PDF page. The render() method sends a signal to the applet to render the PDF page. When the applet is complete, it sends the data back the PDFPage object which updates the image source with the new data.

        // Short reference to PDFPage constructor
        var PDFPage = xataface.modules.pdfreports.PDFPage;
        
        // Create a new page  (first page of document)
        var page = new PDFPage({
            width: 800,
            url: 'test.pdf',
            page: 0     
        });
        
        // Append page’s <img < tag to the document body
        $(‘body’).append(page.el); 
        
        // Render the page  (done asynchronously)
        page.render();
    

Inside the Box: The implementation

We need to set up a mechanism for passing messages back and forth from Javascript to the applet and back. For javascript to java communication it is as easy as calling public methods defined on the applet directly. These are exposed and callable in Javascript. For the reverse, we make use JSObject which is available standard as part of the Java plugin. There is quite a bit of information on Java-Javascript communication on Oracle’s website.

We created a global queue that is used to pass the messages:

PDFPage.queue = [];

Note that is is just an empty array at this point.

Calling the render() method will pass a message to the applet – it essentially works as follows (this has been simplified for the example, but gets the point of strategy across):

function PDFPage_render(){
    
    PDFPage.queue.push(this);
    startDispatch();
        
}

So render a page involves just two things:
1. We push the PDFPage object onto the message queue so that the applet can access it.
2. We call startDispatch(). This function essentially tells the applet that the queue has been updated so it can start processing.

The startDispatch() function up-close:

function startDispatch(){
        var applet = $('applet[name="'+PDFPage.appletID+'"]').get(0);
        if ( !applet ){
             var attributes = {
                name: PDFPage.appletID,
                code:       "com.xataface.modules.pdfreports.PDFRendererApplet",
               codebase: PDFPage.codebase,
                archive:    "WebPDFRenderer.jar, commons-codec-1.5.jar, commons-logging-1.1.1.jar, icepdf-core.jar",
                width:      1,
                height:     1
            };
            var parameters = {
                startDispatch:"PDFPage.startDispatch()",      
                queue:"PDFPage.queue"
            }; 
            var version = "1.5"; 
            deployJava.runApplet(attributes, parameters, version);
            
            
        } else {
        
            try {
                applet.startDispatch();
            } catch (e){
                setTimeout(function(){
                    startDispatch();
                }, 1000);
            }
        }
    
    }

At its core this is just a wrapper around the applet’s startDispatch() method – but it needs to handle some edge cases to load the applet the first time it is called. If the applet hasn’t yet been added to the dom, it adds it and tries to call itself again. If the applet is there but not loaded yet, it waits 1 second and tries again.

The deployJava.run() method is from the standard Java deployment code available here. Notice that we pass 2 parameters to the applet when we load it:

  1. We pass the javascript path to the queue so that the applet knows where to find its message queue.
  2. We pass the startDispatch() method call so that the applet is able to call its own startDispatch() method through javascript.

Inside the applet:

The start() method just loads the parameters that we passed, then calls the startDispatch() method for the first time:

public void start(){
        
        queue = (JSObject)JSObject.getWindow(this).eval(this.getParameter("queue"));
        JSObject.getWindow(this).eval(this.getParameter("startDispatch"));
    }

The guts can be found in the startDispath() method:

public synchronized void startDispatch(){
        
        if ( running ) return;
        running = true;
        
        
        Thread dispatcher = new Thread(new Runnable(){

            public void run() {
                while (!stopDispatcher){
                    try {
                        JSObject next = null;
                        try {
                            next  = (JSObject)queue.call("shift", null);
                        } catch (Exception ex){
                            break;
                        }
                        
                        if ( next == null ) break;
                        
                        String pdfURL = (String)next.getMember("url");
                        if ( pdfURL == null ) break;
                        
                        
                        PDFRenderer renderer = new PDFRenderer();
                        URL baseURL = PDFRendererApplet.this.getDocumentBase();
                        String baseURLStr = baseURL.toString();
                        int queryPos = baseURLStr.indexOf("?");
                        if ( queryPos >= 0 ){
                            baseURLStr = baseURLStr.substring(0, queryPos);
                        }
                        
                        if ( !baseURLStr.endsWith("/") ){
                            int lastSlashPos = baseURLStr.lastIndexOf("/");
                            baseURLStr = baseURLStr.substring(0, lastSlashPos+1);
                        }
                        if ( pdfURL.indexOf(":") < 0 ){
                            if ( pdfURL.indexOf("/") == 0 ){
                                pdfURL = baseURL.getProtocol()+"://"+baseURL.getHost()+
                                        (baseURL.getPort()>0?(":"+baseURL.getPort()):"")+pdfURL;
                            } else {
                                pdfURL = baseURLStr+pdfURL;
                            }
                        }
                        
                        System.out.println("PDF URL is "+pdfURL);
                        renderer.setPDFURL(new URL(pdfURL));
                        
                        String req = null;
                        try {
                            req =  (String)next.getMember("request");
                        } catch ( Exception ex){
                            
                        }
                        
                        if ( "numPages".equals(req) ){
                            next.call("update", new Object[]{renderer.getNumPages()});
                        } else {

                            int page = ((Number)next.getMember("page")).intValue();
                            int width = ((Number)next.getMember("width")).intValue();


                            renderer.setWidth(width);
                            renderer.setPage(page);
                            BufferedImage img = renderer.getResult();
                            ByteArrayOutputStream os = new ByteArrayOutputStream();
                            ImageIO.write(img, "png", os);
                            os.flush();

                            String encodedImage = new Base64().encodeToString(os.toByteArray());
                            os.close();

                            next.call("update", new Object[]{encodedImage});
                            
                        }
                        //System.out.println(encodedImage);
                        
                        
                        
                        
                    } catch (Exception ex) {
                        //Logger.getLogger(PDFRendererApplet.class.getName()).log(Level.SEVERE, null, ex);
                        ex.printStackTrace(System.out);
                    } 
                    
                    
                    
                    
                    
                }
                running = false;
            }
            
        });
        dispatcher.start();

See full applet source code

Edit: In newer versions of Java there are additional security restrictions on network requests when the method call is initiated from Javascript. Therefore is is necessary to wrap the startDispatch() method in an AccessController.doPrivileged() call (Solution found here). To accommodate this I renamed the startDispatch() method as int_startDispatch() and I created a new startDispatch() method as follows:

public synchronized void startDispatch(){
        AccessController.doPrivileged(
                
                new PrivilegedAction(){
                    public String run(){
                        int_startDispatch();
                        return "";
                    }
                }
            
            
        );
        
    }

This ensures that we won’t run into security issues when loading the PDFs.

Basically this spawns a thread the runs a loop. In each iteration, a message is loaded from the queue. It gets the PDF’s URL, and other information about what is being requested. When it is done, it calls the update() method of the original message which is a javascript method. It this method is responsible for adding the image back to the DOM.

See a demo of the Web PDF Renderer

JavaFX on Mac, iOS, Android, Windows Phone… Future looking bright

At the recent JavaONE conference in San Francisco, Oracle demonstrated JavaFX running on iOS and Android. While this hasn’t been officially approved by Apple, the proof of concept is also hope for the future of Java. I am heartened by Oracle’s new investment into Java and it looks like the coming years will be exciting ones for Java developers.

Update: It appears that Apple has loosened its restrictions on the technologies that can be used in their iOS app store. Therefore it seems very unlikely that Apple would try to block the use of JavaFX/Java applications in the iOS app store. Even more exciting!!

In addition to these demos, Oracle also announced the release of a JavaFX beta for Mac, and the developer of preview of JDK 7 for Mac – which promises a very bright future for Mac Java development.

Adobe CQ5 Developer Training

I just spent the past week in a developer training course for Adobe Communiqué 5.4 – a content management system on steroids. I thought I’d jot down some of my thoughts while they’re fresh in my mind.

CQ5 is a Java based CMS that is built around the JCR-283 (Java Content Repository) spec which essentially defines a sophisticated object database that is indexed by Lucene for easy searching and cross-referencing of objects. CQ5’s JCR implementation is called CRX, but there is also an open source reference implementation named Apache Jackrabbit if you have an allergy to commercial software.

It is not entirely correct to call the JCR an object database as it isn’t used to store Java objects directly – but the fact that it defines a tree of nodes and that all content is stored and accessed in a hierarchical fashion makes its use very similar to that of an object database. As such, it is natural to draw comparisons with Zope and its object database, the ZODB.

JCR vs ZODB

Zope, a python-based application framework, is radically different than the traditional relationship database model of web application development. The ability to store Python objects directly in the database and have them indexed solved many development problems, but it also created a few problems that would make maintenance of an ever-changing web application more difficult. Namely:

  1. When you make changes to a class, it can break all of the existing objects of that class in the database (you need to run a migration).
  2. If you try to load an object whose class definition can’t be found the system barfs.

This problem of class versions, managing upgrades of content types etc.. , was the single biggest problem with devleoping on Zope – and while I’m sure that there are best practices to work around this problem, I believe that the JCR solution of of storing content nodes but not actual objects is a much cleaner way of handling content.

The JCR stores a tree of content nodes, each of which have properties and their own child nodes. These structures translate well to different formats like XML (so you can dump entire branches of the repository as XML) and JSON – not so with a pure object database like the ZODB whose structures can be far more complex and include dependencies to classes. Data in the JCR can always be browsed independent of the component libraries which may be loaded into the system. You can browse the repository using WebDAV, the web-based content explorer that is built into CRX (the JCR implementation that is packaged with CQ5), or using CRXDE (the Eclipse-based development environment that is freely available to developers).

You can still define custom node types for your repository but this would merely dictate the name of the node type and perhaps which properties are required.

So, at first glance, this seems like a very stable base upon which to build web applications.

The Stack

The CQ5 stack looks like this:

  • WCM – The web content management layer consisting of a bunch of flashy UI components built using the ExtJS javascript library. (this part is proprietary).
  • Sling – HTTP server that makes it easy to read and write from the repository using HTTP requests. Very slick (this part is open source).
  • CRX – The content repository itself. Handles all permissions, storage, replication, etc… This part is proprietary. It performs the same function as Apache Jackrabbit, but includes a number of enterprise level improvements including a more powerful security model (I am told).

Author & Publish Deployment Instances

The recommended deployment is to have separate author and publish environments each running their own stack, and use the built-in replication feature to propagate authors’ changes to the publish instance whenever a piece of content is activated. This functionality, luckily, has been streamlined to hide most of the complexity. Workflow is built-in to allow you to activate each piece of content individually. Activation automatically triggers replication to the publish instance(s). This model seems to be very well suited to websites with few authors and many public viewers. It is scalable also, as you can add as many publish instances as you want to share the load.

This standard flow control (replicating changes from the author instance to the publish instances) leads me to wonder about cases where you do want the public to be able to interact with your site (e.g. through comments). We didn’t get into this scenario very much in the training, but, as I understand it, any content posted to the publish instance will go into an “outbox” for that instance that will be replicated to the author instances and await approval. They will then be re-replicated back to the publish instances once approved.

Security Model

The security model is quite different than that of most systems. Rather than having security attached to content types (because there are no content types) like with a relational database, or defining a large set of permissions corresponding to each possible action in the system as Zope does, security is 100% attached to the nodes themselves. Each node in the JCR includes an ACL (access control list) which maps only a small set of permissions to each user. There are only a few possible permissions that can be assigned or denied on each node. Basically it boils down to permission to read, write, delete, create new, set permissions, and get permissions on a node level. If there are no permissions assigned to a user on a particular node, then it will use permissions from the node’s parent.

One implication of this security model is that you must pay attention to the content hierarchy when developing applications. You cannot treat this like a relational database!

This is important. I suspect that many developers coming from a relational database background will be tempted to try merge the best of both worlds and try to create pseudo-content types in the system. After-all, all properties in the JCR are indexed, so you could easily just add a property called ‘contentType’ to your nodes to identify them as a particular content type, then build functionality that allows users to add instances of this content type. You could then create view templates that aggregate these content types to treat them as a table. You could do this, but you must be aware that you don’t have the same level of control that you have in a relational database system over what a user can do with your content types.

If you are querying the repository solely based on a property on a node – and not based on the path, then you may be surprised by the results that you obtain. At the very least, the JCR security model, despite appearing to be simple, is actually far more difficult to implement than its relational cousin – when trying to imitate the functionality of a relational database. You cannot control what properties are added to every node in the repository so querying based on property values may produce undesirable results. Instead you have to fully embrace the hierarchical model of data step very carefully when you try to import concepts from other paradigms as they could cause you to inadvertently introduce holes.

Custom Content Types (Sort of)

While CQ doesn’t have custom content types, it does allow you to map content nodes to a set of rendering scripts which produces something very much likc a content type. By setting the “sling:resourceType” property on a node to the path to a “component” that you develop, you can dictate where CQ looks for scripts that are used to render the node when requests are made. Components can be either “page” components, which represent an entire page, or regular components, which are included inside a page.

You can register page components to show up in the list of types of pages that can be added by authors when they add a new page to the system. Similarly you can register your regular components to show up in the “sidekick” (i.e. component palette) for authors when they are editing a page, so that it can be dragged onto a page. You can define which types of components are allowed to be parents or children of other components, and you can define which parts of site are allowed to have a particular component types added.

The Component Hierarchy

You can also define a “resourceSuperType” for components to allow them to inherit from other components in the system. This is handy for code reuse as there are hundreds or thousands of existing components that can be overridden or extended. We ran through several exercises creating and extending components. I’m satisfied that this process is not difficult and quite powerful.

Component Dialogs

A component without a dialog is really a lame duck. Users (especially authors) need to be able to interact with your components. E.g. if you create a photo album component, you need to allow your user to add photos to it. Adding dialogs is not difficult but I suspect that the development process is slated for improvements and more automation for future releases. The dialog forms are created entirely by creating appropriately named subtrees under your component’s node. E.g. you would create a child node of a particular type named “dialog”, which contains a child node named “items”, which contains a subnode named “tabs”, etc… 6 or 7 layers deep.

Each tab, each widget, each panel, is represented by a node in the repository. This is clever but somewhat tedious. It is like building a UI using only the UI hierarchy tree in the left panel of the IDE without the visual editor. I suspect that future versions will probably include a proper WYSIWYG UI editor for developing these dialogs but for now this manual system will have to do.

Despite the tediousness of the process, in the scheme of things it is still quite efficient. In only a few minutes you can produce a multi-tab, multi-field UI with rich widgets that allows your users to add and edit a myriad of content types on your site.

Why do you need an app for that?

I have recently stopped using the Facebook and Mail apps for my iPad and iPhone in favour of the HTML equivalents offered through Safari. The reasons: Mobile-optimized web applications are now as good, or even better in many cases, as their native app equivalents. In the case of the Mail app, I have found that the gmail mobile version has a superior search feature and is much faster in loading my messages. For facebook, I just found the app limited and more buggy than the HTML equivalent.

In general, I’ve come to the conclusion that if you’re going to build a native app for something, you’d better have a good reason – and good reasons are becoming fewer as HTML browsers improve their support for HTML5. The only valid reason at this point is a requirement for significant client-side processing – e.g. a 3-D game. But as WebGL matures even this will be quite possible in HTML.

So here are some reasons for developing your mobile applications in HTML5:

  1. Increased productivity in most cases over development of native apps.
  2. Real standards-based, cross-platform support so you can write it once and have it work on all major platforms (Android, iPhone, Blackberry, etc…).
  3. True freedom of deployment. You are not locked into Apple’s (or the next would-be gatekeeper’s) store or subject to their sovereign choice of what can and cannot be installed.

The remaining reasons why you may still need to develop a native app:

  1. You want to use a platform specific feature that isn’t available through the web api. E.g. the accelerometer, or the contacts list, or push notifications, GPS, etc..
  2. You need more client-side processing power than you can harness through the web.

The compelling reasons to want to develop a native app:

  1. To get it into the app store and maybe make some money.
  2. Toolkits (e.g. XCode/Interface builder) are developed and promoted specifically for making apps target a specific platform. This can make it seem easier to break into the market since there are lots of learning resources on how to use these tools.

The biggest challenge right now facing HTML mobile developers is that the tools are less promoted and more scattered. This is because the major stakeholders (e.g. Apple) have a significant interest in getting you to develop your app natively so that it will be exclusively available for their platform, and they will get a cut of any revenue through their store model. If you look, however, there are tools out there for building slick HTML mobile apps that look and feel very much like native apps. Some examples include:

  1. Sencha Touch
  2. jqTouch

And there are more where those came from. If you still want to develop a native app and you would prefer to work with open technologies like HTML and Javascript you may want to look at Appcelerator which provides tools to develop applications in Javascript, HTML, and CSS that can be compiled into native apps all the major smart phones (e.g. Android, iOS).

.NET: The most unlikely savior of Desktop Java on the Mac

When Microsoft first introduced the .NET platform and its flagship programming language C#, I, like many Java developers, looked at it and said “They’re just copying Java”. Why would I want to develop on the .NET platform which is targeted exclusively at Windows, when I could use the more established Java language and deploy across all platforms including Mac and Linux?

When I heard rumblings of the open source version of .NET, called Mono being developed, I was curious but ultimately wrote it off as it was likely that, since .NET was married to Windows in so many ways, probably most of the libraries for .NET would be platform specific and wouldn’t run properly on Mono anyways.

The past 10 years have seen many new trends come and go, and quite a few shifts in mindshare between the different technologies. One stream of trends that I have been quite interested in as a Java developer and a Mac user, is the role of Java on the Mac.

When Apple first unveiled OS X, Java was at the center of it. They promoted Mac as an excellent platform for Java developers to deploy their programs. They pledged to provide a Mac-specific implementation of Java that would blend into the OS and work seamlessly with the slick new look and feel of Mac. Two key pieces of this puzzle were the Swing Aqua look and feel and the Java Cocoa Bindings. The Swing Aqua look and feel allowed all Swing (the default java UI toolkit) programs to look like native programs on the mac. The Java Cocoa bindings allowed even deeper integration by allowing Java programs to use the native Objective-C classes and widgets directly.

Fast forward to 2011. If you try a google search for Desktop Java on the Mac or related terms you’ll notice that there are lots of articles, tutorials, and documents from the period ranging from 1997 to 2005. But very little after that. This is due to a number of trends and developments during that timespan. Some of these include:

  1. Apple deprecating the Cocoa Java bridge. (Turns out that is wasn’t used very much anyways because Java developers could achieve an almost native look and feel using Swing and keep cross-platform compability).
  2. Java mindshare had moved predominantly to server-side technologies.
  3. The emergence of higher-productivity interpreted dynamic languages like Ruby and Python had stolen a lot of mindshare on the desktop.
  4. Objective-C through the introduction of the iPhone and iPod had drastically increased in mindshare – so more developers were familiar with the native tools and would forego trying to work with a language such as java for native applications.
  5. Sun seemed to be confused as to which direction it wanted to go – and years of progress were lost (* more on this later).

Nonetheless, during the period of 2005 to present, there have still been some good options for producing high quality desktop applications for the mac. After all it was the only OS with a large user base that shipped with Java. This meant that you could develop for Java and be certain that the target users would be able to use your software without having to download anything extra. Swing still has the Aqua look and feel, and all of those tools and widgets that were developed pre-2005 still worked nicely (except of course those tools that were built upon the Cocoa-Java bridge).

Unfortunately the writing was on the wall and Apple made it official in October 2010 when it announced that it would be deprecating Java on the mac and that future versions of the operating system would not ship with it. It would be up to the open source community and Java’s owner Oracle to provide a Java for the future of the Mac (and this future is still very much unfolding as I type).

So now, at a time when the future of Java on the Mac is as bleak as ever, an unlikely ally enters the fray: .NET – or rather its open source twin, Mono.

Mono has quietly been picking up a following over the past 10 years. It reached 1.0 status in 2004, and has facilitated the development of 2 separate projects, which, together appear to offer the best hope for the future of Java on the Mac:

  1. IKVM.NET – A Java virtual machine that runs on .NET and Mono. This tool is able to run Java byte code in Mono and use java libraries natively. It also includes tools to statically compile java as a .NET executable which can be used in .NET or Mono applications. This has opened many doors to both C# and Java allowing libraries developed with java to be quickly compiled a distributed for .NET (e.g. Apache’s PDFBox which is developed in Java but available for both .NET and Java).

  2. The Mono Mac project – An objective C binding that allows C# code to directly access the Mac Cocoa classes. The current versions of MonoDevelop (the Open Source Mono IDE) work seamlessly with Apple’s developer tools, especially interface builder so that developing and deploying an application for the Mac using C# is a first class experience.

These two projects together open up a myriad of possibilities for Java on the mac that haven’t been available since the deprecation of the Cocoa-Java bridge. If you have a large existing source base of java code and libraries, you can quite easily now compile them into a Mono library that can be used in a Mono Mac application – and then deploy it natively on the Mac and even distribute your applications in the Mac App store.

And this is how a Windows-Only competitor of Java has evolved into an unlikely ally in staying relevent on the Mac platform.

How to get a Java Application into the Mac App Store

I just received a reply from Apple support confirming that there is nothing in their guidelines that prohibits apps that include an embedded Java virtual machine. Therefore, Apple’s regrettable decision to deprecate Java and disallow apps that depend upon it from entry into their new App store is a mere speed bump and not the road block that I had originally believed it to be. So there are a number of strategies for getting Java into the Mac app store: 1. Embed a java runtime environment as part of the application bundle. Currently there are no Mac JVMs that can be distributed which support Swing with the Quartz UI (i.e. graphical Swing apps won’t look native, if you can even get them to work). However Oracle and Apple’s recent announcement that Apple will be donating source code to the OpenJDK project suggests that this will be resolved with Oracle’s release of JDK7 on the Mac. In the mean time it would be possible to embed third party JVM like Soy Latte or IKVM (a .Net implementation of java) and use Rococoa as the UI. 2.** Use GCJ (The GNU Java Compiler)** which can be installed via Mac Ports, in conjunction with Rococoa for the UI to compile your application into a native binary.

Resources

  • Mono for Mac Resource: This page shows some resources related to Mono support on the Mac. Why is this relevant? Because Mono is the open source, cross-platform implementation of the .Net framework and it supports IKVM, a fully-functional java virtual machine that runs on Mono. IKVM will allow you to convert your java applications into .exe files that will run on Mono, which can be embedded in your application bundle.
  • Example using Rococoa to use Quartz framework from java
  • Rococoa GUI Demo : This tutorial shows a short sample of how to use Interface builder to build an interface for your java application using Rococoa.

The Chrome App Store Swinging the Pendulum WAY open

I’ve been very critical of Apple’s recent moves in deprecating java and excluding java-based applications from their new Mac App store. Their restrictive policies seem to be strong-arming developers into using their closed set of tools (and allowing apple to take 30% of all sales). So when I heard about Google’s new Chrome App store, I was naturally excited about the promise of a more open environment where apps are not shunned merely because they were built with a non-approved tool.

Unfortunately, it seems that they have gone a little too far in the opposite direction of Apple. The Chrome App store is meant to be a marketplace for web-based applications (i.e. anything that runs in a web browser). It allows users to install web applications so that they are added to their dashboard and can be easily accessed. Google provides tools for developers to handle purchases through the app store and keep track of user’s licenses. Unlike Apple’s closed model where only Cocoa-built apps need apply, the Chrome store invites all technologies that run in a web browser including HTML5, Flash, Silverlight, and Java (applets and webstart apps). I applaud Google for taking this step and for keeping the market open to such a wide array of technologies, but the current state of store is a little disorienting because they don’t seem to be very strict about the applications integrating into the app store frameworks. Many of the “applications” are just glorified web pages that can’t really be called an “app”. Others that could be classified as apps aren’t integrated at all with the app store other than the initial link – so that when you click on the icon, you are just taken to the app’s homepage that contains all kinds of advertising and gives an option to register for an account or login. When you’re clicking on an “app” icon to open an application, you expect to be taken directly into some sort of app interface…. you don’t expect to have to fill out a registration form or log in every time you visit.

My suggestion to Google would be to become a little more discerning about what kinds of apps get accepted into their store. They should start with a set of rules about one quarter the size of the Apple App store submission guidelines to ensure that the user experience is as close as possible to using a desktop application. If users have to sift through too much garbage, they’ll just give up and the app store will become a glorified set of bookmarks.

Posthumously discovering JavaFX

It turns out that JavaFX is actually kind of cool. Sun had been pushing it to no avail for over 2 years before Oracle finally pulled the plug (sort of .. they’re killing the language and porting it to Java), but they seemed to do a lousy job of making it seem relevant.

Now that it is dead, I have gone back over the docs for a final look and have discovered that it actually looks kind of cool. Being able to lay out 3d objects in a scene graph, apply transformations and animations to any branch; binding variables to one another to keep them in sync. It actually makes for a nice system.

The problem is that I’m having trouble forming a complete picture of where I can use my JavaFX apps. They say it’s for all the screens of your life, but with no iPhone, no Android, limited to no Mac support, I’m not sure what I’m left with. I’ve seen the odd reference to Blackberry support, but nothing definitive yet.

A shame, really.

Apple’s assault on Java is an Assault Java Developers

Yesterday, Apple announced that its next version of OS X, called “Lion”, will include an App store to simplify the process of downloading, purchasing, and updating applications on the Mac.

This is great news!

Apple will take a 30% cut of all transactions performed through the app store.

I’m mixed about this. If the App store is able to increase the sales of my apps by more than 30%, then it might be worth it. Otherwise, this is just a 30% tax off the top of all of my app income.

In a much less hyped announcement, Apple has deprecated Java as of version 1.6 update 3, and may not continue to bundle Java with future releases of OS X.

This is bad news… and it could be terrible news if Oracle or someone else doesn’t pick up the slack and provide a Mac deployment for the Java VM. Although, this could be somewhat good news if the “new” steward takes better care, as Java on the Mac does lag behind other platforms in both performance and stability. Oracle taking over Java could be a good thing if it results in a better product.

One final non-announcement came in the form of a guideline for admission to the Mac App store: Applications that use deprecated API’s (e.g. Java) will be rejected from the App store.

This is terrible news! A betrayal of the first order. It appears that Apple now feels confident enough with its market position to close the doors on all but their faithful Objective-C developers.

I currently have 3 desktop Mac applications on the market, and 2 of them are written almost entirely in Java. I chose, and still prefer, Java as a language because it provides a fast, robust, multi-threaded environment that is easy to scale, and even easier to port over to different platforms. It took me about 4 hours to port PDF OCR X over to Windows, and most of that time was spent on the non-java portions.

I was actually intending to blog about my experience with Java and multi-platform application development and was going to advocate Java as a good platform for development on the Desktop.

In light of yesterday’s announcements, I must take a pause to consider my future. Since Apple seems to be dictating its terms in a heavy-handed way, my instinct is to stick it to “the man” (Apple) and boycott its App store and its products altogether.

But then I lean back and glance around the room at my iPad, iPhone, Apple TV, Mac Mini, and 2 Mac books…. and I question my resolve as they do indeed make nice products.

I watched from afar when Apple shut out Flash from its iPhone. But this didn’t affect me as I didn’t develop for the iPhone. And, after all, Flash runs down the battery life too fast so there was a logical technical reason for its omission. When they brazenly placed a prohibition on all advertisements served by competitors to their iAd service, I shook my head – but it didn’t really affect me. And since the iPhone was a new device, the fact that it was closed wasn’t too much of a concern, because we only had non-existence to compare to its “closed existence”, and a closed existence surely is better than non-existence

But with the Mac we are moving from an era of openness, and stepping into one that would appear to be much more closed, so I get the distinct and unpleasant feeling of having my virtual civil rights stripped away. For now, we will probably be able to download and install applications the conventional way, but such applications will likely be treated as 2nd class citizens, branded with a star, and forced to live in some hard-to-find ghetto folder. Once Apple has achieved wide-spread acceptance amongst users that the App store is the only acceptable way to obtain software, perhaps they’ll close the technical loop hole that allows those second class apps from even breathing the same air as Apple’s preferred class of fully-taxed App store apps, and we’ll be left with a completely closed, yet beautiful utopia of an operating system housing only the master race of applications that descend from Apple’s own genetic components (i.e. Objective-C) and bear the 30%-tax stamp of Approval.

Mister Jobs, you have stepped too far. I was a true believer. I am now agnostic.