Category Archives: Javascript

JavaOne: A Look Back

I am now home from my Bay area adventure at Java One. This was the first conference I’ve attended since 2005, and is by far the largest conference I’ve ever been a part of. I was blown away by the scale of this one. The downtown core was plastered with Oracle and Java signs. Buses dipped in oracle insignias roamed the streets constantly, and you couldn’t walk a block without seeing at least 5 people brandishing Oracle swag. (I quite liked the back pack that they provided).

In addition to the sessions (which I’ll get to next), there was a steady stream of entertainment events for the benefit of conference attendees. Union square (and a few other locations) were blocked off for music performances; and then there was the “fan appreciation” event featuring Pearl Jam and Kings of Leon. Doesn’t sound like nerd conference, does it?

The Sessions

My reason for attending was the sessions. From Sunday to Thursday my schedule was jam packed with talks about topics ranging from JavaFX to MongoDB and a smattering of Java ME, iOS, and Android development. For any given one hour slot, I had to choose between 2 or 3 talks that I really wanted to see, and another 8 or 9 that would have been interesting. Being able to hear about new technologies directly from the people who are behind it is awesome. It imparts a kind of understanding that cannot be achieved through a book or reading blog posts.

This conference has sparked a fire in me to seek out more conferences like this, similar to the way that a my first exposure to Disneyland led me to look for other parks that offer the same experience. In fact, the way a child reacts to his first experience at Disneyland is an accurate way to describe my reaction to the conference schedule.

The Topics

Currently I have a number of desktop applications that have been developed in Swing and have been deployed as applications on OS X. Therefore, I was most interested in learning about Swing’s successor, JavaFX, and how to deploy applications on OS X. The conference was heavy on JavaFX (yay!) but a little light on OS X. There were two talks on OS X deployment, but sadly I could only attend the first one (which way a fantastic talk, by Scott Kovatch). Nonetheless, I have left the conference with stockpiles of new ammunition for building applications with JavaFX. In fact, I’m really exciting about using JavaFX and Scene Builder (the GUI editor for JavaFX) to start building some business applications at work.

The demos provided by some of the community served as proof of concept to me that JavaFX is something special. E.g. Gerrit Grunwald’s (@hansolo_) set of gauges in the JFXtras project show what can be achieved with a little time and artistic flair.

On the OS X front, it looks like we’re all set to go now with the Mac App store. The javafxpackager tool provides the ability to easily deploy applications as native bundles. This is a new direction for Java (up until now, they have been recommending Web Start distribution), and I think it is a good one.

Another topic that I am keenly interested in is mobile development (iOS and Android). Last year’s java one included a demonstration of JavaFX running on an iPad, but then nothing more was heard. This year they announced that JavaFX will be available for Java7 SE Embedded, and they had a conference scheduler application running on an small, embedded device in a kiosk at various stations in the conference. But there were no announcements about JavaFX for iPad or Android. The announcement of its availability on embedded devices is a step in the right direction, though, as this is a prerequisite for running on the major embedded platforms.

I did attend one talk (actually 2 talks on the same topic) on Java for iOS and Android. The CodenameOne project (that I’ve blogged about before) is a java library and framework for building cross-platform mobile apps (i.e. deployable on Android, iOS, Blackberry, and J2ME). This looks very promising.

When I had reached my saturation point on JavaFX, I wandered across to the Parc55 building for a couple of Java Enterprise Edition talks. I attended a talk on the new Project Avatar, which provides a thin-server approach to web applications (i.e. most view logic and layout is handled on the client in javascript, and the server just sends over the data). This project looks quite interesting, both in concept and execution. The concept coincides with much of the more recent development I’ve been doing with Xataface.

The last talk of the conference that I attended was on NoSQL and scaling strategies. There, Nike’s director of engineering discussed some of their architectures for handling heavy traffic with NoSQL databases and Data grids.

My head is full. My notebook is full. Now to put all of this new knowledge into practice.

Ingredients required to port HTML/Javascript App to JavaFX

I have been taking in as many JavaFX talks during Java One as I can. I’m keen to move some of my development from LAMP/HTML5 to LAMP/JavaFX (Yes.. still not ready to move into the JavaEE space – though PLAY makes it compelling).

The Scene Builder application, complete with its powerful CSS support, looks like it will make the UI portion of app development quite easy. In fact, I suspect it may even improve my productivity over my existing HTML development.

The one piece that I’m concerned about is the HTTP client for consuming the LAMP web service. In Javascript you get quite a bit for free, but the two biggest are:

  1. Sessions/Auth are automatically retained.
  2. AJAX calls and conversion to/from JSON is automatic.

To replicate the functionality in a JavaFX app, I suppose I need to embed an HTTP Client object of some kind to maintain all of the session information and allow me to load JSON data sets from the server in a simple manner. At the “JavaFX for Business App Developers” talk, the speaker used Jersey in is example application to consume an XML web service. The Jersey website makes it sound like it can also be used to consume JSON web services. Will have to give it a try.

Another option, which I’ve yet to try, is to actually embed a Web Engine into the app for dealing with AJAX calls. This is either a very good idea, or a very bad one. I would be concerned about such things as performance, and security restrictions that may exist inside the web box.

I’ll post again with my findings.

PLAY Framework Observations

Going into James Ward’s talk on the PLAY framework I had no hands-on experience with the framework. I only knew of it from the odd mention in other developers’ Facebook updates.

Highlights:

  1. A light-weight web application framework with its own integrated web server (Netty).
  2. Stateless. This is a key principle of PLAY that helps encourage scalability.
  3. Encourages non-blocking asynchronous request handling to further enable scalability (e.g. it can handle more than one request per thread).
  4. Applications can be written in either Scala or Java.
  5. Nice MVC separation (Views, Models, Controllers).
  6. Uses Ebean and JPA for ORM.
  7. Integrated test framework.
  8. Asset compiler (to minify javascript automatically etc…).

The framework reminds me very much of Ruby on Rails, but the fact that it runs on a JVM makes me far more likely to try it.

I particularly like the light-weighted-ness of PLAY. It really embraces the HTTP protocol, not trying to hide it from the developer. This actually gives me more trust as a developer who would be just diving into a new system. Some of the heavier Java frameworks hide too much, resulting in extra time spent trying to figure out how to adjust the long levers of the framework to achieve something as simple as an HTTP redirect.

Ward made note that the stateless aspect of PLAY is important. He said that if you require state (as most applications do), you should relegate this to a store like memcached that can be scaled across multiple instances. This makes a lot of sense. I frequently reach the edge of what can be achieved on a single web server. If you depend too much on “state”, it makes it terribly difficult to introduce a 2nd server effectively. Moving to a “stateless” paradigm enables you to fully leverage the HTTP protocol, introducing caching and proxies effectively at many different positions in the request chain.

The Metro UI

I attended a talk yesterday on “Java FX meets Metro” by David Qiao. Going in, I was hoping for some information about deploying JavaFX on Windows 8 in some format that is acceptable by the new windows 8 app store. In fact, this talk was not about that at all. Rather, it discussed strategies for creating a Metro-esque user interface using Java FX and didn’t talk about deployment at all. Even so, I got quite a bit useful information from the talk.

Some highlights:

  1. Metro is a set of guidelines for the development of user-interfaces that are optimized for touch-based interfaces.
  2. Many user interface paradigms that worked well with desktop applications, don’t work so well for touch devices (e.g. tree navigation, breadcrumb controls).
  3. Java FX makes it relatively simple to design a Metro-esque UI with its animation and CSS support.

There are basically two kinds of metro-style apps:

  1. Flat – e.g. just one view, maybe some tabs.
  2. Hierarchical – Multiple levels.

Hierarchical apps should contain only 3 levels:
1. The hub page. Like the homepage with a view of all of the content – or at least a good summary.
2. A Section page. This would be a list of individual items for the user to select.
3. A Detail page. This shows a single item of the application.

Metro also introduces the notion of a semantic zoom to be able to move to a more or less detailed view of the information. E.g. in a section page you can “zoom” in on an item to show its sub-items. Similarly you can zoom out of the sub-items to see a more succinct view of the items.

Just as desktop applications of some common components (e.g. menu bars, windows, etc..), Metro apps have their own common components. These include:

  1. A tile (like an icon, but usually presented as a flat rectangle).
  2. Tile pane (a tile that contains a bit more content).
  3. Nav Bar – A menu usually at the top or bottom of the UI providing transient access to other areas of the app.
  4. App Bar – A more contextual menu providing actions that the user can perform in the current context.

One key design principle that is a departure from traditional UIs is that you almost never see vertical scrolling. Scrolling is almost always horizontal. This, intuitively seems more natural to the user (like turning the page of a book) when on a touch interface like an iPad.

I’m keen to try to adopt some of these principles to my own apps. Especially html5 apps. It should be relatively easy to build a metro-esque application using HTML and CSS. The rewards could be great to the user as it really seems like Microsoft has gotten a lot right with these principles.

Now to see what I can find in the way of existing Metro HTML5 stylesheets and Javascript/jQuery libraries.

A Bug Hunter’s Diary: A Guided Tour Through the Wilds of Sofware Security

A Bug Hunter’s Diary is written as a journal of a “Bug Hunter”. It takes us through seven bugs that the author (Tobias Klein) has discovered in the past five years, giving the reader a glimpse of his process and an awareness of how easily bugs can creep into even the most carefully written software.

All of the bugs discussed in this book are related to memory errors that lead to a proofs of exploit that involve overwriting the program counter (and thus hijack subsequent program execution with arbitrary code). This narrow focus sets the stage for a book that is very heavy on Assembly code, debuggers, and a few easily compromisable C functions.

Some of the bugs involve open source software so we have source code to step through, but others provide nothing more than the finished binary to step through with a debugger and a disassembly of the machine code. For someone who doesn’t do much C programming anymore and hasn’t done Assembly in over 10 years, such as myself, the journey can be quite daunting. The guide on this journey stops occasionally to explain his intuition for certain decisions, but we’re largely left to just follow him along without really knowing how he knew to check for certain things and not others. After following him on 7 missions, however, some patterns do become evident. This repetition enables the reader to feel like more of a partner in the later journeys to contrast his bystander role in the first few.

Being primarily a Java, PHP, and Javascript developer of late, the tools and languages used in this book (Assembly and Debuggers) really stretched my ability to follow along in real-time. It reminded me a little of the first few computer programming books I picked up as a newby when I would be reading pages and pages of code and waiting (hoping) for the eureka moment when all the code would make sense. Like listening to an old song from my past, this feeling of having to follow the master and just have faith that things would make sense if I just stared a little longer, opened a flood of memories of my first days in programming. I’m thankful to the author for that.

If you can think of my train of thought during the reading of this book as a process, then this process had a few different threads running in parallel. In one thread, I was just trying to follow the Bug Hunter and understand the lessons as they were presented. In another thread I was trying to identify analogs in these lessons to my own domain of programming (e.g. web programming, and database programming). There were also a thread that was periodically assessing the author’s style and the the book’s relevance, to various target audiences, and its importance to software.

Thread 1: Just trying to keep up

If I had read this book as a tutorial where I actually do as the author says, it would have taken me a lot longer to read. At a certain point I just settled for getting the gist of what he was doing and focused more mental cycles to Thread 2.

Thread 2: Applying the Lessons to My Domain

The process that Klein uses to track down bugs serves as a forceful reminder of how bugs could slip into my own software if I’m not careful. In trying to exploit memory errors, Klein would focus on the interfaces between user space and kernel space as logically any exploit must pass through one of these bridges at some point. There is an analog in any programming environment. In web development you are looking for the entry points where user input is passed to the application. This book is a reminder that you must always be conscious of where data could possibly have come from before working with it as it is often a combination of small glitches that result in a large exploit down the road.

Thread 3: The Author’s Style

Klein writes much of this book as a sort of diary of his actions. As such it is more like a day in the life of a bug hunter than it is a tutorial on how to become a bug hunter. This style works well for imparting an authentic view of what a bug hunter really does – and as such might serve as inspiration for those who want to follow this craft.

At times (more early on than later) I struggled to grapple with the direction Klein was taking us. Is he just reciting his exploits as a way of bragging, or is he recounting his steps and building up to an eventual purpose. Throughout the second and third bugs I was bogged down a little with thoughts like “this is way too dense in Assembly for the average person”, but these would be countered with rebuttle thoughts like “but this isn’t intended for the average person…”. As I settled in, though, these thoughts were replaced by less critical internal commentary like “wow, it would be really easy to make that mistake” or “that’s really neat!”

By the time I reached the sixth and seventh bugs and I had seen the same types of bugs appear over and over, I had become a true believer. Not only was this book interesting; It was essential! Despite the lip service that is paid to “good programming practice” in computing science programs, I have never been made so acutely aware of some of the simple programming mistakes that can result in dreadful consequences. As I finished the seventh bug, the one clear capstone thought was that this book should be required reading to anyone who wants to write any software. Just as engineers are required to take an ethics course, computer scientists should be required to take a “Bug Hunters” course – if for no other reason than to raise their awareness of the nature of bugs.

When I think about the “buggy” code that I have read from new University grads it makes me shiver. Much of this buggy code is in production around the world and quite often the develop has no idea that the bugs are there. This book highlights a few specific functions to handle with great care in C, but every language has its own set of traps that all new programmers fall into. (E.g. in PHP registering global variables, unvalidated input to SQL queries leading to SQL injection, etc…). Computer books very rarely spend much time highlighting these traps, and as a result young programmers regularly fall into them. Unfortunately with software these traps often don’t give immediate feedback. Instead they linger for years before they are exploited – and sometimes to great disaster.

Summary

I meandered a bit in this review, but to summarize. If you can slog through the dense debugger readouts and assembly code and latch onto the core message in this book you will become a better software developer. Even if you don’t learn anything new in this book, it will serve to raise your awareness and prevent a bug.

And one more thing: This book should be required reading for new software developers.

Javascript Development with the Xataface Javascript Compiler

Javascript is a wonderful language, but I always felt as if it wasn’t terribly scalable. Not so much in the performance sense, but in the component sense. When working on large projects (like Xataface) Javascript can seem a little too “scripty” to build components in. I’m always fighting between the two conflicting goals of library size and library functionality.

If you build a useful library in Javascript it is usually either comprehensive and large, or small and single-purpose. It is possible to build the library in pieces and just include the parts that you want, but then you end up with a cumbersome set of script includes in your HTML page every time you want to use your library’s functionality. This is error-prone as it is easy to forget which scripts you need to load for any particular function to work.

In server-side languages you don’t have this problem. When you write a script in C, PHP, Python, Java (or any other language), you can just include your dependencies at the top of your file using an import, include, or require statement (syntax depends on the language). Then using your library is as simple as including one file – and all of that file’s dependencies will automatically be imported as well.

To solve this problem, I developed a Javascript compiler as part of Xataface. Some features of this compiler include:

  • Compiler directives to process dependencies.
  • Javascript Compression – to minimize the footprint.
  • Duplicate Dependency tracking (i.e. If multiple scripts all require the same script as a dependency, then the dependency will only be included once).
  • Compiler directives to include CSS files.
  • Compiler directives to import files as strings.
  • Configurable source paths. (i.e. Just like with other languages, you can specify multiple paths where source files will be located).

This doesn’t sound like a lot, but these features really improve the Javascript development experience. It allows me to develop reusable components that can be shared and imported into many different projects without having to deal with the complexities of including script tags in an HTML page, or making sure that the proper CSS files are included. Once you have built a component, it is done. All you need to do in order to use it in your application, is include it using the //require directive.

Let’s look at an example…

All of these features are provided by the Dataface_JavscriptTool class in Xataface. So you begin by obtaining an instance of this, and adding the source path where you want it to look for Javascripts.

import('Dataface/JavascriptTool.php');
$jt = Dataface_JavascriptTool::getInstance();
$jt->addPath('/path/to/javascripts', 'http://www.example.com/url/to/javascripts');

At this point the compiler knows to look in the /path/to/javascripts directory for scripts. But you haven’t told it to include any particular script. That’s our next step:

$jt->import('myscript.js');

If you were write this code inside the context of a Xataface application, your myscript.js file would run because Xataface handles the rest of the rendering at the end of the body of any template extending the Dataface_Main_Template.html template (which is pretty much all templates).

Note that by default the following directories are included in your javascript paths:
– DATAFACE_SITE_PATH/js
– DATAFACE_PATH/modules/XataJax/js
– DATAFACE_PATH/js

So you are able to use any script in those directories without needing to call the addPath() method of the JavscriptTool.

A look at the Javascript

myscript.js:

alert('hello world');

This just pops up an alert. Let’s get to some best practices right away though. The general structure of a script usually goes as follows:

//require <myotherscript.js>
(function(){
    alert('hello world');
})();

Note the //require statement on the first line. This tells the compiler to include the script myotherscript.js in its place. The compiler will look first in the /path/to/javascripts directory, then in DATAFACE_SITE_URL/js, then in DATAFACE_URL/modules/XataJax/js, then in DATAFACE_URL/js to see if it can find a script by that name. If it cannot find one it will throw an exception with a stacktrace. If it finds it, it will include the contents of the script in place of the require statement.

Example: Requiring jQuery

Xataface comes with lots of libraries included in its source paths. jQuery is one that is used in almost every script:

//require <jquery.packed.js>
(function(){
    var $ = jQuery;
    alert('Hello world');
   
})();

Note that on the first line, we make a reference $ to jQuery. This is because in Xataface jQuery.noConflict() is set so that it doesn’t conflict with other libraries that want to use the $ variable. By declaring it inside our wrapping function(){}, we are able to still use the $ shorthand without polluting the global namespace.

Using jQuery UI

The following example is meant to show why we need to be able to bind CSS with your javascript libraries. We are going to use jQuery UI’s dialog to display our ‘Hello world’ message, and this requires a CSS file in order to look proper:

//require <jquery.packed.js>
//require <jquery-ui.min.js>
//require-css <jquery-ui/jquery-ui.css>
(function(){
    var $ = jQuery;
    var dlg = $('<div>').append('Hello World').appendTo('body');
    dlg.dialog();
})();

These examples are just the tip of the iceberg of possibilities that a Javascript compiler brings. More cool examples will be forthcoming as Xataface 2.0’s release nears.

Using a “Silent” Java Applet to Render PDFs in HTML

I have read many obituaries for the Java applet. Once upon a time, in the late 1990’s the applet was king of the internet and hope for the future of rich web-based applications. Its decline and (almost) demise can be attributed to a number of factors, not the least of which is the fact that HTML and Javascript have improved to the point where most of the the functions that used to require an applet can now be performed directly in the web browser without requiring any third-party plugins. Nowadays, you scarcely run across any websites that use applets, and when you do, they generally stick out like a sore thumb. They are generally slow to load, and they almost always pop up with a security dialog asking you to approve their execution based on a signed certificate.

Signed Applets – Full Access or No Access

This clumsy security model is the other reason for the growing irrelevance of applets. An applet can only run in two different modes:

  1. Run inside the “sandbox”. The applet cannot access any system information from the client computer, cannot access the file system, and cannot make any network connections to any hosts other than where the applet was loaded from.
  2. Signed Applet – Full access. The applet runs with full permissions of the user. E.g. it can connect to any host, access any client system information, and access the hard drive. This level of access is very risky for the user especially if the applet is just running inside some random webpage that they are trying to open. The applet has the ability to delete the user’s hard drive or copy files from their hard drive without the user even knowing.

Because applet developers are so limited with the sandbox model, most developers end up distributing their applets as signed applets so that they can be run on clients’ computers with full access privileges. This works out great if the client fully trusts the applet developer – but that isn’t the typical client/developer relationship. Most users just click “okay” whenever these types of security dialogs pop up, so it becomes very easy for malicious developers to create trojans for unsuspecting users who browse to their web pages.

Since the proliferation of signed applets across the internet would result in a very dangerous browsing experience for the average internet user, we can be thankful that the web has opted for the applet-less direction that instead makes use of HTML and Javascript – which work inside a secure sandbox. Signed applets are now generally reserved for B2B applications and in-house apps where the client computer has a trusting relationship with the developer.

Unsigned Applets Are Still Safe

All that said, unsigned applets that run inside the applet sandbox are safe for a client to run since they don’t provide any access to the client’s computer. Unsigned applets’ fade from prominence is the natural result of improvements to Javascript and HTML. Javascript/HTML/CSS now have the tools required to create complex user interfaces with rich client interaction. It is no longer necessary to embed a Swing-based user interface inside the browser to give the user a rich experience. Further, it is inconvenient to work with a hybrid of technologies so the developers will often steer clear of applets entirely – opting for pure javascript solutions. For many years, applets were still necessary to accomplish things like drag-and-drop from the desktop (though that required a signed applet), or some more complex user interface widgets, but HTML 5 has introduced most of the tools to do all of these things inside Javascript natively. So the applet has been even further crowded out.

So What Are Applets Useful For Now?

The applet’s relevance may have been eroded down to a few grains of sand, but it turns out that some of those grains still shine like diamonds. Javascript can do many things well, but there are still a few items that Java can bring to the party. For example:

  1. Javascript doesn’t handle processor intensive operations well. It runs in a single thread so any intensive operations will make the browser hang and possibly even crash. Java, on the other hand, is multi-threaded so it can perform parallel processing in the background and pass results back to the browser in way that doesn’t interfere with the user experience.
  2. Java provides a rich, almost limitless set of libraries. – Hence you can find a library to do just about anything you need. The provides an alternative to making server-side requests for processing using AJAX. Instead requests can be passed to a “silent” applet that runs on the client computer – saving both network usage and server usage.

The “Silent” Applet

Observing the strengths of Java vs Javascript yields an interesting model for making use of applets in a web page. I call this the “silent” applet because it is designed to be completely transparent to both the Javascript developer, and the user. A silent applet is one that is loaded silently in the web page and does not manifest itself visually in the web page. Rather is runs invisibly along side the web page as a daemon waiting for requests from the web page which it processes and returns to the web page. This is similar to the AJAX model where the client sends background requests to the server, and the server returns a response which the client then processes. With this model, the request is not sent to the server at all. This saves both network and server resources since much more processing can take place on the client.

The silent applet is useful in situations where a significant amount of processing is required so that performing the “calculation” inside javascript is either impossible or would be disruptive to the user experience. This might include rendering a PDF page as an image, generating a complex chart, performing an image, video, or sound transformation, etc….

Example Silent Applet: Web PDF Renderer

I am developing a PDF reporting module for Xataface. I want the UI to allow the user to arrange database fields and content over top of an existing PDF that they upload as a means of creating report templates that can then be rendered as full reports later on. For this I need to be able to display the PDF as a background image in the user’s work space. I also need the user to be able to navigate through the PDF to different pages and to zoom in and out. The current state of Javascript and HTML is such that the rendering of PDFs needs to be relegated to a different system. Either I need to render the PDF pages as images on the server-side, or I need to find a way to do it in the client.

Rendering the PDFs on the server side would require me to install some server-side extensions or applications that can handle PDFs. There are many, but my goal is to keep the server requirements minimal so that it will work inside a standard LAMP install. So my preference is to find a way to render the PDF on the client side. There are various flash and Java applets already that display a PDF inside the browser, however I’m concerned with mixing flash or applets into the UI of my editor as it will likely result in painful, or intermittent conflicts down the road as the technologies choose not to place nicely together. So I decided to create a silent applet for rendering the PDF.

The Web PDF Renderer applet runs as a silent daemon in the background. There is a thin javascript API that can be used to ask the applet to render pages of a PDF at various sizes. When the processing is done, the applet calls a javascript callback to update an <img> tag in the user interface. This allows me to keep the entire user interface in standard HTML and manipulated via the DOM.

Using the Library

One example use of the library is to render the PDF to an HTML <img> tag. The following example creates a new PDFPage wrapper (which creates an img tag), then appends the image to the body of the document, and finally renders the PDF page. The render() method sends a signal to the applet to render the PDF page. When the applet is complete, it sends the data back the PDFPage object which updates the image source with the new data.

        // Short reference to PDFPage constructor
        var PDFPage = xataface.modules.pdfreports.PDFPage;
        
        // Create a new page  (first page of document)
        var page = new PDFPage({
            width: 800,
            url: 'test.pdf',
            page: 0     
        });
        
        // Append page’s <img < tag to the document body
        $(‘body’).append(page.el); 
        
        // Render the page  (done asynchronously)
        page.render();
    

Inside the Box: The implementation

We need to set up a mechanism for passing messages back and forth from Javascript to the applet and back. For javascript to java communication it is as easy as calling public methods defined on the applet directly. These are exposed and callable in Javascript. For the reverse, we make use JSObject which is available standard as part of the Java plugin. There is quite a bit of information on Java-Javascript communication on Oracle’s website.

We created a global queue that is used to pass the messages:

PDFPage.queue = [];

Note that is is just an empty array at this point.

Calling the render() method will pass a message to the applet – it essentially works as follows (this has been simplified for the example, but gets the point of strategy across):

function PDFPage_render(){
    
    PDFPage.queue.push(this);
    startDispatch();
        
}

So render a page involves just two things:
1. We push the PDFPage object onto the message queue so that the applet can access it.
2. We call startDispatch(). This function essentially tells the applet that the queue has been updated so it can start processing.

The startDispatch() function up-close:

function startDispatch(){
        var applet = $('applet[name="'+PDFPage.appletID+'"]').get(0);
        if ( !applet ){
             var attributes = {
                name: PDFPage.appletID,
                code:       "com.xataface.modules.pdfreports.PDFRendererApplet",
               codebase: PDFPage.codebase,
                archive:    "WebPDFRenderer.jar, commons-codec-1.5.jar, commons-logging-1.1.1.jar, icepdf-core.jar",
                width:      1,
                height:     1
            };
            var parameters = {
                startDispatch:"PDFPage.startDispatch()",      
                queue:"PDFPage.queue"
            }; 
            var version = "1.5"; 
            deployJava.runApplet(attributes, parameters, version);
            
            
        } else {
        
            try {
                applet.startDispatch();
            } catch (e){
                setTimeout(function(){
                    startDispatch();
                }, 1000);
            }
        }
    
    }

At its core this is just a wrapper around the applet’s startDispatch() method – but it needs to handle some edge cases to load the applet the first time it is called. If the applet hasn’t yet been added to the dom, it adds it and tries to call itself again. If the applet is there but not loaded yet, it waits 1 second and tries again.

The deployJava.run() method is from the standard Java deployment code available here. Notice that we pass 2 parameters to the applet when we load it:

  1. We pass the javascript path to the queue so that the applet knows where to find its message queue.
  2. We pass the startDispatch() method call so that the applet is able to call its own startDispatch() method through javascript.

Inside the applet:

The start() method just loads the parameters that we passed, then calls the startDispatch() method for the first time:

public void start(){
        
        queue = (JSObject)JSObject.getWindow(this).eval(this.getParameter("queue"));
        JSObject.getWindow(this).eval(this.getParameter("startDispatch"));
    }

The guts can be found in the startDispath() method:

public synchronized void startDispatch(){
        
        if ( running ) return;
        running = true;
        
        
        Thread dispatcher = new Thread(new Runnable(){

            public void run() {
                while (!stopDispatcher){
                    try {
                        JSObject next = null;
                        try {
                            next  = (JSObject)queue.call("shift", null);
                        } catch (Exception ex){
                            break;
                        }
                        
                        if ( next == null ) break;
                        
                        String pdfURL = (String)next.getMember("url");
                        if ( pdfURL == null ) break;
                        
                        
                        PDFRenderer renderer = new PDFRenderer();
                        URL baseURL = PDFRendererApplet.this.getDocumentBase();
                        String baseURLStr = baseURL.toString();
                        int queryPos = baseURLStr.indexOf("?");
                        if ( queryPos >= 0 ){
                            baseURLStr = baseURLStr.substring(0, queryPos);
                        }
                        
                        if ( !baseURLStr.endsWith("/") ){
                            int lastSlashPos = baseURLStr.lastIndexOf("/");
                            baseURLStr = baseURLStr.substring(0, lastSlashPos+1);
                        }
                        if ( pdfURL.indexOf(":") < 0 ){
                            if ( pdfURL.indexOf("/") == 0 ){
                                pdfURL = baseURL.getProtocol()+"://"+baseURL.getHost()+
                                        (baseURL.getPort()>0?(":"+baseURL.getPort()):"")+pdfURL;
                            } else {
                                pdfURL = baseURLStr+pdfURL;
                            }
                        }
                        
                        System.out.println("PDF URL is "+pdfURL);
                        renderer.setPDFURL(new URL(pdfURL));
                        
                        String req = null;
                        try {
                            req =  (String)next.getMember("request");
                        } catch ( Exception ex){
                            
                        }
                        
                        if ( "numPages".equals(req) ){
                            next.call("update", new Object[]{renderer.getNumPages()});
                        } else {

                            int page = ((Number)next.getMember("page")).intValue();
                            int width = ((Number)next.getMember("width")).intValue();


                            renderer.setWidth(width);
                            renderer.setPage(page);
                            BufferedImage img = renderer.getResult();
                            ByteArrayOutputStream os = new ByteArrayOutputStream();
                            ImageIO.write(img, "png", os);
                            os.flush();

                            String encodedImage = new Base64().encodeToString(os.toByteArray());
                            os.close();

                            next.call("update", new Object[]{encodedImage});
                            
                        }
                        //System.out.println(encodedImage);
                        
                        
                        
                        
                    } catch (Exception ex) {
                        //Logger.getLogger(PDFRendererApplet.class.getName()).log(Level.SEVERE, null, ex);
                        ex.printStackTrace(System.out);
                    } 
                    
                    
                    
                    
                    
                }
                running = false;
            }
            
        });
        dispatcher.start();

See full applet source code

Edit: In newer versions of Java there are additional security restrictions on network requests when the method call is initiated from Javascript. Therefore is is necessary to wrap the startDispatch() method in an AccessController.doPrivileged() call (Solution found here). To accommodate this I renamed the startDispatch() method as int_startDispatch() and I created a new startDispatch() method as follows:

public synchronized void startDispatch(){
        AccessController.doPrivileged(
                
                new PrivilegedAction(){
                    public String run(){
                        int_startDispatch();
                        return "";
                    }
                }
            
            
        );
        
    }

This ensures that we won’t run into security issues when loading the PDFs.

Basically this spawns a thread the runs a loop. In each iteration, a message is loaded from the queue. It gets the PDF’s URL, and other information about what is being requested. When it is done, it calls the update() method of the original message which is a javascript method. It this method is responsible for adding the image back to the DOM.

See a demo of the Web PDF Renderer

Javascript matching all characters including new line

In most languages you can add a modifier to the regex function to change the behavior of the ‘.’ operator so that it matches all characters including new lines (by default it matches everything except new lines). Javascript doesn’t!

This post shows a nifty solution to this problem:
http://siphon9.net/loune/2011/02/match-any-character-including-new-line-in-javascript-regexp/

EDIT:

Turns out this trick doesn’t work in IE. Here is another one that is supported by IE and apparently all other browsers: Use [\s\S] instead.

http://simonwillison.net/2004/Sep/20/newlines/