I gave a tutorial on Codename One at OSCON this year. Part way through the conference, I learned that OSCON had published its schedule as a public JSON feed and were encouraging developers to create their own schedule apps out of it. I was too busy at the time – but I wish I had known about this before hand as it would have made for a good subject for the tutorial. Alas, five months too late, here is a rough stab at this challenge, for the purpose of showing off a Mirah macro I just developed to make JSON Parsing easier in Codename One.

The State of JSON Parsing In Codename One

Codename One apps are Java apps, so they usually consist of a well-structured data model of Java classes and their instantiated objects. JSON (Javascript Object Notation) data is not strongly typed. It encapsulates a generic tree of maps, lists, and primitive data types. Codename One can easily parse JSON strings to a corresponding but generic data structure consisting only of java.util.Maps, java.util.Lists, Strings, and Doubles:

   JSONParser parser = new JSONParser();
   Map data = parser.parseJSON(jsonString);

This is a start, but it is not satisfactory. For example, our schedule application will have classes like Schedule, Event, Speaker, and Venue. We will definitely want to convert this Map into more specific Java types to help achieve a better level of abstraction in our app. But how?

Currently We need to Manually copy the contents of the generic data structure into the associate java types : Tedious and Error-prone

Why can’t we use a library like Jackson?

Libraries that automate the mapping of JSON data to Java types (like Jackson does) all require reflection. Codename One doesn’t support reflection because it needs to know what code is being used before it is deployed as a means of optimizing the performance and size of the app. If it were changed to support reflection, it would mean that the entire Java class library would have to be included in the app because the compiler wouldn’t know until runtime which classes were in use.

So using a library to handle this is out.

Mirah Macros to the Rescue

If you’ve been following any of my recent development, you know that I have been getting pretty deep into Mirah lately. Why Mirah?

It provides Ruby-like syntax, yet compiles directly to .class files with no runtime dependencies. This makes it ideal for Codename One development. You get all of the performance of Java but the productivity of Ruby.

In order to be able to use Mirah in my own projects, I developed a plugin for NetBeans that allows you to include Mirah source inside existing Java projects and work interchangeably with Java source. This capability is handy since some things are still better done in Java. It also allows me to sprinkle little bits of Mirah into my projects as I see fit.

Macros

One of the most powerful features of Mirah is its support for macros. A Mirah macro is a bit of code that is executed at compile time and is able to modify the AST. This allows you to do cool things like add properties and methods to classes, or even generate new classes entirely. This ability is also very powerful for Codename One development as it gives us the ability to perform compile-time reflection.

The data_mapper Macro

I created a macro named data_mapper that generates a class that knows how to convert between JSON and objects of a particular class. It works as follows:

   data_mapper MyClass:MyClassMapper

This generates a class named MyClassMapper that knows how to convert between JSON data and objects of MyClass. All generated “mapper” classes are subclasses of DataMapper.

The OSCON Schedule

The OSCON schema includes 4 types:

1. Schedule : The umbrella structure.
2. Event : Represents a single event on the schedule.
3. Speaker : Represents a speaker at OSCON.
4. Venue : A venue or room.

In my OSCON application, I created 4 corresponding classes. A simplified version of the Schedule class is:

    public class Schedule {
        private List<Event> events;
        private List<Speaker> speakers;
        private List<Venue> venues;

        // getters & setters, etc...

And the Event class is roughly:

public class Event {
    private String serial;
    private String name;
    private String eventType;
    private String venueSerial;
    private String description;
    private String websiteUrl;
    private Date timeStart;
    private Date timeStop;

    private List<String> speakers;
    private List<String> categories;

    // getters & setters, etc...

Parsing the JSON schedule into my class types, first, involved using the data_mapper macros to generate mappers for my classes:

data_mapper Event:EventMapper
data_mapper Venue:VenueMapper
data_mapper Speaker:SpeakerMapper
data_mapper Schedule:ScheduleMapper

The following is the sum total of the code that is used to actually load the JSON feed and parse it into my Java types:

ScheduleMapper scheduleMapper = new ScheduleMapper();
DataMapper.createContext(Arrays.asList(
        scheduleMapper,
        new EventMapper(),
        new VenueMapper(),
        new SpeakerMapper()
), new DataMapper.Decorator() {

    public void decorate(DataMapper mapper) {
        mapper.setReadKeyConversions(Arrays.asList(DataMapper.CONVERSION_CAMEL_TO_SNAKE));
    }
});
Schedule schedule = scheduleMapper.readJSONFromURL(
        "http://www.oreilly.com/pub/sc/osconfeed", 
        Schedule.class, 
        "/Schedule"
);

Let’s walk through this example, because there are a couple of implementation details included in this example because it is real-world, and thus not completely trivial.

1. If we were only mapping a JSON feed that contained a single Java type, we could just instantiate the DataMapper object and parse the feed. However, since there are multiple types in the feed, and the mappers need to know to use each other (e.g. The EventMapper needs to know that when it encounters a Speaker, that it should use the SpeakerMapper etc..), we need to set up a shared context. That is what the DataMapper.createContext() call does. It binds all of the mappers together so that they work as one.
2. The properties in the JSON feed use snake_case, whereas our Java classes use camelCase for their properties respectively. We use the setReadKeyConversions() method on all mappers to specify that it should convert these automatically.
3. The readJSONFromURL() method includes a third parameter that specifies the sub-key that is treated as the root of the data structure. Without this it would try to map the top-level JSON object to the Schedule object which would fail. (The schedule is contained in a sub-object with key Schedule).

There are other settings that you can play with such as date formats, but luckily the rest of the default settings work fine with this JSON feed.

At this point, the schedule object is completely populated with the OSCON schedule data, so it can be used to build our app.

Painless, right?

Screenshots

Download the App Source Code

This app is just a work in progress, but it may form the foundation for your own schedule app if you choose. You can download the source from GitHub here.

Build Instructions

Requires Netbeans 7.4 or higher with the following plugins installed:

1. Codename One
2. Netbeans Mirah Plugin
3. Netbeans CN1ML Plugin

Steps:

1. Clone the cn1-mirah-json-macros Gitub Project clone https://github.com/shannah/cn1-mirah-json-macros.git
2. Open the OSCONSchedule project that is a subdirectory of the main project directory, in NetBeans.
3. Build the project. Modify it at will

License

Apache 2.0

Screencast

I created a short screencast of this tutorial so you can see the data_mapper macro in action.

Links

1. Codename One Homepage
2. DataMapper API Docs
3. Codename One Reflection Utilities Download – Contains the libraries necessary to do the JSON parsing in a Codename One app.
4. Mirah Netbeans Module
5. OSCON App Source
6. OSCON DIY Schedule JSON Feed