Category Archives: Software Development

Posts about software development. Generally I use Java, PHP, and Python for development but occasionally I delve into other things as well.

clever-visuals-iMwiPZNX3SI-unsplash

Take me there

I love reading, but I don’t have time to “just read” so I tend to consume a lot of written material in “audio” format. This allows me to “read” while I do other things, like walking, driving, cleaning, and cutting the lawn. I “read” a lot of audio books, and follow a short list of podcasts. For the past year or so, I’ve also been experimenting with the latest in neural text-to-speech systems like Amazon Polly for converting blog posts into audio format so that I can listen to them during my walks. The results are surprisingly good. In many cases, I actually prefer the “machine” narration to a human narration. The voice is natural-sounding and consistent.

My favourite type of book (or blog post) is one that tells a true story, especially stories that intersect my personal lived experience. E.g. Stories about the birth of technologies that I use or remember. Insider accounts behind the scenes of movies or TV shows that I have watched. Memoirs of people who experienced certain events that remember living through. The more I “read”, the more specific my “tastes” become. You might say I’ve become more demanding of writers.

One of the most important qualities that I look for in writing is the ability to “take me there”. Books that give a mere account of what happened are barely better than reading a wikipedia article. I want a story to transport me into the time and place in which the described events occurred. I want to feel like a fly on the wall, so that I can imagine what it was like to be living in the story. Memoirs and personal anecdotes have a natural advantage for achieving this level of intimacy because the default is to see the events through the story-teller’s eyes. However, it is still possible to miss the target by focusing too much on sequence of events, and not enough on setting the scene and conveying how it felt to be there.

“The map is not the territory” is a well-known mental model that provides an analogy of what I’m looking for in a story. One way to explain this model is to consider that a map of Paris is not Paris. It is only a map the shows you where things are located from a birds-eye view. It doesn’t provide you with any information about what it feels like to walk the streets of Paris, or experience any of the historical landmarks. When I read a story, I want it to provide me with the territory. I can get the map off of Wikipedia or other reference sources.

My first exposure to this sort of story-telling was Console Wars by Blake J. Harris. It tells the story of the early nineties’ battles between Sega and Nintendo using a technique called as “Scene-based storytelling”. I had never experienced anything quite like it. It felt almost like I was living through a movie, as each bit of history was told through a scene. I don’t know how he was able to put together such a vivid picture the characters and conversations, but however he accomplished it, the end result was magic.

I immediately read his follow-up book History of the Future which uses the same technique to similarly vivid results.

These two books raised the bar for me, and I still have not found anything that quite “takes me there” like they do. I’m always looking, so any recommendations are appreciated.

More recently I’ve started “reading” the Mad Ned Memo, that includes stories from the computer/software industry by a 40-year veteran. His posts are always insightful, and usually combine a theme or timeless truth with some entertaining anecdotes. Not only do his stories “take me there”, they also take me back to my own parallel experiences in my early days of software development. I really wish I could find more content like this.

If you have done any type of software development, or participated in the development of long forgotten projects, I’d love to read about your experiences.

Photo by Clever Visuals on Unsplash

Maven2iOS

Video: Building a Codename One Project for iOS

This is the third video in my series about our new online tool, Codename One initializr, which allows you to generate a Maven starter project for a native mobile app in one click. The first video showed how to generate the starter project, and run it in the Codename One simulator. The second video showed how to build and deploy the project on an Android device. In this video I show how to build and deploy the project on an iOS device.

TLDW (Too Long Didn’t Watch):

This video starts out with my Codename One project already opened in IntelliJ. See this post for steps on how to generate this project.

In the video I demonstrate two different approaches for building the iOS app.

  1. Locally (0:55-2:45) – Requires a Mac with Xcode Installed.
  2. Using Build Server (6:45-8:35) – Can be built on Windows, Linux, or Mac. With no special requirements beyond Maven and the JDK. You just need a free Codename One account.

NOTE: I also show how to generate your iOS certificates and provisioning profiles using the Certificate Wizard (2:45-6:45), as this is required to build apps for iOS.

Building Locally

The local build option generates an Xcode project, which we then open and build using Xcode.

To trigger this build, select “Local Builds” > “Xcode iOS Project”:

Screen Shot 2021-04-06 at 5.51.41 AM

Then press the “Run” button.

It takes the ParparVM compiler a minute or two to do its thing, but when it’s done, it opens the generated Xcode project in Xcode.

Screen Shot 2021-04-06 at 5.55.03 AM

Once opened, I press the “Run” button on the Xcode toolbar and wait while it compiles the project. When it is done, it opens the iOS simulator with my app running in it.

Screen Shot 2021-04-06 at 5.58.05 AM

Building with the Build Server

One of the nice things about Codename One is that it provides a build server with all of the native build tools installed and up-to-date. This simplifies the process of building native apps greatly. You can build your project for iOS, Android, Mac Desktop, Windows Desktop, Windows UWP, and Javascript without requiring any special build tools installed beyond the JDK. Building for any of these targets is as simple as pressing a button, or running a Maven goal.

Generating Certificates

Building for iOS requires that you have an Apple developer account. Additionally, Apple requires you to generate certificates and provisioning profiles for your apps. This is by far the most painful part of app development. To help ease the pain, Codename One provides a certificate wizard to help generate these. Before I can submit my first iOS build, I need to walk through the certificate wizard to generate these certificates. The certificate wizard process starts at approx 2:45 in the video, and runs until 6:45.

To access the certificate wizard, I need to open Codename One Settings. I do this by selecting “Tools” > “Codename One Settings” from IntelliJ’s configuration menu, then pressing the “Run” button.

Screen Shot 2021-04-06 at 6.11.27 AM

This will open The Control Center (aka Codename One Settings, aka Codename One Preferences):

Screen Shot 2021-04-06 at 6.12.53 AM

Once there, I select “Device Settings” > “iOS” > “Certificate Wizard” from the navigation menu on the left.

Screen Shot 2021-04-06 at 6.13.55 AM

This displays the login form for the certificate wizard:

Screen Shot 2021-04-06 at 6.15.40 AM

IMPORTANT: You need to use your Apple Developer account to login to this form. NOT your Codename One account.

In the video I spliced out some of the waiting time. The login can take a little while, so be patient. Once logged in, it shows me a list of my registered development devices, and I can select which ones I want to be able to deploy this app to for testing and debugging.

Screen Shot 2021-04-06 at 6.17.06 AM

The above screenshot has all of the rows greyed out. When you log in, you’ll see device names and UDIDs listed on this form.

Generally I select all of them. If this is your first time building an iOS app, then you may not have any devices listed yet, and you’ll need to click on the “Manage Devices” button and follow the instructions there.

Next, it asks me to confirm that I want to regenerate my certificates, as it has detected that I already have certificates generated in my Apple account. In my case, I say “yes”, I’d like to regenerate them, but in most cases, you would select “no”, to just use your existing certificates.

TIP: If your certificates were generated by the certificate wizard, then a copy of them has been stored inside the $HOME/.codenameone/iosCerts directory, and the wizard will use them automatically. If they weren’t generated by the certificate wizard, and you choose not to regenerate them, then you may need to specify the location of your certificates in the iOS Settings section.

Screen Shot 2021-04-06 at 6.25.52 AM

Next, it asks whether we want to generate push certificates. In this case, since this is just a basic Hello World app, we don’t need push, so I leave these options OFF.

Screen Shot 2021-04-06 at 6.27.38 AM

After clicking next, it will churn for a bit, and if all goes well, it will show us the message that our certificates were generated and installed successfully.

Just to be sure that my settings are saved. I click on the hamburger menu in the upper right, and select “Save”.

Screen Shot 2021-04-06 at 6.29.40 AM

Sending the Build

Now that the certificates are generated, we can send the build. Back in IntelliJ, I select “Build Server” > “iOS Debug Build”

Screen Shot 2021-04-06 at 6.32.38 AM

NOTE: If this is your first time building with the build server, you may be prompted for your Codename One username and password.

I then follow the progress of the build on the Codename One website.

When it’s finished, I get a set of links to do things like download the .ipa, or install the app on device.

Screen Shot 2021-04-06 at 6.34.32 AM

Get Started

Getting started with your own native app is really easy. Just go to the Codename One initializr, enter your app details, and press “Download”.

For more information about Codename One, see the Codename One website.

domenico-loia-1000x667

Deploying Apps on Multiple Form-Factors

Photo by Domenico Loia on Unsplash

The other day I stumbled across this post whose title seemed to suggest that Flutter is not a cross-platform framework.

Screen Shot 2021-04-06 at 10.08.33 AM

The thrust of his article is that, even though Flutter allows you to build your app for 6 platforms, that doesn’t mean that you should:

Yes, you can deploy your app on 6 platforms, but honestly, I am not planning to do so. Basically, because YOU SHOULD use different design patterns depending on the platform. I can’t imagine deploying my apps on a different platform.

At first glance, he appears to be arguing for writing separate apps for each platform (e.g. Android, iOS, Mac, Windows, etc…). This idea that you need to write a separate app for each platform using the platform’s native UI toolkit is widespread in the developer community. “Native Widget Maximalists”, as I call them, believe that using cross-platform UI libraries will result in a sub-par, “non-native” experience, and will, therefore, be rejected by the user. Generally, adherents to this philosophy are fine with sharing “business logic”, but the user interface must use the native UI widgets. Much of this dogma is based on dated observations of clunky, cross-platform, desktop apps of the mid to late nineties – many of them developed by novices using early incarnations of Swing.

Since that time, cross-platform toolkits have matured, and platforms have converged on some common UI design patterns. This is especially the case on mobile, where many popular native apps look nearly identical on Android and iOS. Mobile developers have realized that it is more important to create a nice, consistent design than it is to try to “look native”. Yes, there are differences between iOS and Android, but the differences are the exception – not the rule. In my opinion it is overkill to maintain two separate codebases for the 2% of the UI where they diverge. Better to provide abstractions that allow that 2% delta to be satisfied in platform-specific ways.

If you read further into the article, you’ll see that the author actually isn’t a “native widget maximalist”. I.e. He isn’t arguing that you should build separate apps for iOS and Android using their native SDKs. He isn’t even arguing that you need to write separate apps for iOS and Android.

Usually what works on mobile won’t work on desktop and the other way around.

What he’s saying is that you shouldn’t deploy the same app on desktop as you do on mobile, because the form factor is too different. If this is his thesis, then I agree with him… with some caveats.

Strategies for targeting multiple form factors

Disclaimer: I work for Codename One.

Two of the best cross-platform development tools for mobile development are Codename One and Flutter. They approach the problem of cross-platform development in very similar ways. Both provide 100% code reuse across platforms. Both provide a rich set of UI components and API abstractions for the underlying device capabilities, and both can be deployed to iOS and Android (and other platforms), as native apps. Codename One apps are developed in Java and/or Kotlin. Flutter, in Dart.

Both Codename One and Flutter also allow you to deploy your app as a desktop app. However, if you don’t tweak your UI for the larger screen-size, and desktop usage patterns, the result probably won’t be very good. Even using mobile apps on tablet feels forced if you haven’t customized the UI for the larger screen-size. There are four strategies I use when building a multi-form-factor app (i.e. an app that runs on mobile and desktop):

1. Responsive UI

Screen Shot 2021-04-06 at 10.26.54 AM

This is where the app logic is essentially the same across both form-factors, but the layout manager, and styles are “form-factor”-aware. E.g. On tablet/desktop they use different styles, and the layout managers position elements differently. (E.g. Instead of a hamburger button that reveals a side-menu sliding out over top of the form, the side menu is always visible).

2. Component-level abstraction

Screen Shot 2021-04-06 at 10.47.10 AM

This is where most of the app’s control flow is the same, but certain parts of each form are abstracted to allow for different implementations on desktop, tablet and mobile. This may involve using a different widget for editing some field, displaying some extra sections on desktop that aren’t visible on mobile. This is very similar to Responsive UI, and there is certainly overlap. The distinction is that with Responsive UI, you are keeping all of the same UI elements – you’re just rendering them differently. With component-level abstraction, the UI form may actually include different UI components with different logic on desktop than it does on mobile.

3. Alternate views

Screen Shot 2021-04-06 at 10.57.34 AM

This is where the app’s control flow is the same, but you create entirely different views on mobile than on tablet. If you are very careful with the design of your views, you may be able to reuse your controller classes, as long as the views share common APIs, and fire compatible events. Keeping them in sync can be challenging, so quite often you would also write separate controllers as well.

4. Separate control-flow

Screen Shot 2021-04-06 at 10.58.55 AM

This is where you are basically implementing two separate apps. You can reuse business logic, but the UI layer is written separately for tablet/desktop and mobile.

5. Separate apps

Screen Shot 2021-04-06 at 11.01.43 AM

If you are already implementing your app with separate control flow, then creating separate apps is just one small additional step. Generally you would still share all of your business logic between the apps. You would just provide alternate entry points for the different apps. With Codename One, this can be achieved either by moving all of the code into a shared library (cn1lib), or by simply providing an alternate configuration file (codenameone_settings.properties) that specifies a different main class. Most build targets use Proguard, or equivalent, to strip out unused code, so the app size isn’t impacted by the code-sharing.

Best choice?

It appears that the author of the article is arguing for option #5 – Separate apps. His preference, he says, is informed by his experience working on large enterprise systems where there would be different teams working on the apps for different platforms, and keeping it all in the same app would lead to toes being stepped on. Option #4 (Separate flow control) should adequately address his concern as well, since each form-factor would have its own package, likely, and developers wouldn’t need to tread on anyone else’s garden.

My preference is to use the lowest number on that scale that I can get away with, and progress up the ladder as required. IntelliJ makes refactoring from one strategy to another mostly painless, and the less fragmentation there is in the code-base, the easier it will be to maintain – generally. Obviously adding team members, or splitting the project into multiple teams changes that maintenance calculation.

Still prefer a cross-platform development tool

Suppose your team decides to implement separate apps for each form-factor (Mobile, Tablet, and Desktop). Let’s even go a step further and suppose that you decide to implement separate apps for each platform (Android Mobile, Android Tablet, iPhone, iPad, Mac, Windows, Linux). Then is there still any benefit using a cross-platform toolkit like Codename One or Flutter? Since you’re doing separate apps, wouldn’t it be just as well to just use the native APIs?

Unless you have an unlimited supply of time, developers, and money, then the answer is “no”. You would be much worse off by choosing to use separate native SDKs for each platform. Even if you manage to write some shared modules that you were able to share between the projects, the complexity involved in maintaining separate codebases is staggering. Everything is 7x more difficult. Every bug is fixed 7 times, and testing gets ridiculously complex. In addition, keeping up with the latest on all of these platforms and APIs takes dedication. You would likely need to bring in separate teams for each platform – and very little of the work can be shared between the teams.

Using a technology like Flutter – even if you are building 7 separate apps, would be far easier. Sharing code between projects is much easier, and every developer can work on every project without facing barriers to entry imposed by the idiosyncrasies of each native API.

Summary

Just because you can deploy your app to 8 different platforms, doesn’t mean that you should. Deploying to multiple platforms within the same form factor (e.g. phones) is a solid approach with a proven track record – with countless popular apps on the iOS and Android app stores currently developed with cross-platform tools like Codename One and Flutter. However, deploying to multiple form-factors (e.g. phone and desktop) is more difficult, as what works on one form-factor, may not work well on another. You may be better served by creating separate projects for each form factor, and sharing business logic between them. This doesn’t mean that you should drop your cross-platform development tool (e.g. Flutter/Codename One). Using such a tool is still a benefit as it reduces the combined project complexity, and makes it easier to share code and developers between the projects.

Maven2Android

Codename One Project -> Build Android App

In my last post I showed off the new Codename One initializr online tool, generating a Maven project, and opened it in IntelliJ.

In this video I demonstrate how to build an Android app with this project.

TLDW (Too Long Didn’t Watch):

Here’s the gist of the video. There are two different build options for Android:

  1. Build Server > Android
  2. Local Builds > Android Gradle Project

In this video, I start with option 2, “Android Gradle Project”. This option does NOT require a Codename One account, and performs all of the build on your local machine. It does require that you have Android Studio installed.

I select “Local Builds” > “Android Gradle Project” from the Configuration menu of IntelliJ, and then press “Run”.

Screen Shot 2021-03-30 at 9.25.41 AM

This generates an Android Studio project, and automatically opens it in Android studio.

Screen Shot 2021-03-30 at 9.27.42 AM

I then press “Run” in the Android Studio, and wait while it builds and installs the app on my Android Emulator.

Screen Shot 2021-03-30 at 9.29.25 AM

In the second part of this video, I use the “Build Server” > “Android” build option, which is much simpler, and doesn’t require you to install Android Studio. All you need is IntelliJ (Actually you don’t even need IntelliJ, as you could just build the project using Maven), and it will use the Codename One build server to generate the Android app.

After selecting “Build Server” > “Android” from the configuration menu, I press “Run” to start the build.

Screen Shot 2021-03-30 at 9.30.39 AM

It then redirects me to the Codename One dashboard where I can monitor the build progress and, download the app when it’s done.

Screen Shot 2021-03-30 at 9.31.37 AM

More Background

When we decided to migrate to Maven, we also made the choice to add official local build targets so that developers are no longer reliant on the build server to build their Android and iOS apps. Building locally has always been an option, but it was difficult, and we didn’t provide support for it. By adding an official local build option, we are hoping that developers who balked at Codename One because they didn’t want to be reliant on us for their builds will give us another look.

If you haven’t heard of Codename One yet, I encourage you to check us out. In my biased opinion, we are the best game in town, if you’re looking to build native mobile apps in Java or Kotlin.

It only takes a minute to create and build your first project using Codename One initializr.

Screen Shot 2021-03-30 at 6.45.27 AM

Preview: Online Tool to Generate iOS/Android app starter project

The Codename One initializr is an online tool for generating a Codename One starter project. You can select either Kotlin or Java, then you can download the project and open it in your local IDE.

In this 5-minute video I use the initializr to generate a bare-bones Java project, which I open and run in IntelliJ. I also give a brief tour of the project structure and build targets.

TLDW (Too Long Didn’t Watch):

Here’s the gist. Go to Codename One initializr, select either the “Java Bare-bones Project” or “Kotlin Bare-bones Project” from the “Template” select box, and press “Download”

Screen Shot 2021-03-30 at 6.36.39 AM

Extract the resulting project, and open it in IntelliJ (or your preferred IDE).

Screen Shot 2021-03-30 at 6.39.27 AM

Press “Run” and wait while Maven downloads the build dependencies. It will open the Codename One simulator with the simple “Hello World” app running.

Screen Shot 2021-03-30 at 6.41.00 AM

The project is a Maven project, with the following build targets:

  • iOS
  • Android
  • Mac Desktop
  • Windows Desktop
  • Javascript App
  • Windows UWP App,
  • JavaSE Desktop App
  • Xcode Project
  • Android Studio Project

In the video, I also demonstrate how to add a Button that, when clicked, opens a dialog. This is achieved by opening the common/src/main/java/com/example/myapp/MyApp.java file, and adding the following to the start() method:

Button btn = new Button("Click Me");
btn.addActionListener(evt->{
    Dialog.show("Hello World", "You Clicked Me", "OK", null);
});
hi.add(btn);

I’ll soon be posting some follow-up videos to demonstrate how to build and deploy the project to iOS and Android devices, so watch the RSS feed or follow me (wherever you receive my posts), to be notified when these are posted.

More Background

I’ve been working hard over the past several months to migrate Codename One from Ant to Maven. That process is now complete, and it has enabled us to introduce a new, simpler workflow for creating Codename One projects. The Codename One initializr uses our Codename One App Project maven archetype to generate a starter project. Right now we just have two starter templates: Bare-bones projects for Java or Kotlin. We will be adding more templates soon, including some templates for full-featured apps that you can take and customize.

I’ve very proud of this work, and I’m excited about some of the new things that it will enable.

If you’re a Java or Kotlin developer and you’re interested in making cross-platform native mobile apps, you should give Codename One a try. You just might be surprised at how pleasant the experience is.

New Mobile Theme for Xataface

During my two week “vacation” from Codename One, I’ve been madly working on a new project using Xataface. For this project, I really needed the mobile interface to be smooth, so I decided to finally make Xataface’s core theme responsive. Along the way, I also made numerous improvements to the flow of the UI especially in relation to sorting and filtering results. Before I go into detail about the new features, here are some screenshots of my app, which uses this new mobile theme.

The new login screen is much cleaner and mobile friendly.
The new login screen is much cleaner and mobile friendly.
The mobile registration form - fields generated based on the fields in the users table.
The mobile registration form – fields generated based on the fields in the users table.
The list view for the "News Feed" table.
The list view for the “News Feed” table.

Screen Shot 2020-08-30 at 7.21.34 AM

Let me unpack that above screenshot of the list view to highlight the various aspects you can see here.

  1. The “tables” menu is rendered along the bottom of the screen as tabs.Table tabs
  2. Sorting and filtering buttons are rendered at the top of the list. Sort and filter buttons When you scroll down the page, these buttons are converted into floating buttons: Floating sort and filter buttons
  3. Notice the Floating Action Button in the lower right for adding new records. Floating Action Button. By default this shows the “New” and “Delete” actions, but you can add your own actions to this menu using the “table_actions_menu” category.
  4. Notice that action icons below each row. These are rendered from the “list_row_actions” category. List row actions

Sorting

Clicking on the “Sort” button displays a sheet with the various options available for sorting.The sort dialog. You can select which columns should be sortable in the fields.ini file using the new sortable directive.

Filtering

Clicking on the “Filter” button displays a sheet with the various options available for filtering.

Screen Shot 2020-08-30 at 7.40.27 AM

This filter dialog is “live”. The button at the bottom that says “Show 977 Results” will dynamically update as you enter your query so that you can see how many results there will be.

Optional Search Header

On some tables you may want the header to be a “search” field. This can be achieved using the the new fieilds.inii “search_field_header” directive, as demonstrated in this table:

Search Header

New/Edit Record Forms

Forms are Xataface’s bread and butter, so they need to be very mobile friendly. I’ve completely revamped the stylesheet to be responsive so that forms are a pleasure to use on the smaller displays.

New Record Form

I’ve also added a new feature to help reduce clutter on forms. You can now make field groups “hidden” by default. Hidden field groups are collapsed into buttons that are rendered at the bottom of the form:

Fieldgroups menu

The user can display a hidden field group by clicking on the corresponding icon. E.g. In the example form shown above, the user might want to edit the “narration”. They can do so by clicking on the “Narration” icon at the bottom of the form, which will reveal the narration-related fields.

Narration field group

More to Come

This is just a quick post to share some of the work. There are tons of new features that I didn’t cover here. I’ll be blogging more about them soon.

I’ve been slowly assembling a “definitive” guide for Xataface. You can see the current version (in progress) at https://shannah.github.io/xataface-manual/

After I’ve ported all of the existing documentation into this manual, I’ll be using it as the basis for a new website. There is lots of new stuff in the pipe for Xataface, so stay tuned.

JAXB Hell on JDK 9+

JAXB has been removed from JavaSE starting in JDK 9, so we’ve had to make some changes to some of our code to work around this. We have some custom ANT tasks that use JAXB to process some XML. The task it used inside an ANT script using the <taskdef> tag as follows:

<taskdef name="myCustomTask" 
    classname="com.example.tasks.MyCustomTask" 
    classpath="MyCustomTask.jar"/>

Then laster the task us run using syntax like:

<myCustomTask />

If you try to run these ANT tasks on JDK 9 or higher, you just get a big ClassNotFound error when it tries to load the JAXB classes. So the obvious solution is to bundle the JAXB classes into our jar. This, however, only solves part of the problem. This will, indeed, allow the task to load, but when we try to run the task, it says

javax.xml.bind.JAXBException: Implementation of JAXB-API has not been found on module path or classpath.

Which is strange because we have the API (jaxb-api.jar) and implementation (jaxb-impl.jar, jaxb-core.jar, activation.jar) embedded inside our MyCustomTask.jar file, which should be available on the classpath.

After banging my head against this problem for a few hours, I discovered that the problem is the way which JAXB looks for the implementation. It uses the threads classloader for searching for an implementation, rather than the classloader for our task. When running inside an ANT task, this will be the root classpath for Ant, and not the classpath for my custom ant task.

For example, we have some code like:

JAXBContext componentContext = JAXBContext.newInstance(ComponentEntry.class);

This will fail to find the JAXB implementation (unless we included the JAXB jars in Ant’s classpath – which is not portable in our case, because ANT will usually be run inside an IDE like Netbeans).

I ultimately worked around this problem by wrapping all of my JAXB code inside my own spawned thread. E.g.

private String processFileWithJAXB(final File xmlFile, final boolean full) throws JAXBException {
        final JAXBException[] error = new JAXBException[1];
        final String[] result = new String[1];

        Thread t = new Thread(new Runnable() {
            @Override
            public void run() {
                try {
                    result[0] = processFileWithJAXBInternal(xmlFile, full);
                } catch (JAXBException ex) {
                    error[0] = ex;
                }
            }

        });
        ClassLoader cl = getClass().getClassLoader();
        t.setContextClassLoader(getClass().getClassLoader());
        t.start();

        try {
            t.join();
        } catch (InterruptedException ex) {
            Logger.getLogger(MyClass.class.getName()).log(Level.SEVERE, null, ex);
        }
        if (error[0] != null) {
            throw new JAXBException(error[0]);
        }
        return result[0];
    }

In this example, all of my JAXB stuff is inside the processFileWithJAXBInternal() method. The processFileWithJAXB method creates a thread, sets its context classloader to the current classes class loader, and runs it. And magically, it can find my bundled JAXB implementation.

Posting in my blog to help my memory as I’m bound to run into this issue again.

PSA: Prefer to use AdoptOpenJDK’s jdk-11 builds for embedding in Mac Apps

If you are planning to distribute a Java app on Mac, you should avoid using the JDK builds from jdk.java.net as they won’t necessarily work on Mac OS older than 10.13. This is because the libjvm.dylib is build with MACOSX_MIN_VERSION set to 10.13. This doesn’t necessarily cause a problem until you try to run a signed app on Yosemite or older (10.10). Your app just won’t open. Checking the logs you’ll receive an error like:

​Error: dl failure on line 542
Error: failed /Applications/MyApplication.app/Contents/Java/jre//lib/server/libjvm.dylib, because dlopen(/Applications/MyApplication.app/Contents/Java/jre//lib/server/libjvm.dylib, 10): no suitable image found.  Did find:
    /Applications/MyApplication.app/Contents/Java/jre//lib/server/libjvm.dylib: code signature invalid for '/Applications/MyApplication.app/Contents/Java/jre//lib/server/libjvm.dylib'

Now, you might be fine if you’re building the app on 10.10 or older, but not sure. This particular issue is a combination of:

  1. libjvm.dylib set with a min version of 10.13.
  2. codesign on 10.11 and higher automatically signs libs targeting 10.11 and higher with a different signature than is can be understood by gatekeeper pre 10.11.
  3. Gatekeeper barfing when it hits this signature.

So, If you’re building (signing) your app on the latest Mac OS and you want to be able to distribute it to older versions of OS X, you need to make sure that all of your libraries are built with the MACOSX_MIN_VERSION set to 10.10 or lower.

You can verify this using otool. Inside the standard openjdk build on jdk.net, you can go into the Contents/Home/lib directory, and run:

$ otool -l  */libjvm.dylib | grep VERSION -A 5 | grep version
  version 10.13
  version 0.0

(Note: libjvm.dylib is the only problematic one. All the other dylibs are built with 10.8 min version).

However, if you download the build from AdoptOpenJDK, and do the same thing, you’ll find

$ otool -l  */libjvm.dylib | grep VERSION -A 5 | grep version
  version 10.8
  version 0.0

Just another reason to use AdoptOpenJDK for your Java distro.

High Sierra, Ruby, RVM, CocoaPods, ipatool, xcodebuild Ughh!!

This is a very terse post to record a problem/solution for Cocoapods on High Sierra.

I have a build process that involves running cocoapods and xcodebuild on the command line. Some time ago, I installed rvm in an attempt to fix a build error. It ended up not fixing the error, but I kept it installed because it seemed like a nice thing to be able to easily switch between ruby versions. However, xcodebuild and its sub-tools are picky about using the system ruby, so before running xcodebuild, I would always have to switch back to the system ruby using rvm use system. This was an inconvenience, but it wasn’t hard to do, so I just endured.

Some time later, I upgraded to High Sierra, which broke my system ruby. Running ruby would give an error like

dyld: Library not loaded: /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/libruby.2.0.0.dylib
  Referenced from: /usr/local/bin/ruby
  Reason: image not found
Abort trap: 6

This seems to be because High Sierra upgraded its ruby to 2.3.0, but rvm some things still referenced the old 2.0 installation, which had been removed. In trying to resolve this situation I first uninstalled rvm – because I didn’t seem to be helping anything and could possibly be hurting things:

rvm implode

After doing this, I still received the error message above. I then noticed that I had two ruby binaries on my Path: /usr/local/bin/ruby and /usr/bin/ruby. The former referenced the old 2.0 libs, while the latter seemed to correctly reference the new location. So, I deleted the former:

sudo rm /usr/local/bin/ruby

And things almost started working. When I tried installing cocoapods again with:

sudo gem install cocoapods

I received “Permission denied” messages. I needed to do

sudo gem install -n /usr/local/bin cocoapods

And, voila!, things magically start working again.

Use NPM to Distribute Your Command-Line Java Apps

jDeploy flow

From time to time, I need to develop and distribute a command-line application to my customers, and I prefer to write these apps in Java. But I’ve been frustrated by the Java eco-system’s lack of good distribution options for this type of app. I look at tools like Cordova and Ionic that provide a neat “single-line” installation instructions and wonder why I can’t achieve the same thing with Java.

My general solution up until now has been to zip up the executable Jar with its dependencies, and post it for my users to download. The installation instructions then become something like:

  1. Download myapp.zip
  2. Extract it
  3. Then run the app using something like “java -jar /path/to/myapp/myapp.jar”

Sometimes I go as far as writing shell script launch scripts that people can add to their PATH, but this adds an additional installation step…. we were already at 3 possible places for users to get stuck.

Compare this to the installation instructions for Cordova:

  1. Open a command prompt and type “npm install -g cordova”. No need to download anything manually. Just one command
  2. Once installed, you can just run “cordova”, you don’t need to know where it was installed. A symlink is automatically installed in your environment PATH.

That is the type of user experience I want for my command-line apps. Java offers some great package management options for distributing libraries (e.g. Maven and Gradle), but these don’t offer a simple way to distribute applications. Wouldn’t it be nice if we had a way to distribute command-line apps the same way?

NPM to the Rescue

The examples I provided above (Cordova and Ionic) both use NPM as their distribution mechanism. That is what allows them to provide such a smooth process. If you’re not familiar with NPM, it is the package manager that is bundled with NodeJS. Unlike Maven, which is constrained to use in the Java ecosystem (you can only distribute .war, .jar, and pom files), NPM allows you to distribute all manner of files, so why not use it to distribute Java apps. Sounds crazy, I know. But let’s look at some of the features of NPM that make it ideal for distributing Java applications:

  1. WORA – NPM is cross platform, and is automatically installed when users install NodeJS on their system. NodeJS provides a simple double-clickable installer for Mac, Windows, and Linux, so your application can easily be installed on all major platforms.
  2. Pain-free Publishing – Publishing an app to NPM is as simple as npm publish. If you don’t yet have an account on NPM, it’s as simple as npm login. Literally could take you under a minute to get set up and rolling.
  3. Adds to PATH – NPM will automatically add symlinks for your application to the environment path when installed globally. On windows it automatically generates a .cmd script to launch the app. That is the magic that allows apps like Cordova to provide single-line installation instructions.

In addition to all this, NPM provides automatic versioning and updates so that it is easy for you to push updates out to your users. This is a big deal.

But How Do I Deploy my Java App using NPM

You say: This is all fine and well, but how do I leverage NPM to deliver my Java application. NPM is designed to distribute apps written in Javascript, not Java.

I respond: Let me introduce you to jDeploy

jDeploy is a tool that allows you to publish your Java applications on NPM so that users can install it directly using a simple command like

npm install -g your-app

It will take an executable Jar file and bundle it in a node module with all of its dependencies, and publish it to NPM. The resulting app doesn’t even require that the users have Java installed, as it will automatically install a JRE if it detects that Java isn’t installed.

Example

Installing jDeploy

$ npm install -g jdeploy

Publishing an App to NPM

Suppose you have an app in an executable jar file, “myapp.jar”. Place this in a directory on its own, and open a command prompt in this directory, and type:

jdeploy init

This will generate a package.json file for your app that is ready and configured to publish on NPM. You may want to edit this package.json slightly to suit your purposes. The default will look something like:

{
  "bin": {"myapp": "jdeploy-bundle/jdeploy.js"},
  "preferGlobal": true,
  "version": "1.0.1",
  "jdeploy": {"jar": "myapp.jar"},
  "dependencies": {"shelljs": "^0.7.5"},
  "license": "ISC",
  "name": "myapp",
  "files": ["jdeploy-bundle"] 
}

The “myapp” key in the “bin” property is the command name for your app. This is the name of the command that users will run on the command-line to use your app. Change this to something else if you want the command to be different.

The “name” property should be globally unique. This is the identifier that will be used to install your app from NPM. E.g. npm install -g myapp. You may need to change this if the name is already taken by someone else.

Once you’re happy with the settings, you can test out the app locally to make sure it runs.

$ jdeploy install

This should install your command locally so you can try it out:

$ myapp

And your app should run.

Once you’re satisfied that your app works the way you like, you can run

$ jdeploy publish

Installing Your App

Now that your app is published, people can immediately install your app from anywhere in the world, on any computer that runs NPM with a single command:

$ npm install -g myapp

NOTE: On Mac/Linux you will need to use sudo to install the app. Windows should work as long as you are using an administrator account.

Screencast

Introduction to jDeploy Screencast