optimistic about maven android integration

I've been using the maven-android-plugin recently. Here's a little video showing building, deploy, and undeploy of an simple application to my phone. I particularly love the android:deploy and android:undeploy goals.



To get the full benefit of these features in Eclipse we need m2e integration. Luckily, the m2e-android project is making good progress.

Good stuff.

windows dexdump failure

This is frustrating because it should be such a trivial thing. I don't know what the fix is. I was hopeful that the binary attached to Android Issue 14746 would fix me up but that didn't work out.

simple dx / dexdump on Mac

In Windows, I was getting a silly-stupid message from dx and then an error from dexdump so I wanted to try a dx / dexdump on Mac for a comparison. I picked a random JAR file.

[jjohnson@jjohnson 20110613]$ cp ~/.m2/repository/commons-lang/commons-lang/2.4/commons-lang-2.4.jar .
[jjohnson@jjohnson 20110613]$ dx --dex --output=test.apk commons-lang-2.4.jar
[jjohnson@jjohnson 20110613]$ dexdump -d -l xml test.apk

<api>
<package name="org.apache.commons.lang"
>
<class name="ArrayUtils"
 extends="java.lang.Object"
 abstract="false"
 static="false"
 final="false"
 visibility="public"
>
...

Good enough. Easy peasy.

simple made difficult

C:\Documents and Settings\Jeremiah Johnson\20110613>copy ..\.m2\repository\commons-lang\commons-lang\2.4\commons-lang-2.4.jar .
C:\Documents and Settings\Jeremiah Johnson\20110613>dx --dex --output=test.apk commons-lang-2.4.jar

UNEXPECTED TOP-LEVEL EXCEPTION:
java.io.FileNotFoundException: commons-lang-2.4.jar (The system cannot find the file specified)
        at java.util.zip.ZipFile.open(Native Method)
        at java.util.zip.ZipFile.(ZipFile.java:127)
        at java.util.zip.ZipFile.(ZipFile.java:144)
        at com.android.dx.cf.direct.ClassPathOpener.processArchive(ClassPathOpener.java:205)
        at com.android.dx.cf.direct.ClassPathOpener.processOne(ClassPathOpener.java:130)
        at com.android.dx.cf.direct.ClassPathOpener.process(ClassPathOpener.java:108)
        at com.android.dx.command.dexer.Main.processOne(Main.java:313)
        at com.android.dx.command.dexer.Main.processAllFiles(Main.java:233)
        at com.android.dx.command.dexer.Main.run(Main.java:185)
        at com.android.dx.command.dexer.Main.main(Main.java:166)
        at com.android.dx.command.Main.main(Main.java:90)
1 error; aborting

C:\Documents and Settings\Jeremiah Johnson\20110613>dx --dex --output="C:\Documents and Settings\Jeremiah Johnson\20110613\test.apk" "C:\Documents and Settings\Jeremiah Johnson\20110613\commons-lang-2.4.jar"
and was unexpected at this time.

and was unexpected at this time.

That's for real - did you see that in the previous output? That's the whole error message. Neat. So you can't use relative paths and you can't use an absolute path with a space? Perhaps Windows users already know what to really do but I'm just trying to be a good citizen and test a defect fix on Windows without tons of recent Windows experience.

final dexdump failure on windows

C:\Documents and Settings\Jeremiah Johnson\20110613>cd ..
C:\Documents and Settings\Jeremiah Johnson>move 20110613 C:\
C:\Documents and Settings\Jeremiah Johnson>cd C:\20110613
C:\20110613>dx --dex --output=C:\20110613\test.apk C:\20110613\commons-lang-2.4.
jar
C:\20110613>dexdump -d -l xml test.apk
E/dalvikvm( 3152): sysMapFileSegmentInShmem not implemented.
W/dalvikvm( 3152): Zip: cd map failed
Unable to open 'test.apk' as zip archive
Not Zip, retrying as DEX
E/dalvikvm( 3152): ERROR: Bad magic number (0x50 4b 03 04)
ERROR: Failed structural verification of 'test.apk'

C:\20110613>dexdump -d -l xml C:\20110613\test.apk
E/dalvikvm( 1740): sysMapFileSegmentInShmem not implemented.
W/dalvikvm( 1740): Zip: cd map failed
Unable to open 'C:\20110613\test.apk' as zip archive
Not Zip, retrying as DEX
E/dalvikvm( 1740): ERROR: Bad magic number (0x50 4b 03 04)
ERROR: Failed structural verification of 'C:\20110613\test.apk'

jsp pre-compile and proper servlet classloading

Following my previous post, I configured JSP pre-compiling and adjusted my simple WebAppContext a bit to get the servlet classloader working properly. Without fixing the classloader, I couldn't actually use the pre-compiled JSPs. After getting the classloading worked out, I switched from WebAppContext to WebAppProvider: 1) WebAppProvider allows polling for redeploy of the WAR and 2) the WebAppContext is deprecated so I felt like I should use non-deprecated APIs in this new code.

The i-Jetty project is a pretty interesting project but goes a different route than what I'm doing. I followed some of the ideas there but simplified for what I need.

added configuration to WebAppContext

        // start the Web server for config and maintenance
        webServer = new Server(8080);
        try {
            WebAppContext webapp = new WebAppContext();
            webapp.setConfigurationClasses(new String[] { DexWebInfConfiguration.class.getName(),
                    JettyWebXmlConfiguration.class.getName(), WebXmlConfiguration.class.getName() });
            webapp.setContextPath("/");
            webapp.setTempDirectory(new File("/sdcard/TGP-work/"));
            webapp.setWar("/sdcard/TGP-webapps/TGP-ROOT.war");
            webServer.setHandler(webapp);
        }
        catch (Exception e) {
            Log.d(LOG_TAG, "unexpected exception setting temp: " + e);
        }

DexWebInfConfiguration.java

When I started this class, I thought that there would be more to it so it is a regular class. I'll probably make it an inner class if it doesn't get any more complex that this. Right now, this is all that is needed to use the dex version of the servlets in my WAR.

/**
 * Adds dex support to the classloader used in the Jetty WebInfConfiguration.
 * 
 * <code>webapp.setConfigurationClasses(new String[] { DexWebInfConfiguration.class.getName() });</code>
 */
public class DexWebInfConfiguration extends WebInfConfiguration {
    public void preConfigure(WebAppContext context) throws Exception {
        context.setClassLoader(new DexClassLoader(context.getTempDirectory().getCanonicalPath()
                + "/webapp/WEB-INF/lib/classes.zip", context.getTempDirectory().getCanonicalPath(), null, getClass()
                .getClassLoader()));
        super.preConfigure(context);
    }
}

the WAR file

I'm using a simple Maven project within Eclipse for my WAR file. I'm using the standard structure and added the following to the POM. The first chunk pre-compiles the JSPs and then the second chunk runs dex on all of the compiled code in the WAR (i.e. the JSPs as well as the regular servlets).

The first chunk was pretty must straight out of the jetty-jspc-maven-plugin docs and the second from i-Jetty.

   <plugin>
    <groupId>org.mortbay.jetty</groupId>
    <artifactId>jetty-jspc-maven-plugin</artifactId>
    <version>7.3.0.v20110203</version>
    <executions>
     <execution>
      <id>jspc</id>
      <goals>
       <goal>jspc</goal>
      </goals>
      <configuration>
      </configuration>
     </execution>
    </executions>
   </plugin>
   <plugin>
    <artifactId>maven-war-plugin</artifactId>
    <configuration>
     <!-- use web.xml from jspc rather than src version -->
     <webXml>${project.build.directory}/web.xml</webXml>
    </configuration>
   </plugin>

      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>exec-maven-plugin</artifactId>
        <version>1.1</version>
        <executions>
          <execution>
            <id>generate-dex</id>
            <phase>process-classes</phase>
            <goals>
              <goal>exec</goal>
            </goals>
            <configuration>
              <executable>${android.home}/platform-tools/dx</executable>
              <arguments>
                <argument>--dex</argument>
                <argument>--verbose</argument>
                <argument>--core-library</argument>
                <argument>--output=${project.build.directory}/classes.dex</argument>
                <argument>--positions=lines</argument>
                <argument>${project.build.directory}/classes/</argument>
              </arguments>
            </configuration>
          </execution>
        </executions>
      </plugin>
      <plugin>
        <artifactId>maven-antrun-plugin</artifactId>
        <executions>
          <execution>
            <id>copydex</id>
            <phase>process-classes</phase>
            <goals>
              <goal>run</goal>
            </goals>
            <configuration>
              <tasks>
                <mkdir
                  dir="${project.build.directory}/${project.artifactId}-${project.version}/WEB-INF/lib" />
                <jar basedir="${project.build.directory}" update="true"
                  includes="classes.dex"
                  destfile="${project.build.directory}/${project.artifactId}-${project.version}/WEB-INF/lib/classes.zip" />
              </tasks>
            </configuration>
          </execution>
        </executions>
      </plugin>

switch to WebAppProvider

I thought that I'd just need to change the WebAppContext stuff but I ended up also having to adjust the configuration as well. Temp directories are handled differently in the provider stuff and I don't totally understand it. Here's where I'm at with the code that sets up the server and then the configuration.

MyApp.class onCreate

        // start the Web server for config and maintenance
        webServer = new Server(8080);
        try {
            HandlerCollection handlers = new HandlerCollection();
            ContextHandlerCollection contexts = new ContextHandlerCollection();
            handlers.setHandlers(new Handler[] { contexts, new DefaultHandler() });
            webServer.setHandler(handlers);
            
            DeploymentManager deployer = new DeploymentManager();
            deployer.setContexts(contexts);
            webServer.addBean(deployer);

            WebAppProvider webAppProvider = new WebAppProvider();
            webAppProvider.setConfigurationClasses(new String[] { DexWebInfConfiguration.class.getName(),
                    JettyWebXmlConfiguration.class.getName(), WebXmlConfiguration.class.getName() });
            webAppProvider.setExtractWars(true);
            webAppProvider.setScanInterval(10);
            webAppProvider.setMonitoredDirName("/sdcard/jetty/webapps/");
            deployer.addAppProvider(webAppProvider);
        }
        catch (Exception e) {
            Log.d(LOG_TAG, "unexpected exception setting temp: " + e);
        }

        try {
            webServer.start();
            Log.d(LOG_TAG, "started Web server @ " + getPublicInetAddress());
            
            NotificationManager notificationManager = (NotificationManager) getSystemService(Context.NOTIFICATION_SERVICE);
            Notification notification = new Notification( R.drawable.web_server_icon, "WebServer", System.currentTimeMillis());
            notification.flags |= Notification.FLAG_ONGOING_EVENT; // Notification.FLAG_NO_CLEAR;
            Intent notificationIntent = new Intent(this, MyApp.class);
            PendingIntent contentIntent = PendingIntent.getActivity(this, 0, notificationIntent, 0);
            notification.setLatestEventInfo(getApplicationContext(), "Web Server" , "Web Server Listening @ " + getPublicInetAddress(), contentIntent);
            notificationManager.notify(NOTIFICATION_ID_WEB_SERVER, notification);
        }
        catch (Exception e) {
            Log.d(LOG_TAG, "unexpected exception starting Web server: " + e);
        }

DexWebInfConfiguration.java

/**
 * Adds dex support to the classloader used in the Jetty WebInfConfiguration.
 * 
 * webapp.setConfigurationClasses(new String[] { DexWebInfConfiguration.class.getName() });
 */
public class DexWebInfConfiguration extends WebInfConfiguration {
    /**
     * The parent preConfigure is going to load the JAR and class files in so we
     * need to adjust the class loader before
     */
    @Override
    public void preConfigure(WebAppContext context) throws Exception {
        String tempDir = getCanonicalNameForWebAppTmpDir(context);
        if (tempDir != null) {
            tempDir = Environment.getExternalStorageDirectory() + "/" + tempDir;
            String dexZip = tempDir + "/webapp/WEB-INF/lib/classes.zip";
            context.setClassLoader(new DexClassLoader(dexZip, tempDir, null, getClass().getClassLoader()));
            Log.d("Jetty", "Added DexClassLoader for " + dexZip);
        }
        super.preConfigure(context);
    }
}

running embedded Jetty in Android app

I expect that we'll see this more commonly as more Android-based hardware devices come out other than mobile phones. In this particular case, the Jetty server is intended to drive a dynamic Web-based management tool for an application running on an Android device. I need to get a proper Web app project set up that pre-compiles the JSPs before they get to the Android device but this post is intended to show what libraries are required and how to set up Jetty to run on the Android device. I'm using a regular Android emulator out of Eclipse and a G1 running Cyanogen (API 8) for testing at this point.

General Stuff

I had to use Jetty version 7.3.0.v20110203 to avoid XML validation feature that causes troubles on Android 2.2 devices.

I merged JARs from the Jetty download into a single one to add to my app that I called jetty.jar. For example, here's a little shell script I used for my WAR Deployed Web App.

jar xf ../jetty-continuation-*.jar
jar xf ../jetty-http-*.jar
jar xf ../jetty-io-*.jar
jar xf ../jetty-security-*.jar
jar xf ../jetty-server-*.jar
jar xf ../jetty-servlet-*.jar
jar xf ../jetty-util-*.jar
jar xf ../jetty-webapp-*.jar
jar xf ../jetty-xml-*.jar
jar xf ../servlet-api-2.5.jar
jar cf ~/Documents/ECLIPSE_PROJECT_PATH/libs/jetty.jar *

I see that a work-around for the IPv4 / IPv6 issue is still required. I used

  // work-around for Android defect 9431
  System.setProperty("java.net.preferIPv4Stack", "true");
  System.setProperty("java.net.preferIPv6Addresses", "false");

Since the emulator uses slirp for the guest networking, you have to set up port forwarding from the host. For the G1, I used the real address of the phone (shows up in the status bar). So for the emulator I'd point my browser at http://localhost:8080/ but from my G1 I'd point at http://10.1.1.8:8080/ (of couse, this IP is local to my LAN).

adb -e forward tcp:8080 tcp:8080

Simple Web App

Just to make sure that I could start a server and have it listening as expected, I deployed a simple static app.

This is the list of JARs that I had to put into my app.

• jetty-continuation-7.3.0.v20110203.jar


• jetty-http-7.3.0.v20110203.jar


• jetty-io-7.3.0.v20110203.jar


• jetty-server-7.3.0.v20110203.jar


• jetty-util-7.3.0.v20110203.jar


• servlet-api-2.5.jar

Here is the Jetty server clip I put in my app onCreate

        webServer = new Server(8080);

        Handler handler = new AbstractHandler() {
            public void handle(String target, Request request, HttpServletRequest servletRequest,
                    HttpServletResponse servletResponse) throws IOException, ServletException {
                servletResponse.setContentType("text/html");
                servletResponse.setStatus(HttpServletResponse.SC_OK);
                servletResponse.getWriter().println("<h1>Hello World</h1>");
                ((Request) request).setHandled(true);
            }
        };
        webServer.setHandler(handler);

        try {
            webServer.start();
            Log.d(LOG_TAG, "started Web server @ " + getPublicInetAddress());
            
            NotificationManager notificationManager = (NotificationManager) getSystemService(Context.NOTIFICATION_SERVICE);
            Notification notification = new Notification( R.drawable.web_server_icon, "WebServer", System.currentTimeMillis());
            notification.flags |= Notification.FLAG_ONGOING_EVENT; // Notification.FLAG_NO_CLEAR;
            Intent notificationIntent = new Intent(this, MyApp.class);
            PendingIntent contentIntent = PendingIntent.getActivity(this, 0, notificationIntent, 0);
            notification.setLatestEventInfo(getApplicationContext(), "Web Server" , "Web Server Listening @ " + getPublicInetAddress(), contentIntent);
            notificationManager.notify(NOTIFICATION_ID_WEB_SERVER, notification);
        }
        catch (Exception e) {
            Log.d(LOG_TAG, "unexpected exception starting Web server: " + e);
        }

WAR Deployed Web App

The next step was to deploy an external servlet via a WAR file. This turned out to be a lot more difficult than I imagined because of checkin r2770 in Jetty-7. It turns out that an XmlParser validation feature was added to Jetty that Android 2.2 doesn't support. Once I moved to a version of Jetty older than that revision I was able to move forward.

This is the list of JARs that I had to put into my app.

• jetty-continuation-7.3.0.v20110203.jar


• jetty-http-7.3.0.v20110203.jar


• jetty-io-7.3.0.v20110203.jar


• jetty-security-7.3.0.v20110203.jar


• jetty-server-7.3.0.v20110203.jar


• jetty-servlet-7.3.0.v20110203.jar


• jetty-util-7.3.0.v20110203.jar


• jetty-webapp-7.3.0.v20110203.jar


• jetty-xml-7.3.0.v20110203.jar


• servlet-api-2.5.jar

The code:

        webServer = new Server(8080);

        WebAppContext webapp = new WebAppContext();
        webapp.setContextPath("/");
        webapp.setTempDirectory(new File("/sdcard/my-work/"));
        webapp.setWar("/sdcard/my-webapps/my-ROOT.war");
        webServer.setHandler(webapp);

        try {
            webServer.start();
            Log.d(LOG_TAG, "started Web server @ " + getPublicInetAddress());
            
            NotificationManager notificationManager = (NotificationManager) getSystemService(Context.NOTIFICATION_SERVICE);
            Notification notification = new Notification( R.drawable.web_server_icon, "WebServer", System.currentTimeMillis());
            notification.flags |= Notification.FLAG_ONGOING_EVENT; // Notification.FLAG_NO_CLEAR;
            Intent notificationIntent = new Intent(this, MyApp.class);
            PendingIntent contentIntent = PendingIntent.getActivity(this, 0, notificationIntent, 0);
            notification.setLatestEventInfo(getApplicationContext(), "Web Server" , "Web Server Listening @ " + getPublicInetAddress(), contentIntent);
            notificationManager.notify(NOTIFICATION_ID_WEB_SERVER, notification);
        }
        catch (Exception e) {
            Log.d(LOG_TAG, "unexpected exception starting Web server: " + e);
        }

The my-work directory gets created fine. I need to push the WAR file out: for the emulator I used adb push ../my-ROOT.war sdcard/my-webapps

Next Steps

• I need to submit a patch to Jetty to get a try / catch around the XmlParser validation feature that is causing trouble so we can use more recent versions.


• I need to set up JSP pre-compiling in my Web project so I can add JSPs to make developing the UI easier.


• I need to dig into the Android device and figure out how to set the low-port listen capability. Right now, I have to listen on the 1024+ port range so I use the common port 80. For a real device, I need to change this to port 80 but that requires a Linux capability change - probably in device boot parameters.

Here's the change that I think needs to be dis-armed a bit.

$ svn diff -c 2770 XmlParser.java 
Index: XmlParser.java
===================================================================
--- XmlParser.java (revision 2769)
+++ XmlParser.java (revision 2770)
@@ -107,6 +107,7 @@
             _parser.getXMLReader().setFeature("http://xml.org/sax/features/validation", validating);
             _parser.getXMLReader().setFeature("http://xml.org/sax/features/namespaces", true);
             _parser.getXMLReader().setFeature("http://xml.org/sax/features/namespace-prefixes", false);  
+            _parser.getXMLReader().setFeature("http://apache.org/xml/features/nonvalidating/load-external-dtd", validating);
         }
         catch (Exception e)
         {