Supporting multi-touch events with GWT on mobile Safari

March 28th, 2010 Alex Moffat

The goal is to be able to handle the touchstart, touchmove, touchend and touchcancel events provided by mobile Safari by writing standard GWT code like the example below for touchstart.


        touchable.addTouchStartHandler(new TouchStartHandler() {
            public void onTouchStart(TouchStartEvent event) {
                event.preventDefault();
                addStyleDependentName("touched");
                Touch t = event.touches().get(0);
                offsetX = t.pageX() - getAbsoluteLeft();
                offsetY = t.pageY() - getAbsoluteTop();
            }
        });

I’ll walk through the steps needed to get this working, concentrating on the implementation needed for the touchstart event.

The easiest place to start is by building the EventHandler and Event subclasses. The code below is the TouchStartHandler. It’s just extends EventHandler.


public interface TouchStartHandler extends EventHandler {

    void onTouchStart(TouchStartEvent event);
}

The TouchStartEvent class is also simple. It implements the abstract methods getAssociatedType and dispatch defined by GWTEvent. I chose to implement a TouchEvent abstract superclass for the TouchStart, TouchMove, TouchEnd and TouchCancel classes following the pattern of the MouseDown, MouseUp etc events provided by GWT. The start of the TouchStartEvent class looks like


public class TouchStartEvent extends TouchEvent<TouchStartHandler> {

    private static final Type<TouchStartHandler> TYPE =
        new Type<TouchStartHandler>("touchstart", new TouchStartEvent());

    public static Type<TouchStartHandler> getType() {
        return TYPE;
    }

    // getAssociatedType and dispatch
}

Notice the value “touchstart” provided as the value of the eventName parameter to the DomEvent.Type constructor. This is used when hooking up the handlers so that they are called when touchstart events are fired by the browser. It does not have to be the native event name but things are less confusing if you follow the convention that it is.

A basic implementation of the TouchEvent superclass just provides access to the touches array of touch objects provided by the native JavaScript.


public abstract class TouchEvent<H extends EventHandler> extends DomEvent<H> {

    public JsArray<Touch> touches() {
        return touches(getNativeEvent());
    }

    private native JsArray<Touch> touches(NativeEvent nativeEvent) /*-{
      return nativeEvent.touches;
    }-*/;
}

I used overlay types to implement Touch so there’s no overhead. The fragment below just shows access to the pageX value.


public class Touch extends JavaScriptObject {

    protected Touch() {
    }

    public final native int pageX() /*-{
      return this.pageX;
    }-*/;

Now all these pieces are in place it possible to write the code that will add support for touch events to a GWT widget. For example you would be able to extend SimplePanel to create TouchPanel and add this method.


    public HandlerRegistration addTouchStartHandler(TouchStartHandler handler) {
        return addDomHandler(handler, TouchStartEvent.getType());
    }

As you probably expect this doesn’t work. To see why we need to take a look at the implementation of addDomHandler in the Widget class and understand a little about how GWT handles events. The critical line in addDomHandler is


  sinkEvents(Event.getTypeInt(type.getName()));

This is what eventually sets the ontouchstart property of the widget’s element.

Each different type of event supported by GWT has a corresponding integer code returned by Event.getTypeInt. Each code is a single bit so that the codes can be combined with an or operation as a bit field. The bit field is used when event handlers are added or removed to know which handlers are currently set on an element. Adding a new type of event requires that Event.getTypeInt return a new unique value for that type which works with the bit field mechanism and the values used for all the existing types. If you trace through Event.getTypeInt you end up at the implementation in eventGetTypeInt in DOMImpl. It’s just a large switch statement with case statements for each type value. For example


    case "load": return 0x08000;
    case "losecapture": return 0x02000;
    case "mousedown": return 0x00004;

The correct subclass of DOMImpl is chosen at compile time using deferred binding so to support these new events you need a subclass appropriate for mobile Safari. My post “The right way to add support for a new browser to GWT” explains how to do this.

I copied the code for eventGetTypeInt from DOMImpl into my DOMImplMobileSafari class and added new case statements for the new touch* event type values.


    case "touchstart": return 0x100000;
    case "touchmove": return 0x200000;
    case "touchcancel": return 0x400000;
    case "touchend": return 0x800000;

An alternative I experimented with was to override eventGetTypeInt with a non-native method, detect whether the native method returned an undefined value, which indicates no case statement matched, and use an extra switch statement to deal with the touch events. In my opinion the extra complexity wasn’t worth it, if GWT adds extra event types the overridden code will have to be investigated anyway to ensure the int values I’m using are still unique.

Next you need to look at the call to sinkEvents in Widget.addDomHandler. This is where the bit pattern returned from Event.getTypeInt is combined with the patterns from other events already being listened for by the widget to determine which event dispatch handlers should be registered. Tracing through the code leads to sinkEventsImpl in DOMImplStandard. This is the method that needs to be overridden in DOMImplMobileSafari. Each separate event has a corresponding if statement that checks whether the current call to sinkEventsImpl is targeting that event and if it is whether the dispatcher function should be added or removed.


    if (chMask & 0x00020) elem.onmouseout    = (bits & 0x00020) ?
        @com.google.gwt.user.client.impl.DOMImplStandard::dispatchEvent : null;

As with the eventGetTypeInt method I just copied the existing code and added new statements to the end for the touch* bit patterns, for instance


    if (chMask & 0x100000) elem.ontouchstart = (bits & 0x100000) ?
        @com.google.gwt.user.client.impl.DOMImplStandard::dispatchEvent : null;

With this change the solution is complete and the addTouchStartHandler presented earlier works correctly.

Posted in GWT | 13 Comments »

The right way to add support for a new browser to GWT

March 22nd, 2010 Alex Moffat

Well, at least if what you’re doing is supporting something that’s a variation of an existing case. As Thomas Broyer pointed out in a comment on my previous post I should have used the new conditional deferred-binding properties feature added in GWT 2.0. With this approach you define a new property that you use to detect the variation you’re interested in. Here I define a mobile.user.agent property, with values of mobilesafari and none, and a property provider that at runtime chooses between mobilesafari and none.


    <define-property name="mobile.user.agent" values="mobilesafari, none"/>

    <property-provider name="mobile.user.agent"><![CDATA[
      var ua = navigator.userAgent.toLowerCase();
      if (ua.indexOf("webkit") != -1 && ua.indexOf("mobile") != -1) {
        return "mobilesafari";
      }
      return "none";
  ]]></property-provider>

In the absence of any other information this will double the number of permutations because as far as the compiler is concerned the mobile.user.agent property can have either the mobilesafari or the none value for any user.agent value. The neat trick with conditional deferred-binding properties is that you can provide the GWT compiler with information to restrict the number of combinations. By using conditionals inside the set-property you can clamp the compile time value for mobile.user.agent to none for user.agent values other than safari.


    <set-property name="mobile.user.agent" value="none" >
        <none>
            <when-property-is name="user.agent" value="safari" />
        </none>
    </set-property>

This means that instead of having two values of mobile.user.agent for each value of user.agent you have one value for all user.agent values apart from safari. So, instead of doubling the number of permutations you just add one (in the case where you have a single locale). If you are supporting more locales the savings are even greater. With three locales you’d normally have 6 (user.agents) * 3 (locales) = 18 permutations. Add two values for mobile.user.agent and you have 6 (user.agents) * 3 (locales) * 2 (mobile.user.agents) = 36 permutations. Using condition deferred-binding you have (5 (user.agents != safari) + 1 (safari + none) + 1 (safari + mobilesafari)) * 3 (locales) = 21 permutations, saving 15 compilations to create JavaScript that would never be used at runtime because ie6 and mobilesafari is a permutation the property providers will never select.

Using mobile.user.agent also makes the overriding of behavior with deferred binding easier because you no longer have to re-implement the safari case for the mobilesafari browser, instead you can use the separate mobile.user.agent property to select the override where it’s needed.


    <replace-with class="com.google.gwt.user.client.impl.DOMImplMobileSafari">
        <when-type-is class="com.google.gwt.user.client.impl.DOMImpl"/>
        <all>
            <when-property-is name="user.agent" value="safari"/>
            <when-property-is name="mobile.user.agent" value="mobilesafari"/>
        </all>
    </replace-with>

So, until someone points out another better way that’s my solution for tweaking safari to support mobile safari.

Posted in EffectiveGWT, GWT | 3 Comments »

Adding support for a new browser to GWT

March 20th, 2010 Alex Moffat

GWT uses deferred binding based on the value of the user-agent property to select browser specific code at compile time. So, if you want to add support for a new browser, or add functionality specific to a variation of an existing browser, you’ll likely need a new user.agent value. The UserAgent.gwt.xml file defines the current user agent values using the define-property and property-provider elements. You can find UserAgent.gwt.xml in gwt-user.jar.

To add a new value to the user-agent property you need to use the extend-property element in your <module>.gwt.xml file, you can’t use another define-property element and add to the list because a property can only be defined once. Here’s an example that adds a “mobilesafari” value to the user.agent property.


<extend-property name="user.agent" values="mobilesafari"/>

The *.nocache.js file picks the appropriate permutation of generated JavaScript at runtime based on values of the various properties detected in the browser. So, in addition to adding the new property value with extend-property you have to arrange for it to be chosen when appropriate. This is where the property-provider element comes in. One of these is needed for each define-property. It contains the body of a JavaScript function that is evaluated as part of the execution of the *.nocache.js script and returns the value for the property. If you look in UserAgent.gwt.xml you can see the code currently used by GWT. In contrast to the define-property the property-provider element can be overridden by providing another definition with the same value of the name attribute. The last one found in the module include hierarchy is the one selected so all you need to do is add a property-provider element to your <module>.gwt.xml file.

I want mobilesafari to be chosen when the code is run in the browser used by the iPhone and iPod touch. If you look at the Browser ID (User-Agent) values sent by this browser you can see that the browser appears as Safari but with an extra “Mobile” in the string. I changed the code that was previously


return "safari";

to be


if (ua.indexOf("mobile") != -1) {
  return "mobilesafari";
} else {
  return "safari";
}

Once you’ve made these changes you’ll have a new permutation selected whenever a mobilesafari browser is used. Unfortunately this permutation will use the default implementations for all of the browser specific code where it should really be using a modified version of the Safari specific code. I’ll cover how to solve this in my next post.

Posted in GWT | 2 Comments »

Measuring AJAX roundtrip time with GWT

March 14th, 2010 Alex Moffat

Tools like Page Speed or YSlow are great for analyzing page load performance. You can use JavaScript profiling or simple timing statements to find problem areas in your code at runtime, bearing in mind of course that its more often than not DOM manipulation rather than pure JavaScript execution that is slowing you down. One other piece of data that is very useful is the roundtrip time for your requests to the server. Knowing the average roundtrip time and the distribution of times helps you decide how to chunk requests and sets a lower limit on the possible response time for a request. You can use the information when testing to help simulate the experience users at remote location will see, and knowing which users have slow connections or are experiencing network problems helps resolve problems.
Read the rest of this entry »

Posted in AJAX, GWT | No Comments »