Expanded multi-touch example

May 10th, 2010 Alex Moffat Posted in GWT, Multi-touch | 1 Comment »

I’ve done some more work on my multi-touch example hosted at touchexample.appspot.com. As well as working with the fingers on the iPad/Pod/Phone it also works on Safari, Chrome and Firefox with the mouse. There is little code difference between the different implementations, it’s mostly hidden behind some GWT.create calls so that the majority of the code works with drag events. You can drag and drop the squares into various locations. As you drag a square the others nudge slightly horizontally or vertically to indicate where the dragged square will go if you drop it.

A few things I found interesting, or problems I ran into.

1. Knowing what to do with multi-touch is harder than writing the code to support it and multi-touch drag and drop is going to be hard to do well. When you have only one object moving around it’s fairly simple to work out how to move the rest of the objects on the screen in response. Multiple moving objects make this much more difficult. At the moment though you can drag multiple things at once in touchexample the rest of the animation really only works for one. It is likely the case that just because you can support multiple drags at once you probably shouldn’t in many cases. Of course the importance of user experience design isn’t really a surprise but it does bear thinking about.

2. On the iPad etc., and even on the iPad simulator, if you rapidly start a drag you’ll see that the dragged square moves correctly but the other squares don’t move out of the way as they should. As far as I can tell the events are being delivered to my code as they should be and the correct dom manipulations are being performed but things just aren’t moving. Move investigation is needed here.

3. Debugging problems like 2. are difficult. For most of the work I was able to use the mouse driven code and GWT dev mode. When you’re debugging on the iPad though I needed to use a logging based approach and stream the data to the server for examination.

4. The extra support for CSS3 that Chrome/Safari provides over Firefox 3.6.n lets you achieve some nice effects very simply. Just compare the way touchexample behaves. The animated motion of the non-dragged objects and the fade in and out of the grid is all done with -webkit-transition CSS properties. It’s very easy to get nice animation with no JavaScript/GWT coding at all.

5. No support for IE yet. I’ve not looked very hard but it’s probably just the canvas I’m using to draw the grid when an item is being dragged.

6. Weird focus artifact on Chrome. A small square to the right of the selected square appears when you put the press the mouse button.