Microsoft Research takes a second look at multi-touch mice interaction

Bringing multi-touch technology to the desktop mice might seem trivial but Microsoft Research doesn’t just want to stop at a couple gestures. A group of researchers there have been thinking of new ways to redefine the dated but familiar traditional mice interaction model with new multi-touch-enabled ones.

In their report “Design and Evaluation of Interaction Models for Multi-touch Mice“, they acknowledge that a multi-touch mice has the potential to combine the best of surface computing with the ergonomic benefits of a mice, however existing prototypes and products including the Apple Magic Mouse might not be utilizing the technology to their full potential.

Using the prototype hardware they built, they designed four interaction models around an interesting idea that not only would the cursor be displayed on screen but also multi-touch visualizers that showed the fingers activity on the sensor, including if they were “activated” and how they are positioned. This also allowed the “hand” position of the cursor to be separated from the “finger” interaction controls in a graphical user interface.

Unfortunately their experiments with a dozen participants concluded all of their proposed models were slower than using a multi-touch screen or a three-buttoned mouse, but their findings establishes some groundwork for future research and development in the area.

I’m sure Microsoft Hardware is taking notes.

19 insightful thoughts

  1. I can’t say I’m surprised… during the video I kept trying the gestures on my touchpad to see if I could get them down, and I was definitely one who kept changing my hand positions to accomplish the task, because I can’t comfortably move 2-3 fingers as effectively as a finger and thumb, for example. Considering most people hate touchpads I find it odd that people are even researching it.

    Touchscreens, on the other hand, are simple, because it’s direct interaction with the objects on screen. There’s no translation between what you’re doing to the touchpad/mouse and what you see on screen. This is also why a mouse works so well since you are able to focus on the cursor not the mouse… BUT, it is a reason new computer users struggle with mouse input for the first time as well.

    1. Mac users love their trackpads. There are many productive gestures that are easy to learn and the trackpads are huge and use the same chip as iPhone. The desktop version that was just released has sold very well, and many Windows users were excited by it because it has Windows drivers. But Windows doesn’t have most of the gestures, so it’s much less useful.

      But you learn the gestures one at a time as you work. If you sit some people down in a lab you will prove it doesn’t work, yet it’s working great for Mac users.

      1. “But Windows doesn’t have most of the gestures, so it’s much less useful.”

        The issue is driver support, not Windows lack of gestures.

  2. Microsoft Research is primarily a punchline. The fact that they are researching something Apple shipped long ago isn’t helping.

    1. I don’t think you paid attention. They are well aware of the Magic Mouse and even reference it in their report. Whilst a mouse + multi-touch gestures is cool, you have to think there might be something even cooler you can do with them.

    2. I dont think you’ve been paying attention to Microsoft Research that much either. If you’ve read Engadget & Crunchgear, MS Research has done a lot of multitouch mice way ahead than Apple’s Magic Mouse. It’s just that Magic Mouse came out first.

  3. The Apple Magic Mouse is usability impaired crap like almost all their keyboard/mouse products. Apple has a nearly unbroken track record in punishing its users through input devices. Remember the hockey puck? The people who designed that just hate computers and love shapes.

  4. When I was watching the video, I was thinking: Why would I want to use such a mouse? So, it wasn’t much of a surprise to find that the response times were slower than a traditional mouse or a touchscreen. Looking at a picture and description of the Magic Mouse on Wikipedia gave me a similar sense and I remember from my college’s computers that the Mighty Mouse predecessor was annoying.

    This makes me wonder if it is possible to make a mouse with multi-touch capabilities that has a net value added. There may be some way to do it, but it may just be a case where the platform doesn’t mesh well with new features.

    1. I tend to agree. Actually, I would like to see Microsoft expend more resources to let us leverage the upcoming slate/tablet devices in desktop environments. I could see myself wanting to use pen capabilities, intuitively move files between PC and slate, use the multitouch features for navigation/control and perhaps function as a SideShow type of device when in proximity of my desktop/laptop PC.

  5. After 4 years since multitouch was ‘invented’ by Apple, we should probably know that it has no good practical uses besides mundane tasks such as:

    – Rotating photos… but why when we usually only do 90 degrees rotation anyway?
    – Zooming photos
    – Navigating maps
    – Weird pinching, 3-finger, 4-finger, 5-finger or 12-finger gestures that do obscure things, and nobody will remember except hardcore users who might as well remember keyboard shortcuts
    – Finger-painting software

    …with the exception of:
    – Tons and tons of games
    – Live audio manipulation

    That’s pretty much it.

    For the rest of the tasks, we already had invented way better methods for accurate manipulation of objects.
    – A drawing software – better off with a pen tablet for accuracy
    – Illustration or CAD software – way more accurate with a mouse or a trackball
    – Text input – way more accurate handwriting detection with a pen
    – Word processing, spreadsheets, computer programming – use a keyboard

    It’s sad that Microsoft could never bring pen computing into fashion, as it is a much more accurate method when it comes down to more delicate tasks.

Leave a Reply