Orientation in Firefox and beyond

One of the coolest apps I saw when the iPhone came out was the wooden balance game Labyrinth.  You basically tilt your phone to move a ball around the screen, avoiding holes, and trying to get the ball to a goal.  It made use of a feature of many modern devices — accelerometers.

Obviously clear that there should be a webapp for doing just that.  What was missing was a javascript API.

In recent Mozilla trunk builds, I have added support for an orientation event.  This new event will allow you to build applications and listen for changes in orientation.  (note, the first platform to support such an event is any MacBook Pro.  Others will follow).

Simple Call:

To use this new event, you will add an event listener as you normally would:

window.addEventListener(“MozOrientation”, orientationChange, true);

Your callback will be called, when there is a change in acceleration, passing the current orientation:

function orientationChange(o) {

}

Simple Result:

The passed object has 3 attributes – “x”, “y” and “z”.  Each value is between -1 and 1 where zero is the “balance point”.  For example, suppose you device is a MacBook Pro and it is sitting on a desk that is perfectly level, you would expect to see:

x = 0
y = 0
z = 1

x is the axis in the direction from the left side of the keyboard to the right side of the keyboard (basically the axis that is along the home row keys) is level.  If I lift up the left side of the keyboard, x will increase.  if I lift up the right side, x will decrease.

y is the axis in the direction from the front of the laptop (where the mouse is) to the back of the laptop.  If I lift up the front of the laptop (the side closest to me), y will decrease.  If I left the back towards the front, y will increase.

Got that?  Yeah, physics is pretty hard. :-(

x and y can easily be visualized.  If you have a recent trunk build for the Mac, try loading this demo page:

http://people.mozilla.org/~dougt/ball.html

z basically will tell you that the laptop is sitting right side up.  if z was -1, you would know that the laptop (probably closed) and is sitting on its screen.  Of course the value will change as you rotate the laptop / device in this direction.

Right now, there is only support for the Macbook Pro.  It is pretty easy to add support for different OSs.  We have code for Samsung Windows Mobile devices, and for the HTC Windows Mobile devices.  We still need support for linux and for Windows.  If you are interested in adding support file a bug and start looking at http://mxr.mozilla.org/mozilla-central/source/widget/public/nsIAccelerometer.idl

The API isn’t fixed and may change.  I do invite you to comment.  Keep in mind that we want a really simple and straight forward API to expose orientation events to web developers.  If your response has either “RDF” or “DCCI” in it, please reread the last sentence.  :-)

Thanks again and hope you enjoy.

This entry was posted in mozilla and tagged , , , , , , . Bookmark the permalink. Post a comment or leave a trackback: Trackback URL.

22 Comments

  1. Posted August 23, 2009 at 8:59 pm | Permalink

    Oh this is sweet! I sense many quirky games and applications taking advantage of this. Cool stuff.

  2. Marcin
    Posted August 24, 2009 at 1:44 am | Permalink

    Hi.

    The values you are getting are just pure values from the accelerometer sensor (fraction of earth’s acceleration constant G: value 1 from the sensor = force of 1G applied in that direction).

    I think that naming it ‘orientation’ is not a good idea. Orientation data should consist of yaw, pitch and roll of the device.

    yaw = rotation around Z axis
    roll = rotation around new X axis (after rotating around Z)
    pitch = rotation around new Y axis (after both previous rotations).

    There is a problem with getting Z axis rotation (yaw) on most of current devices, as there is no way to get it from accelerometer sensors. That’s why all HTC Android phones have both accelerometer and magnetic field (digital compass) sensors. Yaw is taken from direction towards Earth’s magnetic pole and pitch and roll from acceleration values. That is true Orientation data.

  3. Marcin
    Posted August 24, 2009 at 2:49 am | Permalink

    I forgot to mention two great examples of using orientation data in real world applications (both from Android powered phones).

    1. Google Maps Street View in Compass Mode which allows to watch photos from Street View by tilting and rotating the phone. It also uses digital compass to show a proper direction (when you are facing north then SV is also showing you this direction).

    2. Google SkyMap is a great example of using both GPS and Orientation data. It shows exact view of sky where you are (GPS), the direction you direct your phone at (Orientation) and at the time of watching. Great app for learning stars’ and constellations’ positions.

  4. Posted August 24, 2009 at 3:11 am | Permalink

    Nice! I hope Webkit will follow so we get this event on the iPhone.

    Does x^2 + y^2 + z^2 == 1?

    • Marcin
      Posted August 24, 2009 at 7:36 am | Permalink

      @Sjoerd: it is just a force of 1G applied to the device which can change direction relative to the device (as device is being tilted by user) so squares of all the values along 3 axes should sum to 1, but only when the device is not moved along those axes in any direction. You need to remember, that this isn’t an orientation sensor (e.g. digital compass or gyroscope) but ‘just’ an accelerometer.

      When you move device, then you apply additional, external force vector to device, which is added to gravitational force vector. Depending on the direction of move you can get values from 0 (for example when your device is free falling) to some big values (for example when you hit the device hard, or when it drops on the floor).

      When on the plane you can measure how many G is affecting you :) .

  5. Posted August 24, 2009 at 4:36 am | Permalink

    The next maemo devices from Nokia (codenamed RX-51, probably sold as N900 later this year) are not only phones, but said to have accelerometers, I guess it would be nice to get our hands on one of those to implement this feature :)

  6. Posted August 24, 2009 at 5:56 am | Permalink

    It would be nice if there was a way to get input from different accerometers in the future (e.g. seperate wiimotes)

  7. Michael Ventnor
    Posted August 24, 2009 at 7:00 am | Permalink

    Hi dougt,

    Great work on this. I recently managed to implement this for laptop Linux and Maemo 5, so I’ll file a bug for that soon.

  8. Posted August 24, 2009 at 7:23 am | Permalink

    Is there a bug number associated with this, so I can make sure it winds up documented at the appropriate time? :)

    Also, eventually there should probably be something done to standardize the values from the accelerometer instead of returning raw values from whatever hardware is at hand.

  9. Posted August 24, 2009 at 7:35 am | Permalink

    @Marcin – Do you think getting compass notifications is separate from orientation? Should we spec out these separately? Should they be different notifications?

    @stu – I wonder if we could somehow pass multiple orientation objects to the event callback in that case. Smaug would probably know!

    @Michael Ventnor – Awesomeness. Looking forward to reviewing those implementations and for your further feedback.

    @Eric Shepherd – https://bugzilla.mozilla.org/show_bug.cgi?id=485943. I posted this to WhatWG and the Device API WG at the w3c. Standardization is going to be a long process, and for now this event is mozilla specific (note the “Moz” prefix before the event name). Thanks for the docs!

    • Marcin
      Posted August 24, 2009 at 8:15 am | Permalink

      There are many types of sensors out there, so I think that there should be a way to access any of them separately.

      For example my HTC Magic has 3 physical sensors:
      - single value temperature sensor (I don’t really know what it exactly measures :) , maybe battery temp),
      - 3-axis accelerometer and
      - 3-axis ambient magnetic field sensor (digital compass).

      The chip also has a convenient, virtual sensor which acts as an orientation sensor. It takes values from both accelerometer and compass sensor and computes yaw, pitch and roll.

      Each of these sensors produces an event which contains variable number of values in an array (temperature sensor has only one, others in Magic have 3 values).

      Each sensor has its own unit:
      - accelerometer gives 3 values in Newtons (so it is a little different than in your solution, in Magic, when the device lays down on the desk, you get something about 9.8 instead of 1),
      - magnetic sensor gives 3 values in micro-Tesla
      - orientation sensor gives 3 values in degrees of rotation around each axis.

      I think that you shouldn’t register orientation event on window object. It would be better to provide some kind of factory or object registered in window object (window.sensors ?), that will allow to retrieve objects for specific sensors. For example you can get objects for all sensors of specific type. Then you register event listeners on those objects for each sensor separately.

      That way you can easily provide support for other types of sensors in the future.

      • Marcin
        Posted August 24, 2009 at 8:17 am | Permalink

        Sorry, my mistake. Accelerometer in Magic gives values in N/kg or m/s^2 :) .

    • skierpage
      Posted August 25, 2009 at 12:46 am | Permalink

      In August 2008 when I asked about heading in the context of the Geolocation WG’s spec (a third W3C group!), you responded “And yes, yaw-pitch-roll, acceleration, and tilt, are _not_ part of the spec. navigator.deviceOrientation is probably the right place for them.” I hope different W3C groups don’t end up standardizing overlapping sets of info.

      • Posted August 25, 2009 at 9:24 am | Permalink

        The initial approach I took was to add navigation.orientation. However, there was a bunch of push back and encouragement to use dom events instead.

        Right now, there are a bunch of proposed APIs for orientation by a bunch of different people. I think mine is the simplest and therefore best. :-) All of this will be ironed out when we begin to standardize this event and call it “orientation” not “MozOrientation”.

  10. skierpage
    Posted August 25, 2009 at 1:27 am | Permalink

    Very cool!

    How can JavaScript code check to see if MozOrientation events will ever fire? Is mHasAcceleration exposed, and does it mean “the computer has a bitchin’ accelerometer!” or merely “this window would like acceleration events”?

    This overlaps with “orientation” as in simple portrait/landscape switching. AIUI, there’s hardware that can do that without having an accelerometer, would/should it trigger MozOrientation or does the interface assume a fancy 3-axis accelerometer? MSDN’s article on screen orientation and rotation advises watching for WM_DISPLAYCHANGE messages and then checking screen dimensions to tell if a tablet PC has changed orientation, does that make sense as a fallback?

    • Posted August 25, 2009 at 9:28 am | Permalink

      In this implementation, there is no way to know if a MozOrientation event will ever fire.

      I think watching WM_DISPLAYCHANGE is something different. I think we want to look into the sensor APIs of Window 7. The display change is caused if change the monitor’s orientation — it is in one of two states (portrait or landscape). This really isn’t that interesting, is it?

  11. Jesper Kristensen
    Posted August 25, 2009 at 7:44 am | Permalink

    Can you give a hint on how to implement this? Can it be done as an extension? I know how to read acceleration from my ThinkPad (Windows), and would like to try it out in Firefox. (ThunkPad only supports the X and Y axis)

  12. Posted August 27, 2009 at 10:39 pm | Permalink

    I made a version of the demo that moves the ball every 30ms instead of only at the frequency of MozOrientation events. It feels a lot smoother, and only slightly laggy.

    http://www.squarefree.com/smoother-orientation-demo.html

  13. Melebius
    Posted August 31, 2009 at 12:59 am | Permalink

    It works also with a “regular” MacBook, not only Pro. Just tested with MacBook 3,1.

  14. Posted January 24, 2010 at 10:38 am | Permalink

16 Trackbacks

  1. By Firefox supports accelerometers - Mozilla Links on August 23, 2009 at 10:12 pm

    [...] Doug Turner announced that support for accelerometers has landed to Firefox’s main development repository, the [...]

  2. By Device orientation at Justin Dolske’s blog on August 24, 2009 at 1:21 am

    [...] just posted about the device orientation support he recently landed. Sitting here with my MacBook Pro, I just [...]

  3. By webFaktum » Firefox-Modernisierungen: Multitouch on August 24, 2009 at 4:54 am

    [...] anhand einiger Codebeispiele. Außerdem hat Doug Turner gerade in seinem Blog (http://dougt.org/wordpress/2009/08/orientation/) bekanngegeben, dass mittlerweile bereits erste Funktionen zur Erkennung der Rechnerlage in Firefox [...]

  4. [...] Turner de Mozilla ha anunciado que ha llegado el soporte para acelerómetros en el repositorio principal de Firefox, también conocido como trunk o [...]

  5. [...] also recently announced that it was adding support for orientation events to the development trunk for the Firefox browser. [...]

  6. [...] is tilted – you can find one in the Macbook Pro and the iPhone, amongst others – is getting some love from the Firefox developers at Mozilla, who have just added orientation support to the latest trunk [...]

  7. [...] way your device is tilted – you can find one in the Macbook Pro and the iPhone, amongst others – is getting some love from the Firefox developers at Mozilla, who have just added orientation support to the latest trunk [...]

  8. By FireFox will support accelerometer devices on August 28, 2009 at 5:41 am

    [...] to make their famous browser to support accelerometer devices in their next releases(versions). Doug Turner one of the developers of Firefox has confirmed that Firefox will accept inputs from [...]

  9. [...] can find more information on two posts by Doug Turner and an update on documentation for orientation from Eric [...]

  10. [...] site – with the direct link to the post timing out at the moment – and there are more posts on DougT’s [...]

  11. [...] site – with the direct link to the post timing out at the moment – and there are more posts on DougT’s [...]

  12. [...] site – with the direct link to the post timing out at the moment – and there are more posts on DougT’s [...]

  13. [...] can find more information on two posts by Doug Turner and an update on documentation for orientation from Eric [...]

  14. [...] Doug Turner: Orientation in Firefox and beyond on Planet MozillaTopics: MacBook Pro, MacBook [...]

  15. By Direct manipulation interfaces | Dao's City for Blog on February 21, 2010 at 1:38 am

    [...] user is accessing them via a WIMP system or a touchscreen device thanks to CSS media queries and Javascript orientation events in latest [...]

  16. [...] can find more information on two posts by Doug Turner and an update on documentation for orientation from Eric [...]

Post a Comment

Your email is never published nor shared. Required fields are marked *

*
*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>