Fast Flash in Fennec

By dougt | Published: November 29, 2009

For the last few days, we have been making Flash fast in Fennec.

Our situation in Fennec is that we have a hidden browser element that contains the flash object/embed element. When a paint needs to happen, we draw whatever the plugin wants to draw into a canvas or sent of canvas elements. These canvas elements are what the user sees on the screen — they are part of Fennec’s tile manager. Now these draws to the tile manager consistent of a 16bpp to 24bpp conversion (I am told that the flash is optimized for 16bpp), then a copy to a gfxXLibSurface, then a final blit to the screen. This final bit also contains a 24bpp->16bpp conversion because the screen is 16bpp. To make matters much worse, many plugins intersect multiple canvases in the tile manager which causes this drawing path to happen multiple times per video frame. The end result is that we were getting no more than 4-5 fps.

We took some incremental steps to improvement performance, but we were no where close to double digit fps.

At this point, we decided to just draw directly to the screen avoiding the tile manager completely. This allowed us to render without any conversions and only one copy — the plugin could write directly to X11 shared memory.

In making this decision, we would lose a bunch of information that Gecko provides such as where the plugin should be position relative to other elements, and how it should be clipped.  The solution we came up with was to let Fennec tell each of the object and embed elements where they should be drawn.  The frame painting code would honor the position and clip that the front end set.  In this way, we could have plugins do the right thing during pans and when content is below the Fennec sidebars/urlbar.

You can check out how Fennec positions elements here:

http://mxr.mozilla.org/mobile-browser/source/chrome/content/browser.js#2937

The end result in this work is that we get over 25fps when Sorenson encoded videos.  Over the next weeks and months, I hope to see more video content honoring Fennec’s user agent and provide optimized content.  (YouTube and other sites do not recognized Fennec’s UA, yet).

This fast path is only implemented on Maemo, but it can be implemented on any platform assuming the plugin has support to draw directly into a memory buffer.

You also can check out the details in bug 528551.

Posted in General, mozilla | Tagged , , , , , | 5 Comments

Geolocation support in add-ons

By dougt | Published: November 12, 2009

Today, I checked-in a set of changes that will allow mozilla addons the ability to acquire the location information. It is pretty simple to do:

geolocation = Cc["@mozilla.org/geolocation;1"]
.getService(Ci.nsIDOMGeoGeolocation);
geolocation.watchPosition(successCallback);

In this trivial example, the object |geolocation| is fully compatible with the W3C’s geolocation object that exists under the navigator object in web content. |successCallback| will be called when Firefox acquires your position.

The prompting of the user is left up to your application. All addons hosted on addons.mozilla.org must follow the guidelines and prompt the user.

If you have questions about what is permissible, please contact me.

Posted in mozilla | Tagged , , | 5 Comments

Location Privacy

By dougt | Published: October 26, 2009

A month ago, I was at the MetaPlaces conferences on a panel on Privacy. Dev, the moderator, asked a really interesting question: “what are you most scared about”. It is a very interesting question. Sitting next to me was the head lawyer of the CDT, someone who has giving me tons of grief about the w3c’s approach of geolocation on the web. He joke that my biggest fear should be not listening to his advise on this. However, I am a lot more scared of something else.

The previous day at MetaPlaces, I heard a lot about mobile advertising and the targeting operators can provide. What was most scary for me was the amount of information operators have and their use of this information to place you into a very detailed market segmentation… all of this without your expressed permission…

One company was able to take a two week data drop from an undisclosed operator and tell you the sort of lifestyle, socioeconomic status, age range, and other demographics of the phone owner. The data drop merely consisted of longitudes and latitudes of where the phone was at given times. Following a single phone you are able to known where the person lives, what kind of coffee he drinks, what area the person works, what stores he shops at, what their work hours are like, are they hitting clubs at night or are going home, and do they spend time at the library. And the user is aware that this sort of tracking is happening!

This is wrong. Operators should always be up front about this. The location data is yours. Where you take your phone, like who you call, is personal information.

Recently, a group of privacy advocates are calling on Congress to address some of these concerns. Some of the more interesting requests are:

* Sensitive information should not be collected or used for behavioral tracking or targeting.

* Individuals should be protected even if the information collected about them in behavioral tracking cannot be linked to their names, addresses, or other traditional “personally identifiable information,” as long as they can be distinguished as a particular computer user based on their profile.

* Individuals should have the right to confirm whether a data controller has their personal or behavioral
data, request such data, and delete it.

For more information about this effort, please check out the CDD press release.

Posted in General | Tagged , | Leave a comment

how i made the 192 pushlog ugly

By dougt | Published: September 30, 2009

in other words, how i screwed up a hg push. I wanted to push changes from mozilla-central to mozilla-1.9.2. The standard way is to:

=============================================================
how to apply a m-c committed patch to a release branch
=============================================================
hg qimport http://hg.mozilla.org/mozilla-central/raw-rev/
hg qpush
hg qref -e # add approval information
hg qfin .
hg push

What I did was basically the same thing, with a few shortcuts (insert your favorite story about how cutting corners can cause harm).

Basically I started out with a bug report. It has the information about what it fixed, who reviewed it, who approved it, etc. The first thing to do is verify that it all checks out and is a reasonable thing to do. In the bug, there usually a comment with the hg changeset information. Here was my first shortcut; I copied the url to the html version of the changeset from the bug, and run the qimport command. `hg qimport` has no problem with importing that patch (which should probably generate an error, but anywayz). What this did was imported a big html file — not what I wanted at all.

I then jumped over the `hg qref -e` command. This would have allowed me to see the changeset’s original comment. I would have been able to see that the qimport terribly failed. Also, this is a good idea to do so that you can add the appropriate a=foopy approval flag(s).

The last shortcut I took was I didn’t do a sanity check by running `hg out -p` which would have shown me what I was pushing. This is always a good idea to see what you are pushing. Sure reliance on the tools is fine and dandy, but actually seeing what hg thinks it is going to push doesn’t hurt and only takes a few seconds.

3 shortcuts resulted in a skidmark on the 191 push log:

http://hg.mozilla.org/releases/mozilla-1.9.2/rev/739c7a69d412
http://hg.mozilla.org/releases/mozilla-1.9.2/rev/1898753917d5

Sure there are worse things (e.g. patch bomb, pushing without testing, breaking trees), but I thought I would write this up both for clarification of how I messed up as well as a warning. :-)

Posted in mozilla | Tagged , | 3 Comments