Skip to content

Add-on packaging spec, part 2

This is a specification for a project that will be implemented for the Google Summer of Code, and it can still change as feedback is received. There are no concrete plans of making this a part of Firefox in the near future. Read my previous post for more info.

Part 2 – URL resolution

Here’s a simple chrome.manifest file, as exemplified in part 1:

content   xulschoolhello              jar:chrome/xulschoolhello.jar!/content/
skin      xulschoolhello  classic/1.0 jar:chrome/xulschoolhello.jar!/skin/
locale    xulschoolhello  en-US       jar:chrome/xulschoolhello.jar!/locale/en-US/

overlay chrome://browser/content/browser.xul  chrome://xulschoolhello/content/browserOverlay.xul

There are two things I find silly about it:

  1. The file and directory structure is too complicated for a simple add-on. It could be simpler, but the form above is the current recommended way of doing things.
  2. If this is the default, recommended way of doing things, then why is it necessary to explicitly declare it?

This part of the specification aims to remove as much redundancy as possible from chrome (and other) declarations, as well as provide sensible defaults so that simple add-ons are simple to develop.

The simplest manifest

Let’s begin with the new manifest file introduced in the previous part of the spec:

{
  "id" : "helloworld@xulschool.com",
  "name" : "XUL School Hello World",
  "description" : "Welcome to XUL School!",
  "version" : "0.1",
  "authors" : "Appcoast",
  "homepageURL" : "https://developer.mozilla.org/en/XUL_School",
  "type" : "2",
  "targetApplications" : {
    "id" : "{ec8030f7-c20a-464f-9b0e-13a3a9e97384}",
    "minVersion" : "3.0",
    "maxVersion" : "3.6.*"
  },
  "domains" : "xulschoolhello",
  "overlays" : {
    "source" : "chrome://xulschoolhello/content/overlay.xul",
    "target" : "chrome://browser/content/browser.xul"
  }
}

A couple of notes before diving in:

  • There’s an ongoing discussion of what format is best for the manifest, JSON or some form of XML or HTML. Examples are being presented as JSON but could change eventually. If you want to join the discussion, please do so in Part 1.
  • The “domains” property is now plural, as it should be. (Thanks Wladimir!)

The highlighted sections are the equivalent of the chrome.manifest file presented above. The “domains” declaration covers the “content”,  “skin” and “locale” declarations in the old format. However, the file layout also needs to change, since the defaults for the new format are much simpler than in the old one. Here are the default resolutions for all URLs and other special folders:

  • “content” and “skin” URLs default to the root directory, where the manifest file is located. This means that the simplest of add-ons would consist of the manifest file and an overlay file zipped together. Complex add-ons would of course require directory structures in order to keep everything organized, but that’s left to the authors to determine.
  • “locale” URLs are a little more involved. If the author only wishes to ship the add-on in one locale, then the locale files can be located in the root directory, just like “content” and “skin” files. If there are more languages, they can be organized under “/locale/en-US/” and similar. The “locale” directory in the root would have that special meaning, and all directories within it are considered locale folders and are automatically registered. There’s no need for chrome declarations even in this case.
  • “resource” URLs will also default to the root directory, and will use the same domain. In the example extension above, “resource://xulschoolhello/hello.jsm” would point to the “hello.jsm” file in the root of the add-on package.
  • The “components” and “platform” folders will continue to have their special meaning.
  • The default preferences file will be named “defaultPrefs.js” and will also reside in the package root.

Advanced manifests

The defaults defined above should be adequate for most add-ons I’ve ever seen (and I’ve seen a lot!), but we can’t leave out add-ons that require more than this. It’s very important that the new system is as flexible as possible for authors, and that add-ons can be migrated to it without requiring significant changes or refactoring.
First of all, the “domains” attribute can be extended to include declarations similar to old-style manifests. Here’s the equivalent of the first chrome.manifest declarations in the first example:

"domains" : {
  "name" : "xulschoolhello",
  "content" : {
    "location" : "jar:chrome/xulschoolhello.jar!/content/"
  },
  "skin" : {
    "location" : "jar:chrome/xulschoolhello.jar!/skin/"
  },
  "locale" : {
    "locale" : "en-US",
    "location" : "jar:chrome/xulschoolhello.jar!/locale/en-US/"
  }
},

Some notes about this expanded format:

  • “domains” can be an array, in case multiple domains need to be defined.
  • The name of the skin is still redundant, so there’s no need to specify it.
  • “content”, “skin”, “locale” and “resources” can be arrays, to specify multiple entries.
  • “content”, “skin”, “locale” and “resources” support a “flags” attribute, in order to support manifest flags. This attribute can also be specified on overlays, overrides and style declarations, which are not exemplified but are the same as the overlay declaration shown before.

Moreover, it should be possible to change the locations of any of our “special” files and folders, in order to give developers maximum flexibility. For this, the manifest can have a “locations” attribute, exemplified here:

"locations" : {
  "defaultPrefs" : "prefs/myPrefs.js",
  "components" : "code/myComponents/",
  "platform" : "code/platform/"
},

The only file that has a mandated name and location would be the manifest file. The rest can be organized to the authors’ preference, provided they don’t like the defaults and they want to tinker with the manifest.

Manifest Localization

Localizing manifests is an arduous task. The most modern way to do it involves adding all the localized information in the manifest file, which isn’t yet supported by the most popular add-on localization community, Babelzilla. Keeping the manifest properly localized becomes a burden to add-on developers, and most don’t even bother.

The suggested solution in the new system is to take advantage of the existing localization infrastructure. The manifest localization file will default to the location “chrome://firstDeclaredDomain/locale/manifest.properties”, or can be explicitly defined (needs to be a chrome locale URL):

"localized" : "chrome://xulschoolhello/locale/manifest.properties",

This localized file would hold strings corresponding to the localizable attributes:

name = Hola Mundo de XUL School
description = Bienvenido a XUL School!
translator.1 = Jorge
translator.2 = Somebody else

The .1 and .2 suffix is just a way to handle multi-valued attributes.

Moving this information out of the manifest has one drawback: the add-on installer needs to do some additional effort to show these values during installation, since it will require to figure out the location of the localized file. This isn’t too hard to figure out using the manifest, though, and the advantages outweigh a few additional ms during installation.

Part 3 of this specification will cover add-on installation and any pending topics. It will also introduce a wiki page where the full spec will be fleshed out and all edge cases will be covered.

{ 8 } Comments

  1. johnjbarton | 2010/05/18 at 10:31 PM | Permalink

    Two suggestions Jorge:
    1) the compression expressed by the jar URLs would really be orthogonal to the lookup redirection. By that I mean that the manifest should specific how URLs are mapped to files and it should specify what directories are compressed, but these two things should not be mixed up. The developer should say
    content xulschoolhello content/
    and
    jar! content/
    then the system should work out the mapping and compression.

    2) the locales should not be in the zip file at all, but the should be loaded on demand directly from Babelizlla. The locales make the zip file huge, their updates are asynchronous anyway, and the packaging developer is very unlikely to be able to provide any realistic quality check anyway.

  2. Wladimir Palant | 2010/05/19 at 2:19 AM | Permalink

    There is an issue you are probably not aware of – resource URLs can be used from the web meaning that a web page can detect what resource namespaces are registered. This is the old (and fixed) trick using chrome URLs to detect installed extensions all over again. And it is also the reason why Adblock Plus explicitly doesn’t register resource namespaces even though those would be very useful. If we register resource namespaces for extensions by default we will make this issue worse than it currently is.

    The other issue concerns manifest localization. While using a properties file would be nice (I do that already, the data is being inserted into the manifest by the build tools) the issue is the chrome URL you are using there – it won’t be available if the extension is disabled. Of course the extension manager could resolve the chrome URL using the other data in the manifest but that code would be rather complicate and duplicate much of the logic of the chrome protocol handler.

    @john: I think that installing locales on demand requires a separate discussion. It sounds nice in theory but quickly becomes an unmanageable mess when you start thinking about details. Were you at the Firefox Summit 2008? Somebody (mfinkle?) asked a room full of extension developers about locales on demand – and was met with little enthusiasm. It’s a nice to have feature but most people don’t seem to care much about it. Oh, and I don’t know when was the last time that somebody complained about Adblock Plus download size – really, what are 300kB if the AMO download page weights whooping 540kB?

  3. Mook | 2010/05/19 at 3:49 AM | Permalink

    (I think I made a comment in the last post with JS disabled and it went into Akismet; would you be able to go retrieve it? Sorry about that.)

    The skin name, sadly, isn’t always redundant – see http://github.com/mook/windowshade/raw/e6fea916/chrome.manifest for am example (for the wrong app, but you should be able to get the idea). It’s required for extensions that wish to support non-default themes.

    Could you give an example where content/skin/locale has multiple entries? I couldn’t quite tell from the textual description what you mean, since they seem to fall under the same package name. (And “domains” confuses me, though I do see how it’s analogous. Do I get to register the “example.com” package, then?)

    I assume style and override directives will also be supported? I must be one of the few crazy enough to have used both at some point.

    Flags such as “contentaccessible” and “platform” still only applies to content packages? With this format, it would be possible to have flags for the package name (“domain”) directly, though it looks like you didn’t opt for it. That’s fine, just wanted to be certain :)

    The point of plurals makes me wonder – would it make sense to check both singular and plural (and dictate that one wins when both exist)? Then again, it’s probably too confusing and best avoided…

  4. Axel Hecht | 2010/05/19 at 8:51 AM | Permalink

    Technical detail, I don’t think the chrome registry supports resolving a url for a specified locale, not sure if that’s needed to display add-on details. Also, you’ll need to register the jar in question with the chrome registry before installing it, which sounds like a chicken-and-egg problem.

  5. malte | 2010/05/19 at 11:24 AM | Permalink

    I think the overlay declarations could be simplified by making the URLs (optionally, of course) relative to “chrome://xulschoolhello/content/” and dropping the “source” and “target” declaration:

    “overlays” : {
    “overlayBrowser.xul” : “chrome://browser/content/browser.xul”,
    “overlayMessenger.xul” : “chrome://messenger/content/messenger.xul”,
    “overlay.xul” : [
    "chrome://browser/content/browser.xul",
    "chrome://messenger/content/messenger.xul"
    ]
    }

  6. jorge | 2010/05/19 at 5:19 PM | Permalink

    @Everyone: thanks for the feedback!

    @johnjbarton:
    1) Part 3 discussed add-on installation, and I’ll cover some ideas I have about changing it. Spoiler: I want add-ons NOT to have inner compressed files and the installer to optimize the file layout (creating packages if necessary) on installation.
    2) That’s a nice idea, but I agree with Wladimir on the complexity of such an implementation, and the fact that most add-ons are < 1 Mb, which is rarely a concern these days. Plus, I doubt Babelzilla has the processing power to handle such a workload.

    @Wladimir Palant:
    You’re right, I didn’t know that. For the sake of keeping the default as easy as possible, I think I’ll add some way to revert this default behavior. Something like “resource” : { “disabled” : true }.
    Regarding manifest localization, I am aware that it is non-trivial to get the strings without registering the chrome package, but I also doubt it’s *that* complex. It’s still dirty, but on the implementation side, where it should be, not on the specification side annoying all developers.

    @Mook:
    I don’t see your post. I’m afraid it was lost in space. I’d really appreciate it if you posted again.
    The skin name will not be mandatory, but it will still be possible to use. I’m going to be very strict about supporting *all* non-obsoleted options available in install.rdf and chrome.manifest.
    I know the word “domains” is not very accurate, but I’m having a hard time finding a better name. Suggestions are welcome.
    To have multiple content, locale and skin entries, all you have to do is use an array or objects instead of a single object as a value. “content” : [ {something}, {something_else} ]. I guess I broke my own rule about plurals here… ugh. Oh well, I think the spec will need to be very clear about which attributes will be able to support one or multiple values. This is also another reason why XML might make things a little easier…

    @Axel Hecht: my intention was to code around that completely and just “hard code” finding that specific locale file. See my answer to Wladimir in this comment for more about it. It’s certainly doable, but not pretty.

    @malte: that’s a good idea, but you have to consider that add-ons can register more than one “domain”, so things get tricky there. One could say “let’s assume that if not specified, use the first domain”, but then the single-domain manifest and the multiple-domain manifest files would be too different from each other structure-wise. I also think it’s better for all attributes to belong to a known set of strings, so that it’s easier to validate a manifest file.

  7. jorge | 2010/05/21 at 2:51 PM | Permalink

    @Mook: found your comment and I just published it. Thanks!

  8. pjdkrunkt | 2010/06/13 at 6:26 PM | Permalink

    You could use “packages” instead of “domains” as that seems to be the most common way of referring to them. You could also use “manifest” so that it would be immediately recognizable as the replacement for chrome.manifest, although later on down the road it may be less obvious to new users what that means. Thank you for considering “advanced manifests”, this is an absolute necessity for add-ons that are compatible with Thunderbird and other applications that use different paths and package names than Firefox… it’s also a necessity for light-weight themes which only include a global or browser package and nothing else. In fact, I am not sure if it would even be possible to have a Theme install using the simple manifest without having several vestigial packages, i.e. Communicator and Help, or have these finally been dropped from Firefox?

Post a Comment

Your email is never published nor shared. Required fields are marked *