XPI v2 – Making Extension Development Easier

Note: this is just me throwing some ideas around. It is not an official proposal or spec. Having said that, I would like everyone that has interest in extension development to read this post and tell me what they think.

I’ve been an extension developer for a long time, and I like to think that developing extensions is actually quite easy. Maybe it has to do with my C++/Java background, where setting up a development environment is much more involved than using a text editor and zipping some files together.
That doesn’t negate the fact that the Mozilla add-ons platform is old, and showing its age. There are a number of problems with it that make it hard to take the first steps into add-on development, and it’s amazing how little has been done to solve them over the past years. These are the top 3 in my mind:

• Extensions can’t be installed or uninstalled without a browser restart.
  • That’s bug 256509. Can it be fixed? Probably, but it would take a major development effort.
• The documentation about extension development is incomplete or outdated.
  • I’m working on this in the XUL School project, to be finished soon.
• Getting started with a basic “hello world” extension is too much effort: install.rdf, chrome.manifest, chrome JAR, content, skin, locale, defaults, OMG!

The last point is the one I want to tackle in this post. There’s a ridiculous amount of boilerplate, redundant coding to do even for the most basic of add-ons, specially if you’re making an extension that should be skinable and localizable. These are the specific problems I’ve identified:

• There are two manifest files: install.rdf and chrome.manifest, in completely different formats.
• They are both defined in relatively obscure formats. The install.rdf file is one of the very few places where RDF is still used in Firefox. Sticking to a dying format (at least in this context) is a very bad idea.
• The default chrome structure (with content, locale, skin, etc.) is too bureaucratic and inflexible, and almost completely redundant. Most add-ons have exactly the same structure, and having to define it every single time is unnecessary.

I think we can reimagine add-on packaging in way that simple tasks can be performed in the simplest of ways, and so that it can scale to be as fine-tuned as it is today. So here are my ideas.#1 Merge install.rdf and chrome.manifest into a single file. What format should be used? I think JSON is as good as any other, and Mozilla already includes a native and very fast JSON parser. This manifest file could also match the package.json manifests being used for Jetpacks. package.json is actually a pretty good name for the manifest file. Perhaps the same standard can be use for Jetpacks and other add-ons?

#2 Default to the root extension directory for chrome URLs in order to minimize chrome.manifest declarations. This means that a hello world extension could have this structure:

• helloworld.xpi2
  • package.json
  • overlay.xul
  • overlay.js
  • overlay.dtd

And the manifest file would be something like:

{
  id : “helloworld@xulforge.com”,
  name : “Hello World!”,  
  type: “2”
  compatibility :    
    { id : “{ec8030f7-c20a-464f-9b0e-13a3a9e97384}”,
      minVersion : “3.5”,
      maxVersion : “3.6.*”},     
  domain: “helloworldchrome”,
  overlays:      
    { source : “chrome://helloworldchrome/content/overlay.xul”,
      target : “chrome://browser/content/browser.xul”}
}

(Note: ‘compatibility’ and ‘overlays’ can be arrays when there’s more than one item. And the ‘domain’ value is a general declaration of the chrome domain.)

In this new system, you would be able to have all of your chrome files in the root directory and have no need for chrome directives other than declaring your main overlay. Also, in your root directory you can have a locale or skin folder that the system would know how to handle without any changes to the manifest. If a file isn’t found in the locale or skin folder, then the system falls back to the root directory as a last resource. Of course it would still be possible to declare specific locations for content, locale and skin in the manifest, in order to allow the “old style” to be used.

What about other special locations?

• platform and components will continue to have their special meanings.
• JSM files can be handled just the same as chrome files, except that they use resource:// instead of chrome://.
• The default preferences file should also be moved to the root and have a predetermined name, like defaultPrefs.js.

#3 Installing the package. When you install an XPI file, the file is unpacked in your profile directory. It is common (but not mandatory) practice for chrome files to be packed in a JAR file, which remains packed after installation. According to recent discussions about add-on performance, it’s more efficient to keep files packed together. Instead of requiring authors to use the internal JAR approach, I think it makes more sense to require authors *not* to use JARs, and then keep the packed XPI in the profile on installation. Files that need to be extracted, like the manifest (maybe) and binary components (according to this) can be extracted upon installation. This way it’s up to the platform and not the developer to look after performance.That’s it. Given that this new packaging system would be backwards-incompatible with the current one, it might make sense to change the file extension to something like XPI2, in order to make a clearer distinction between the 2. However it should suffice to look for the manifest file in order to identify the system in use.
How hard is this to implement? I think #1 and #3 are fairly simple to implement. #2 is the one that may present a bigger challenge, since the chrome URL system is something that runs deep in the Mozilla code, and changing its file location rules could cause breakage or vulnerabilities in non-add-on code. It might also be possible to limit the scope of these changes to add-ons only, but again, that may require lots of work. I’d love to hear the opinions of those who work on this part of the platform.

Some may wonder how does Jetpack fit into this whole idea. Well, Jetpack is a different platform, and it may very well replace traditional add-ons in the long term, but we’re still a long ways to go. We shouldn’t see Jetpack as some kind of competition, but as a lesson in what we can do better. If we improve add-on packaging to be closer to Jetpack packaging (I think this would be), then it’s a win for both, because it’ll make it less painful for developers to choose and switch between either platform.

I’d love to hear what experienced developers (both add-on and platform) think about this. I think there’s a real gain for novice developers in this if we were to implement it. I intentionally left out some details for brevity, but I’ll be happy to discuss them in the comments. Thanks for reading.