Silfreed.net

Present: davidwboswell (David Boswell), ericjung (Eric Jung), gjm (Gerry Murphy), silfreed (Doug Warner)

Discussion was held publically in #mozdev

Discussed developer priorities

• setup mercurial commit logging last week
• upgraded (staging) mercurial to 1.0
• not much other mercurial work accomplished
• updated external links in the mozdev.org header
• released the secure update tool; fixed several minor bugs and updated documentation to make it a little easier to understand
• updated the mozdev.org policy to allow us to fix web errors (and fixed some minor errors for projects I had notified several weeks ago)
• working on updating application list so it can be more easily modified for sponsors
• will be setting up an ad system to track displays, views, clicks, etc for sponsors
• hope to finish up sponsor work by Wednesday, then work on testing SSL certs for Mercurial (and SVN)

Discussed sysadmin priorities

• cleaned up disk space sunday night
• log files have been moved to stats.mozdev.org for archival
• nagios notifications were disabled for drive stats; gjm will check into that

Web stats/Top 50 page

• still processing

Staging server migration

• mozilla said things are still moving for getting a VM there

Next meeting May 20th, 2008 @ 15:00 UTC in #mozdev

by Wellington Grey

• Call for hg testers - several project owners stepped up and said they'd test things out. I haven't heard any reports yet on how things work.
• Setup commit logging - Updated my svn2mysql tool to vcslog2mysql to work with mercurial as well. Other tools that rely on the cvslog need to be updated to work with the new tables.
• Upgraded Mercurial to 1.0
• Update external links in mozdev heading
• Released secure update tool
• SSL cert research for new Mozdev websites - Started looking into multi-domain UCC certs to support new Mozdev subdomains
• Updated Mozdev policy to allow for fixing web errors

This week I'd like to finalize plans for OSCON, I need to do some work for Mozdev's sponsors, and I'll continue working on getting Mercurial setup as an option for projects.

Present: davidwboswell (David Boswell), gjm (Gerry Murphy), silfreed (Doug Warner)

Discussion was held publically in #mozdev

Discussed developer priorities

• hg testing released; two projects have asked for testing repos already
• secure update.rdf generation released; there's been some additional interest on the list, but no direct questions
• sent policy update to PO list; no objections so far
• setup admin interface to select active VCS and initialize hg repository
• left for hg: logging of commits to db, writing documentation, figuring out how to deal w/ source.html, figuring out what to do w/ /source/browse

Discussed sysadmin priorities

• connection limitations are in place for CGIs (bugzilla, cvsweb, mailman)
• bots/spiders have been hard on our server recently so this was very helpful
• several software packages were updated on both servers

Web stats/Top 50 page

• still ongoing

Staging server migration

• no updates

Additional SSL Certs

• hg.mozdev.org will be needed
• look into bugzilla.mozdev.org?
• in the future we'll need svn.mozdev.org
• wildcard cert? We're not sure about the techincal details w/ that yet
• probably cheaper to keep buying individual certs for the near future (4x $30/yr < $200/yr wildcard)

Next meeting May 13th, 2008 @ 15:00 UTC in #mozdev

I ended up spending a large part of last week fixing some architectural problems I created with the ProjectVCS/VCS classes and debugging some test cases, but overall the admin page for selecting a project's current VCS and initializing a repository is ready (announcement coming soon asking for beta testers).

Other tasks last week included fixing the Drupal spam module so it notifies the project owner rather than Mozdev admins and cleaning up the change password form to reduce the number of warnings.

This week I plan on getting logging of Mercurial commits into our VCS database log and figuring out how to best publish the new settings for projects that use a different VCS.

Present: cdn-work (Chris Neale), davidwboswell (David Boswell), ericjung (Eric Jung), gjm (Gerry Murphy), silfreed (Doug Warner)
Community present: ccaygill, djc, JesperHansen

Discussion was held publically in #mozdev

Discussed developer priorities

• worked on the backend for choosing VCS and initializing mercurial
• didn't get much time to work on the admin interface; working on that now
• some more cleanup from the spam a week ago
• research into what the proper mime type for .rdf files should be (application/rdf+xml)
• fixed password reset/change forms (missed from php5 upgrade)
• been short on time the past couple weeks due to being down a car and being driver for the family
• Doug will look into any recent errors that should be fixed

Discussed sysadmin priorities

• mostly trying to find spam sources and potential spam sources on our server
• some minor non-user-visible config changes
• doesn't look like there is any spam being generated from hosted php scripts

drupal admin messages going to webmaster@

• spam notifications were fixed to go to the site email (which is set to the project owner) rather than user 1 (which is always webmaster@)

Web stats/Top 50 page

• top 50 page is still blank; web stats are missing for april
• processing choked on new log file format; restarted this morning

email delay

• email was blocked from Saturday to this morning
• virus scanner died

Staging server migration

• no updates

Minor updates for unmaintained projects

• still waiting for Doug to write a policy and post to the PO list; will try to address this week

pay drupal developer to write wiki page on configuring discussed defaults

• the documentation would help with existing projects and new installs until we have the time to update the default install profile
• more discussion needed

Next meeting May 6th, 2008 @ 15:00 UTC in #mozdev

• Working on admin interface for new VCS' (bug#18960, bug#18958)
• Cleaned up spam comments from recent spam attack
• Disabled several forms that could be used for spamming
• Researching proper mime type for .rdf files - Determined that application/rdf+xml is the proper mime type for update.rdf and install.rdf files despite finding other documentation that text/xml is all that's supported (thanks Mossop!)
• Fixed password reset/change forms

I didn't quite get as much done last week as I had hoped so I'm still working on getting the admin interface for making VCS changes finished.

Present: davidwboswell (David Boswell), ericjung (Eric Jung), gjm (Gerry Murphy), silfreed (Doug Warner), djc (autocopy extension owner)

Discussion was held publically in #mozdev

Discussed developer priorities

• spam attack last week has been cleaned up; old notes system is currently disabled
• mercurial is setup on staging at hg.vebzom.org; authentication is currently working against tigris database using existing CVS perms
• currently working on getting admin tools setup to select VCS and create repositories
• once admin tools are in place (hopefully this week) we can have devs test this on staging
• no response from community for secure updates testing; just going to release it as "beta" and deal with bugs later

Discussed sysadmin priorities

• spam attack ate up all the time last week
• logging of outgoing messages was broken since PHP 5 upgrade; fixed and improved
• gjm is monitoring abuse@mozdev now and replied to our upstreams about our recent spam attack
• some security patches were applied to servers
• copied production apache/nginx config to staging for Doug

Web stats/Top 50 page

• log analysis has been restarted (about a month behind)

Staging server migration

• no updates

Firefox updates server load handling

• handled firefox 2.0.0.14 release well

Cooperation between AMO and Mozdev.org on abandoned project adoption policy

• djc adopted auto-copy extension on mozdev.org bug would like to take over addon on AMO
• https://bugzilla.mozilla.org/show_bug.cgi?id=430048
• would be nice if Mozdev could work w/ Addons on this type of thing

ericjung would like to bring passwordmaker.org's mediawiki install back to mozdev.org

• should be possible; we'll need to work on the configuration a little bit

announce Mozdev.org sysadmin meetings?

• Doug will post the next meeting time in his minutes
• we should try to get the agenda for the upcoming week in a wiki for people to edit/comment

project status update form is being spammed

• gjm will post to sysadmin list to ensure it's safe to be disabled
• Doug will look into removing the form and links to it

Next meeting April 29th, 2008 @ 15:00 UTC

Much of last week was spent getting a test web interface setup for Mercurial. We also had a spam attack on our old notes system that required us to disable the script shortly after it was re-enabled. We're still trying to cleanup from that problem but it should be resolved shortly.

This week I plan on continue working on setting up Mercurial and working on the tools needed for supporting it.

I'm trying to hgwebdir.cgi setup and that part is going pretty well; I have a default config file that is being loaded by the script and serving a collection of repositories properly.

What I'm having problems with is configuring authentication. I'm using mod_auth_mysql to try to do authentication under Apache 2.2, but something is getting in the way.

I have the LoadModule mysql_auth_module ahead of all other auth* modules and AuthBasicAuthoritative Off set, but I still can't get authenticated.

Here's a snippet of our virtualhost to do Mercurial hosting; does anyone have any experience with mod_auth_mysql under Apache 2.2? Most things I'm finding on the 'net seem to indicate that it should work, but it's not for me.

        <LocationMatch /\w+/>
                # authentication
                AuthType Basic
                AuthName "Mozdev Mercurial Repositories"

                <IfModule mysql_auth_module>
                        AuthBasicAuthoritative Off
                        AuthMySQLAuthoritative On
                        AuthMySQLEnable On

                        AuthMySQLHost localhost
                        AuthMySQLUser authuser
                        AuthMySQLPassword authpass
                        AuthMySQLDB db

                        AuthMySQLUserTable "`users`"
                        AuthMySQLNameField "`username`"
                        AuthMySQLPasswordField "`password`"
                        AuthMySQLPwEncryption [method]
                        AuthMySQLUserCondition "active = 1"
                </IfModule>
                <LimitExcept GET>
                        Require valid-user
                </LimitExcept>
        </LocationMatch>

Present: davidwboswell (David Boswell), ericjung (Eric Jung), gjm (Gerry Murphy), silfreed (Doug Warner)

Discussion was held publically in #mozdev

Discussed developer priorities

• got mercurial plan together; started getting mercurial setup on vebzom
• deployed update.rdf generation, but need some testers
• various bug triaging
• we have some mod_rewrite bugs still lingering; trying to get my test suite fixed to track them down
• otherwise working on mercurial and update.rdf testing

Discussed sysadmin priorities

• still need to work on log rotation scripts
• fixing mod_rewrite bugs as they come up
• apache2 is serving http and https

Review roadmap changes

• MXR tool (replaces LXR) support, CVS, SVN, and hg
• updated roadmap will be announced to POs later today

Staging server migration

• no updates

Firefox updates server load handling

• no updates

• Triaging various site bugs after Apache 2.2/PHP 5 upgrade
• Putting together Mercurial setup plan
• Test suite improvements - testing various web pages on mozdev.org
• Moved secure update.rdf generation to production for testing - Thanks to bug#18732 being moved to production I was also able to get the update.rdf generation setup; we're still looking for testers, so please let me know if you're interested in helping.
• Trying to highlight Chatzilla extension support at Mozdev.org - We would like to see Chatzilla in our supported applications list, but its plugins aren't XPIs so support will probably need to be manually added.
• Backend changes to support stronger hashes for secure installations were completed - more work is needed to make support live, but the backend was put in place when I was working on supporting the updateInfoUrl for bug#18526

This week I plan to continue working on getting a Mercurial setup on vebzom while testing out the update.rdf generation.

XUL extensions are constructed very simply. Documentation on how they are constructed is excellent.

At Mozdev.org we have been focused recently on helping users find projects more easily as well as preparing for the upcoming Firefox 3 release by creating tools for our projects to perform secure installs and updates more easily. In order to do this we needed information about the extensions hosted at Mozdev.org.

Getting information about an extension is fairly straightforward. I created several classes for opening XPI files and parsing the install manifest. I also created classes to interface with extensions, extension types, applications, and application versions. These all get used when someone adds a new download file to their project (backend sql and data).

The process for adding a new file goes like this:

1. Our CVS download file update script will call MD_ProjectDownloadFile::add()
2. This function determines if the file is an extension. If it is, it:
   1. Parses the install manifest to get guid, name, description, and supported application information
   2. adds the extension if its new
   3. associates the extension with the project
   4. saves supported applications and versions for that specific file (supported applications can change over time)
3. The project owner can then login and publically release the file (this allows the project owner to have test releases as well as give the mirrors time to propagate the file) as well as verify the hash of the file (for secure installs and updates)
4. A project owner also has the option of using Mozdev's update.rdf file to provide secure updates; these are generated/updated when the extension is updated with a new release.

Since Mozdev.org doesn't want to modify our user's files we provide project owners a link to their update.rdf file for each extension so they can include it in their install.rdf when packaging their extension. For new extensions we provide a tool that allows them to upload their install manifest to get the path for the install manifest (or a sample install manifest that has it included)

So while there's a lot of little pieces to providing extension browsing by application or secure installs/updates, the code can be easily broken up in order to make the process easier.

Resources:

• Remora (software behind AMO)
• Install Manifests
• Extension Versioning, Update and Compatibility

Present: cdn-work (Chris Neale), davidwboswell (David Boswell), ericjung (Eric Jung), gjm (Gerry Murphy), silfreed (Doug Warner)

Discussion was held publically in #mozdev

Discussed developer priorities

• testing apache 2.2/php 5 changes
• created project tagging policy/docs
• started update.rdf generation for secure updates
• started setting up test suite for mozdev.org code
• this week is planned to be: finish up the hg setup plan (in progress), start working on hg, and try to get some sysadmin time to close some bugs (apache 2.2 rewrite bugs, web-visible cache directory setup)
• asked about working on mercurial before svn; davidwboswell says an update to the roadmap is coming soon

Discussed sysadmin priorities

• apache 2.2 and php 5 upgrade is complete; working on ironing out some bugs
• https is being setup and tested

Firefox updates server load handling

• no updates

Staging server migration

• no news on VMs
• server move should be highest priority now that apache/php setup is done

Other projects

• if staging server setup drags on too long, sysadmin might move on to cvs perms with pam auth
• project creation automation would be a good item to work on as well
• openid would be a nice authentication mechanism for users; integrating this with other auth mechanisms isn't understood well right now

Code testing

• mostly just unit tests now that PHP 5 is avaialble (PHP 4 couldn't do mock objects which was very limiting)
• Doug will be writing some tests to verify certain web paths are working correctly
• not really focused on full integration testing or continuous integration right now (the entire web stack isn't in version control, so we can't know when changes are made)

Last week was mostly monopolized by testing/debugging after our Apache 2.2 and PHP 5 upgrade. There's still some bugs to iron out, but it appears that the main site and most project sites are running well.

Other tasks include:

• Creating project tagging policy/docs
• Generation of update.rdf files for secure updates - not quite ready for production (waiting on bug#18732)
• Setting up test suite for Mozdev.org code (not in production yet)

I didn't quite get to writing out the plan for adding additional VCSes to Mozdev.org due to the short week and testing/debugging of our new web stack. I plan on getting started on that this week but most of my attention is going to be focused on getting our new Apache/PHP setup stabilized.