volunteer-contributions-fetch

This library fetches contributions from several different sources and saves them in a JSON file. This can be used to track your own volunteer contributions. An example of this can be found on Michael's website.

Sources

Supported Sources

Currently the following sources are supported. Feel free to contribute a PR if you'd like to support another source!

• GitHub
  • Issues created
  • Comments on issues
  • PRs
  • PR Reviews
• Bugzilla
  • Bug reported
  • Bug commented on
• MediaWiki
  • Page edited
• Discourse
  • Topic created
  • Post created
• Mozilla Community Portal
  • Participated in event
  • Participated in campaign
• OpenStreetMap
  • Changesets

Each of the sources can be enabled separately. See the configuration section below.

Possible future sources

• Phabricator
  • Revision created
  • Revision updated

The following sources would be interesting for (at least) Mozilla contributors, however they do not expose an activity API endpoint to easily gather information for a certain user.

• support.mozilla.org
• Pontoon

Usage

First install the package from npm:

npm install volunteer-contribution-fetch

Then you can create a file in your project which runs the fetch operation and provides the necessary config:

import { fetchAll } from 'volunteer-contributions-fetch';

const config = {}; // See below for documentation and a sample config

fetchAll(config);

Usage without output file

This can also be used without having to rely on an outputFile. For this case the fetchAll function returns all contributions. Additionally you can also pass existing contributions to the function and these will be considered as well. The return value and the parameter of the function work the same as having an outputFile.

import { fetchAll } from 'volunteer-contributions-fetch';

const config = {}; // See below for documentation and a sample config
const existingContributions = [{ ... }];

const results = fetchAll(config, existingContributions);
// results is now all existing contributions plus any newly fetched contribution

Note that passing existing contributions this way will be ignored if a outputFile is used. Instead the existing contributions will be read from the outputFile if it already exists.

Configuration

You can configure the different providers with a config you pass into the fetchAll function call. These config parameters are outlined below. The config gets validated and you might get an error if you haven't specified all necessary properties.

Additionally to the configuration file, we also require some environment variables to be set:

• GITHUB_TOKEN with a GitHub token, if the GitHub source is enabled

While these tokens could potentially also be included the config, we want to make it hard to mistakenly publish secrets within code.

Configuration documentation

This section documents all the possible configuration values. For examples see the sample configuration below.

General

Field	Data Type	Required	Default	Description
outputFile	string	No	-	Path to contributions output file. Contributions are read from this file and new contributions will be added here.
bugzilla	Object	No	-	Bugzilla configuration options (see below)
communityPortal	Object	No	-	Community Portal configuration options (see below)
discourse	Object	No	-	Discourse configuration options (see below)
github	Object	No	-	GitHub configuration options (see below)
mediaWiki	Object	No	-	MediaWiki configuration options (see below)

Bugzilla

Field	Data Type	Required	Default	Description
enabled	boolean	Yes	-	Indicates whether this data source is enabled. Alternatively you can also leave off the mediaWiki object entirely.
baseUrl	string	Yes	-	API base url without trailing slash (for example: https://bugzilla.mozilla.org)
username	string	Yes	-	Username on MediaWiki
stopDate	string	Yes	-	new Date() parseable string containing a date of possibly earliest contribution to track
types	Object	No	-	Object containing the different labels below
types.createdType	string	No	Created a Bug Report	Label for entries related to having report a bug/enhancement.
types.commentedType	string	No	Commented on a Bug Report	Label for entries related to having commented on a bug/enhancement.

Community Portal

Field	Data Type	Required	Default	Description
enabled	boolean	Yes	-	Indicates whether this data source is enabled. Alternatively you can also leave off the communityPortal object entirely.
baseUrl	string	Yes	-	Community Portal base url without trailing slash
username	string	Yes	-	Username on the Community Portal
participationType	string	No	Participated in an event	Label for entries related to having participated in an event.
campaignType	string	No	Participated in a campaign	Label for entries related to having participated in a campaign

Discourse

Field	Data Type	Required	Default	Description
enabled	boolean	Yes	-	Indicates whether this data source is enabled. Alternatively you can also leave off the discourse object entirely.
baseUrl	string	Yes	-	API base url without trailing slash
username	string	Yes	-	Username on Discourse
topicType	string	No	Created Discourse Topic	Label for entries related to having created a new topic.
postType	string	No	Posted on Discourse Topic	Label for entries related to having posted on an existing topic.
keepDeletedPost	boolean	No	false	If a post gets deleted and can't be queried anymore, setting this to true will not delete it from the previous contributions.

GitHub

Field	Data Type	Required	Default	Description
enabled	boolean	Yes	-	Indicates whether this data source is enabled. Alternatively you can also leave off the github object entirely.
allowPrivate	boolean	No	false	Will throw if you pass a GITHUB_TOKEN with possible private repo scope. In most cases you will not want to have a token with private scope. Please think twice before enabling this config. When using fine-grained tokens, you need to set this to true manually as we can't detect the scopes. In that case, we can't guarantee anything!
commentsEnabled	boolean	No	true	Indicates whether fetching comments is enabled.
commitsEnabled	boolean	No	true	Indicates whether fetching commits is enabled.
issuesEnabled	boolean	No	true	Indicates whether fetching issues is enabled.
reviewsEnabled	boolean	No	true	Indicates whether fetching reviews is enabled.
username	string	Yes	-	Username on GitHub
stopDate	string	Yes	-	new Date() parseable string containing a date of possibly earliest contribution to track
filter	string	Yes	-	Regex to apply to each entry to filter for organization/repo names. For example: ^mozilla to only include orgs and repos that start with "mozilla".
types	Object	No	-	Object containing the different labels below
types.commit	string	No	GitHub Commit	Label for entries related to GitHub commits
types.privateCommit	string	No	Commit in private repository	Label for entries related to private GitHub commits
types.createdPR	string	No	Created PR	Label for entries related to opening GitHub PRs
types.createdIssue	string	No	Created Issue Report	Label for entries related to creating GitHub issues
types.commented	string	No	Commented on an Issue	Label for entries related to commenting on an issue
types.commentedPR	string	No	Commented on a Pull Request	Label for entries related to commenting on a GitHub PR (review)
types.approvedPR	string	No	Approved a Pull Request	Label for entries related to approving a GitHub PR
types.changesRequestedPR	string	No	Requested changes on a Pull Request	Label for entries related to requesting changes on a GitHub PR
types.reviewedPR	string	No	Reviewed a Pull Request	Label for entries related to reviewing a GitHub PR (fallback)
delayMsPerRequest	number	No	2000	Delay between each request to GitHub. You might hit rate limiting if you set this lower than default.

MediaWiki

Field	Data Type	Required	Default	Description
enabled	boolean	Yes	-	Indicates whether this data source is enabled. Alternatively you can also leave off the mediaWiki object entirely.
baseUrl	string	Yes	-	API base url without trailing slash
username	string	Yes	-	Username on MediaWiki
editType	string	No	Wiki Edit	Label for entries related to having edited an article.
stopDate	string	Yes	-	new Date() parseable string containing a date of possibly earliest contribution to track

OpenStreetMaps

Field	Data Type	Required	Default	Description
enabled	boolean	Yes	-	Indicates whether this data source is enabled. Alternatively you can also leave off the osm object entirely.
displayName	string	Yes	-	Username on OpenStreetMaps
stopDate	string	Yes	-	new Date() parseable string containing a date of possibly earliest contribution to track

Sample configuration

const config = {
  outputFile: `${__dirname}/_data/contributions.json`,
  bugzilla: {
    enabled: true,
    baseUrl: 'https://bugzilla.mozilla.org',
    username: 'me@michaelkohler.info',
    stopDate: '2008-01-01',
  },
  communityPortal: {
    enabled: true,
    baseUrl: 'https://community.mozilla.org/people',
    username: 'mkohler',
    participationType: 'Participated in an event',
    campaignType: 'Participated in a campaign',
  },
  discourse: {
    enabled: true,
    baseUrl: 'https://discourse.mozilla.org',
    username: 'mkohler',
    topicType: 'Created Discourse Topic',
    postType: 'Posted on Discourse Topic',
    keepDeletedPost: true,
  },
  mediaWiki: {
    enabled: true,
    baseUrl: 'https://wiki.mozilla.org/api.php?action=feedcontributions',
    username: 'michaelkohler',
    editType: 'Wiki Edit',
    stopDate: '2012-05-01',
  },
  github: {
    enabled: true,
    username: 'MichaelKohler',
    stopDate: '2008-01-01',
    filter: 'mozilla|common-voice|reps',
    types: {
      commit: 'GitHub Commit',
      privateCommit: 'Commit in private repository',
      createdPR: 'Created PR',
      createdIssue: 'Created Issue Report',
      commentedPR: 'Commented on a Pull Request',
      approvedPR: 'Approved a Pull Request',
      changesRequestedPR: 'Requested changes on a Pull Request',
      reviewedPR: 'Reviewed a Pull Request',
    },
  },
  osm: {
    enabled: true,
    displayName: 'foo',
    stopDate: '2010-01-01',
  },
};

Developing

Any issues or PRs are welcome! Before doing major changes, please create an issue elaborating on the approach.

Setup

npm ci

Tests

npm run eslint
npm test

Testing changes

First copy the example env file and adjust it to reflect your tokens.

cp .env.example .env

You then can use npm run example which uses fetchAll if you would like to test changes within this project. The configuration for this can be adjusted in example/index.js. A full sample config can be found in the configuration section above. Please make sure to not include testing config file changes when committing! The file is not ignored to make it so that new checkouts do not have to copy around a file first.

Deployments

Please use Angular Commit Message Conventions when contributing to this project, as depending on the commit message the GitHub Action will directly publish a new version of the package.