Extension:ORES

MediaWiki 拡張機能マニュアル

ORES リリースの状態: 安定
説明	This extension integrates data from the ORES project into the RecentChanges view.
作者	Kunal Mehta, Amir Sarabadani, Adam Roses Wight
MediaWiki	>= 1.43
データベースの変更	はい
テーブル	ores_classification ores_model
ライセンス	GNU 一般公衆利用許諾書 3.0 以降
ダウンロード	拡張機能をダウンロード Git [?]: Git master をダウンロード リポジトリを参照 (Phabricator · GitHub) コミット履歴 リポジトリの貢献者 (GitHub) コード レビュー
パラメーター $wgOresBaseUrl $wgOresLiftWingAddHostHeader $wgOresModelVersions $wgOresAggregatedModels $wgOresWikiId $wgOresLiftWingRevertRiskHostHeader $wgOresExcludeBots $wgOresUseLiftwing $wgOresModels $wgOresFrontendBaseUrl $wgOresCacheVersion $wgOresRevisionsPerBatch $wgOresLiftWingBaseUrl $wgOresEnabledNamespaces $wgOresFiltersThresholds $wgOresUiEnabled $wgOresModelClasses
使用するフック APIGetAllowedParams ApiQueryBaseAfterQuery ApiQueryBaseBeforeQuery ApiQueryBaseProcessRow ApiQueryWatchlistExtractOutputData ApiQueryWatchlistPrepareWatchedItemQueryServiceOptions BeforePageDisplay ChangesListSpecialPageQuery ChangesListSpecialPageStructuredFilters ContribsPager::getQueryInfo ContributionsLineEnding EnhancedChangesListModifyBlockLineData EnhancedChangesListModifyLineData GetPreferences LoadExtensionSchemaUpdates OldChangesListRecentChangesLine RecentChange_save RecentChangesPurgeRows SpecialContributions::formatRow::flags SpecialContributions::getForm::filters WatchedItemQueryServiceExtensions
提供するフック ORESCheckModels ORESRecentChangeScoreSavedHook
四半期ごとのダウンロード数	12 (Ranked 117th)
translatewiki.net で翻訳を利用できる場合は、ORES 拡張機能の翻訳にご協力ください
Vagrant role	ores
問題点	未解決のタスク · バグを報告

The ORES extension integrates data from an ORES service into the RecentChanges view.

Currently, the ORES backend service is only set up for Wikimedia wikis. Significant work would be required to set it up for third-party MediaWiki installations.

It is installed on several Wikimedia sites, but no longer deployed to new ones. For newer work on machine learning in Wikimedia, see Machine Learning/Modernization .

スクリーンショット

Changes that "need review" and may be damaging are highlighted in Special:RecentChanges.

A "needs review" flag is added to the Special:RecentChanges legend.

インストール

• ダウンロードして、ファイルをextensions/フォルダー内のORESという名前のディレクトリ内に配置します。
  開発者とコード寄稿者は、上記の代わりに以下を使用してGitからインストールします:cd extensions/
git clone https://gerrit.wikimedia.org/r/mediawiki/extensions/ORES
• 以下のコードを LocalSettings.php ファイルの末尾に追加します:

  wfLoadExtension( 'ORES' );
  

•  完了 – ウィキの「Special:Version」に移動して、拡張機能が正しくインストールされたことを確認します。

You need to run CheckModelVersions.php maintenance script once it's deployed (and after that you can run PopulateDatabase.php too)

If you want to setup a local development environment for MediaWiki + ORES extension using the following ORES extension local development guide :

環境設定の変数

Here's the config variables and their default values and a little description about them.

// URL of the ORES service
$wgOresBaseUrl = 'https://ores.wikimedia.org/';
// Either to exclude edits made by bot to score
$wgOresExcludeBots = true;
// Models to score
$wgOresModels = [
	'damaging' => [ 'enabled' => true ],
	'goodfaith' => [ 'enabled' => true ],
	'reverted' => [ 'enabled' => false ],
	'articlequality' => [
		'enabled' => false,
		'namespaces' => [ 0 ],
		'cleanParent' => true,
		'keepForever'=> true
	],
	'wp10' => [
		'enabled' => false,
		'namespaces' => [ 0 ],
		'cleanParent' => true,
		'keepForever'=> true
	],
	'draftquality' => [
		'enabled' => false,
		'namespaces' => [ 0 ],
		'types' => [ 1 ],
	],
];
// Will replace ORES with Lift Wing for fetching scores
$wgOresUseLiftwing = false;
// URL for Lift Wing - Skippeed if null
$wgOresLiftWingBaseUrl = null;
// Thresholds of different sensitivies in ORES
$wgOresDamagingThresholds = [ 'soft' => 0.7, 'hard' => 0.5 ];
// Namespaces the ORES should score. Empty array means all namespaces.
// If not empty, it will only works on the given namespaces.
// Determine namespaces like [ 0 => true, 120 => true ].
$wgOresEnabledNamespaces = [];
// Database id for ORES service. If not determined, it'll use database name.
// You can choose 'testwiki' that ORES service sends last two digits of rev_id flipped.
// For example: https://ores.wikimedia.org/v1/scores/testwiki/damaging/12345
$wgOresWikiId = null;

Debugging an ORES extension deployment

After we deploy the extension with either ORES or Lift Wing as a backend we can use the steps in the ORES extension debugging guide to make sure it is working fine.

ORES service responses

ORES extension is merely more than an interface to the ORES service. The service returns a probability score of edits being damaging like this (API v1):

{
  "724030089": {
    "damaging": {
      "prediction": false,
      "probability": {
        "false": 0.8917716518085119,
        "true": 0.10822834819148802
      }
    }
  }
}

It means this edit (diff=724030089) is 10% likely to have caused damage. Note that 90% likely doesn't mean 9 out of ten cases will be vandalism. Choosing thresholds should be done via analysing recall (percentage of vandalism it can catch) or false positive rate. In ORES the "soft" threshold is when recall is 75% (meaning it will include 75% of all damaging edits) and the "hard" threshold is when recall is 90%. You can get the thresholds from model info (an example).

データベースのスキーマ

ORES extension introduces two new tables: ores_model and ores_classification. See the full database schema description.

Extension workflow

Scores

Once an edit is made the extension triggers a job to hit the service and store the results in the ores_classification table. It means it will not include scores for edits made before the deployment. In order to fill the database you can run the maintenance script PopulateDatabase.php. It will hits the service and keeps the score for the last 5,000 edits. You can run it several times if needed.

Once a model gets updated to a newer version CheckModelVersions.php maintenance script needs to be ran to update the ores_model table which will cause to scores stored in the ores_classification table become deprecated. You can clean these obsolete scores by running PurgeScoreCache.php maintenance script.

インターフェイス

The extension won't show anything when deployed but it will add itself as a beta feature (Extension:BetaFeatures is a dependency of this extension) and once it's enabled by the user it will use hooks in ChangesList (RecentChanges, Watchlist, and RelatedChanges) in both old and enhanced mode and highlights when score exceeds the given threshold.