Perkakas Moderasi/Automoderator
|
Automoderator
|
Automoderator is an automated anti-vandalism tool under development by the Moderator Tools team. Perkakas ini memungkinkan kontributor untuk melakukan patroli suntingan secara otomatis dengan menggunakan model pembelajaran mesin (machine learning). Automoderator memiliki fungsi seperti ClueBot NG, SeroBOT, dan Dexbot. Tidak seperti bot tersebut yang hanya tersedia di Wikipedia bahasa Inggris, perkakas ini nantinya akan tersedia di seluruh proyek Wikimedia. Silakan lihat Extension:AutoModerator untuk rincian teknis tentang ekstensi AutoModerator.
Pekerjaan ini saat ini sedang dilacak sebagai bagian dari hipotesis WE1.3.1 WMF: If we enable additional customization of Automoderator's behavior and make changes based on pilot project feedback in Q1, more moderators will be satisfied with its feature set and reliability, and will opt to use it on their Wikimedia project, thereby increasing adoption of the product.
Communities can now request for Automoderator to be deployed on their Wikipedia.
Pembaharuan
- Oktober 2024 - A dashboard is now available to track metrics about Automoderator's behaviour on the projects on which it is deployed.
- Oktober 2024 - We have made new datasets available to test the Multilingual Revert Risk model. We anticipate that this model should have better performance than the Language Agnostic model which is currently in use, but need your feedback to make an informed decision.
- September 2024 - Indonesian and Ukrainian Wikipedias start using Automoderator (Automoderator; Автомодератор).
- Juni 2024 - Turkish Wikipedia starts using Automoderator (Otomoderatör).
- Februari 2024 - Desain telah diposting untuk versi awal halaman arahan dan konfigurasi. Pikiran dan saran diterima!
- Februari 2024 - Kami telah memposting hasil awal dari proses pengujian kami.
- Oktober 2023 - We are looking for input and feedback on our measurement plan, to decide what data we should use to evaluate the success of this project, and have made testing data available to collect input on Automoderator's decision-making.
- Agustus 2023 - We recently presented this project, and other moderator-focused projects, at Wikimania. Anda dapat menemukan rekaman sesi di sini.
Latar belakang
Sejumlah besar suntingan dibuat untuk proyek Wikimedia yang harus secara jelas dihapus, mengembalikan halaman kembali ke keadaan sebelumnya. Kontributor yang melakukan patroli maupun pengurus menghabiskan sebagian besar waktunya untuk memeriksa dan mengembalikan suntingan tersebut secara manual. Sayangnya jumlah kontributor aktif tidak sebanding dengan banyaknya suntingan masuk yang mengakibatkan beban kerja mereka semakin banyak. Maka dari itu, kami berinisiatif untuk sebisa mungkin mengurangi beban mereka agar bisa melakukan hal lainnya.
Beberapa situs lainnya seperti Reddit, Twitch, dan Discord menyediakan fitur moderasi secara otomatis. Fitur tersebut dikelola oleh komunitas untuk menentukan seberapa ketat penyaringan yang diperlukan dengan menggunakan model algoritme secara otomatis. Sementara itu, Wikipedia memiliki fitur filter penyalahgunaan yang dapat memberikan keleluasaan dalam menentukan filter penyaringan penyuntingan. Namun, perlu diakui bahwa fitur tersebut tidak mudah untuk digunakan oleh semua kontributor. Alasannya adalah filter penyalahgunaan hanya bisa diatur oleh kontributor yang memahami regular expression (regex). Fitur tersebut sangat sulit untuk digunakan oleh sebagian besar kontributor dan berisiko menyebabkan gangguan. Setidaknya beberapa komunitas memiliki akun bot yang bertugas untuk memberantas vandalisme, tetapi akun ini hanya beroperasi secara lokal dan dibutuhkan kontributor yang mumpuni dalam mengatur akun bot agar bekerja sesuai dengan keinginan. Akun bot tersebut umumnya menggunakan model ORES yang notabene tidak digunakan dalam jangka waktu lama dan keterbatasan dukungan bahasa.
Misi
- Mengurangi beban kontributor dalam melakukan patroli suntingan yang bersifat merusak.
- Meyakinkan kontributor bahwa perkakas ini dapat membantu melakukan patroli secara berkelanjutan.
- Memastikan bahwa kontributor dapat menangani kesalahan pemeriksaan (false positive) yang dibuat oleh perkakas ini.
Design research
To learn about the research and design process we went through to define Automoderator's behaviour and interfaces, see /Design.
Pendekatan
Proyek ini akan menggunakan pendekatan model risiko pengembalian yang dibuat oleh Tim Penelitian Wikimedia Foundation. Terdapat dua pendekatan yang akan digunakan dalam fitur ini:
- A multilingual model, with support for 47 languages.
- A language-agnostic model. This is the model which Automoderator currently uses, while we test the Multilingual model to better understand its performance.
These models can calculate a score for every revision denoting the likelihood that the edit should be reverted. Each community can set their own threshold for this score, above which edits are reverted (see below).
Saat ini pendekatan tersebut hanya dapat didukung di Wikipedia dan Wikidata, tetapi ke depannya akan diberlakukan di seluruh proyek Wikimedia. Selain itu, pendekatan tersebut hanya diuji coba di ruang nama artikel. We would like to investigate re-training the model on an ongoing basis as false positives are reported by the community. (T337501)
Before we moved forward with this project we provided opportunities for testing out the language-agnostic model against recent edits, so that patrollers could understand how accurate the model is and whether they felt confident using it in the way we proposed. The details and results of this test can be found at Perkakas Moderasi/Moderasi secara otomatis/Uji coba.
We are also testing the Multilingual model to understand if it is preferable to use it instead of the Language Agnostic model.
See Moderator Tools/Automoderator/Uji coba multibahasa to help us review the model's scores.
How it works
 | To request that Automoderator be deployed on your Wikimedia project, please see Extension:AutoModerator/Deploying. |
Automoderator scores every main namespace edit on a Wikimedia project, fetches a score for that edit based on how likely it is to be reverted, and reverts any edits which score above a threshold which can be configured by local administrators. The revert is carried out by a system account, so it looks and behaves like other accounts - it has a Contributions page, User page, shows up in page histories, etc.
To reduce false positives and other undesirable behaviour, Automoderator will never revert the following kinds of edits:
- An editor reverting one of their own edits
- Reverts of one of Automoderator's actions
- Those made by administrators or bots
- New page creations
Configuration
Automoderator is configured via a Pengaturan Komunitas form located at Special:CommunityConfiguration/AutoModerator, which edits the page MediaWiki:AutoModeratorConfig.json (the latter can be watchlisted so that updates show up in your Watchlist). After deployment, Automoderator will not begin running until a local administrator turns it on via the configuration page. In addition to turning Automoderator on or off, there are a range of configurations which can be customised to fit your community's needs, including the revert threshold, minor and bot edit flags, and whether Automoderator sends a talk page message after reverting (see below).
Certain configuration, such as Automoderator's username, can only be performed by MediaWiki developers. To request such a change, or to request other kinds of customisation, please file a task on Phabricator.
Localisation of Automoderator should primarily be carried out via TranslateWiki, but local overrides can also be made by editing the relevant Pesan sistem (Automoderator's strings all begin with automoderator-).
Caution levels
One of the most important configurations to set is the 'Caution level' or 'threshold' - this determines the trade-off Automoderator will make between coverage (how many bad edits are reverted) and accuracy (how frequently it will make mistakes). The higher the caution level, the fewer edits will be reverted, but the higher the accuracy; the lower the caution level, the more edit will be reverted, but the lower the accuracy. We recommend starting at a high caution level and gradually decreasing over time as your community becomes comfortable with how Automoderator is behaving.
Talk page message
To ensure that reverted editors who were making a good faith change are well equipped to understand why they were reverted, and to report false positives, Automoderator has an optional feature to send every reverted user a talk page message.
This message can be translated in TranslateWiki and customised locally via the Automoderator-wiki-revert-message system message.
The default (English) text reads as follows:
Hello! I am AutoModerator, an automated system which uses a machine learning model to identify and revert potentially bad edits to ensure Wikipedia remains reliable and trustworthy. Unfortunately, I reverted one of your recent edits to Article title.
- Because the model I use is not perfect, it sometimes reverts good edits. If you believe the change you made was constructive, please report it here.
- Learn more about my software.
- To learn more about editing visit your Newcomer Homepage. --Automoderator (talk) 01:23, 1 January 2024 (UTC)
If the same user receives another revert soon after the first, they will be sent a shorter message under the same section heading. Default (English) text:
I also reverted one of your recent edits to Article title because it seemed unconstructive. Automoderator (talk) 01:23, 1 January 2024 (UTC)
Pelaporan kesalahan pemeriksaan
Because no machine learning model is perfect, Automoderator will sometimes accidentally revert good edits. When this happens we want to reduce friction for the user who was reverted, and give them clear next steps. As such, an important step in configuring Automoderator is creating a false positive reporting page. This is a normal wiki page, which will be linked to by Automoderator in the talk page message, and in page histories and user contributions, as an additional possible action for an edit, alongside Undo and/or Thank.
Metrics
You can track data about how Automoderator is behaving on Wikimedia projects at the Activity Dashboard.
For data on the expected number of reverts that Automoderator would make per day on your project, see the testing subpage. Similar data for the multilingual model (not currently in use) can be found at /Multilingual testing.
Usage
Automoderator is currently deployed on the following Wikimedia projects:
| Proyek | Deployment request | Nama pengguna | Konfigurasi |
|---|---|---|---|
| Wikipedia bahasa Indonesia | T365792 | Automoderator | CommunityConfiguration |
| Wikipedia bahasa Turki | T362622 | Otomoderatör | CommunityConfiguration |
| Wikipedia bahasa Ukraina | T373823 | Автомодератор | CommunityConfiguration |
| Wikipedia bahasa Vietnam | T378343 |