Jump to content

Extension:BounceHandler

Issue tracker: #MediaWiki-extensions-BounceHandler
From mediawiki.org
Translate this page
Languages:
MediaWiki extensions manual
BounceHandler
Release status: stable
Implementation Hook , Database
Description Allows users to handle email bounces
Author(s)
Latest version 1.0 (Continuous updates)
Compatibility policy Snapshots releases along with MediaWiki. Master is not backward compatible.
MediaWiki 1.29+
PHP 5.4+
Database changes Yes
Tables bounce_records
License GNU General Public License 2.0 or later
Download
Download extension
Git [?]:
Parameters
  • $wgGenerateVERP
  • $wgVERPAcceptTime
  • $wgBounceHandlerSharedDB
  • $wgUnrecognizedBounceNotify
  • $wgBounceHandlerCluster
  • $wgBounceHandlerUnconfirmUsers
  • $wgVERPprefix
  • $wgVERPdomainPart
  • $wgBounceRecordMaxAge
  • $wgVERPalgorithm
  • $wgBounceHandlerInternalIPs
  • $wgBounceRecordLimit
  • $wgVERPsecret
  • $wgBounceRecordPeriod
Hooks used
Hooks provided
Quarterly downloads 1 (Ranked 133rd)
Public wikis using 847 (Ranked 296th)
Translate the BounceHandler extension if it is available at translatewiki.net
Issues Open tasks · Report a bug

The BounceHandler extension allows wikis to handle bounce emails efficiently, by:

  • Generate a VERP "Variable envelope Return-Path" on UserMailer::send email invocations.
  • Bounces can be directly fed to the bouncehandler API from the MTA using a curl POST request

As a result, users whose address is in error are un-subscribed and they are notified through Echo about this the next time they connect on the wiki. More exactly, their address is un-confirmed, and in case MediaWiki requires confirmed emails no emails will be further sent.

Installation

[edit]
  • Download and move the extracted BounceHandler folder to your extensions/ directory.
    Developers and code contributors should install the extension from Git instead, using:cd extensions/
    git clone https://gerrit.wikimedia.org/r/mediawiki/extensions/BounceHandler
  • Add the following code at the bottom of your LocalSettings.php file: 
    wfLoadExtension( 'BounceHandler' );
  • Run the update script which will automatically create the necessary database tables that this extension needs.
  • Yes Done – Navigate to Special:Version on your wiki to verify that the extension is successfully installed.

Configuration

[edit]

The extension requires the Mail Transfer Agent (MTA) installed in your mail server to HTTP POST the incoming bounce email to the extension API named as bouncehandler. This can be done by adding the corresponding configurations to your MTA configuration.

Adding bouncehandler router and transport configuration to Exim

[edit]

You can redirect all your bounce emails to the bouncehandler API directly to do the processing. You can edit your Exim configurations to route all bounce emails to a bounce-handler-router and do HTTP POST to the extension API, which is the recommended method.

Add this to your /etc/exim4/exim4.conf

Under the variable declaration section: 

VERP_BOUNCE_LOCALPART_REGEXP = \N^wiki-[\w.]+-\w+-\w+-[+/\w]+$\N

under router configuration: 

# Route bounce emails
mw_verp_api:
	driver = accept
	domains = +verp_domains
	condition = ${if match{$local_part}{VERP_BOUNCE_LOCALPART_REGEXP}{true}{false}}
	transport = mwverpbounceprocessor

under transports, we need to write the configuration that would HTTP POST to our bouncehandler API 

# POST VERP bounce emails to a MediaWiki 'bouncehandler' API 	822
mwverpbounceprocessor:
	driver = pipe
	command = /usr/bin/curl --interface 127.0.0.1 -H 'Host: mywiki.org' http://mywiki.org/api.php -d "action=bouncehandler" -d "format=json" --data-urlencode "email@-" -o /dev/null
	user = nobody
	group = nogroup

You can find more details in this blog-post "POST-ing bounce email to a Mediawiki API directly from exim".

Adding bouncehandler configuration to Postfix

[edit]

This section shows how the BounceHandler extension can be configured to handle bounced emails from Postfix, unlike the above section which shows how to do the same with Exim.

First of all, please make sure that Postfix & the package "postfix-pcre" are installed. Once these two installed, open up /etc/postfix/main.cf and add the following in the end of the file: 

virtual_alias_maps = pcre:/etc/postfix/virtual

Also, make sure that alias_maps is set to the following: 

alias_maps = hash:/etc/aliases

Save & close that file, and then create a new file called etc/postfix/virtual, , and add the following ode 

/wiki-[a-z0-9_.]+-[a-z0-9]+-[a-z0-9]+-[a-z0-9+\/=]+@yourdomainname.ext/  curl_email
You need to replace yourdomainname.ext with your own domain name.

This will tell Postfix that all the bounced emails (the ones which have return path matching with the above regex expression) should be passed on to the command specified in the alias 'curl_email'.

Now we need to define curl_email as our alias, so save & close this file, open up /etc/aliases and add the following: 

curl_email: "|curl --interface 127.0.0.1 -d action=bouncehandler -d format=json --data-urlencode email@- http://yourdomain.ext/path-to-mediawiki-install/api.php"
You need to provide the full domain name in place of yourdomain.ext and the full path to your MediaWiki Installation in place of /path-to-mediawiki-install.
If unlogged users don’t have the right 'read', the connections from 127.0.0.1 should anyway be granted the right 'read' to be able to submit bounces to MediaWiki API.

Save & close that file, and now type the following commands, to map your /etc/postfix/ to postfix, and then restart postfix: 

sudo postmap /etc/postfix/virtual
sudo postalias /etc/aliases
sudo /etc/init.d/postfix reload
sudo /etc/init.d/postfix restart

Save and close this file, and head over to your MediaWiki's LocalSettings.php and make sure add the following line: 

$wgJobRunRate = 0;

Once that is done, save and close this file. To test if this works, send an email to an invalid email to your server and then run: 

php maintenance/runJobs.php

To check if your email bounces were captured, check the bounce_records table in your MediaWiki database, and /var/log/mail.log for further testing.

API

[edit]

The BounceHandler extension installs an API bouncehandler to receive the HTTP POST from the mail server. The API has a parameter email to which the entire bounce email is URL encoded to. This helps in avoiding the use of a separate bounce collector inbox or other IMAP features for the extension to work.

Example API call:

http://example.org/api.php?action=bouncehandler&email=This%20is%20a%20test%20email This would send a bounce email with the body This is a test email for processing.

VERP address

[edit]

The extension creates a unique w:VERP address as the Return Path header to every single email send from the wiki installation. The generated VERP address is of the form 

wiki-testwiki-2a-nanrfx-Tn14EQZWaotS2XNn@verpwebhost.wmflabs.org

The general template of the generated VERP address is: 

$prefix-$wikiName-base36( $userID )-base36( $timestamp )-base64( hash( $algorithm, $key, $data ) )@$email_domain
Variable Description
$prefix The prefix applied to every VERP email. Default to wiki
$wikiName The name of the Wiki on which the extension is running
$userID The user id (int) of the recipient from the MediaWiki user table
$timestamp The unix time-stamp value at the time of the VERP address generation
$data The string resulting as the concatenation of $prefix,'-',$wikiId,'-',base_convert( $userID, 10, 36),'-' and base_convert( $timestamp, 10, 36)
$algorithm The hashing php-hmac algorithm used to prepare the hash of $prefix
$key The key used to prepare the hash using the hmac algorithm
$email_domain The domain part of the VERP email address

Parameters

[edit]
Configuration Default Description
$wgVERPalgorithm 'md5' The PHP hashing algorithm you need to employ to generate the VERP return-path address. Can be md5, sha256 etc. More details at [1]
$wgVERPsecret 'MediaWikiVERP' The secret key you need to pass to the PHP HMAC function
$wgVERPAcceptTime 259200
(3 days) 		The threshold time (in seconds) until we are expecting a bounce. Setting it < 3 days will make sure you respond only to the valid bounces.
$wgVERPprefix 'wiki' The prefix of the bounce addresses.
$wgVERPdomainPart null The domain part of the bounce addresses. null is equivalent to $wgServerName
$wgBounceRecordPeriod 604800
(1 week) 		The Bounce allowed period (in seconds). Setting it to a week makes sure that we consider a weeks bounce frequency before taking un-subscribe actions.
$wgBounceRecordLimit 10 Allowed bounces in the given time period.
$wgBounceHandlerInternalIPs [ '127.0.0.1', '::1' ] The internal IPs that are allowed to use the API. Make sure this is configured correctly, so that no outside user can cause unwanted email un-subscriptions.
$wgBounceHandlerUnconfirmUsers false Set to true to enable unsubcriptions when bounces are above the threshold.
$wgBounceRecordMaxAge 5184000
(60 days) 		The period (in seconds) where are kept the records of bounces. Older bounces records are deleted. false to deactivate deletion.
$wgBounceHandlerSharedDB false (to be documented)
$wgBounceHandlerCluster false (to be documented)
$wgUnrecognizedBounceNotify null Array of email addresses where unrecognized bounces are sent.
$wgGenerateVERP true Activate VERP addresses.
Retrieved from "https://www.mediawiki.org/w/index.php?title=Extension:BounceHandler&oldid=6636666"