Manual:Cloudflare

Cloudflare (cloudflare.com) is a commercial content delivery network with integrated distributed denial of service (DDoS) defence. As it acts as a reverse proxy and domain name server to your website, it can provide a useful IPv6 transition mechanism if your hosting provider doesn't provide native IPv6.

As with other reverse proxies (see Manual:Performance tuning#Caching proxy), there are configuration issues with any server that places itself in-line between the user and Mediawiki. Cloudflare needs to be told what to cache (and when to discard outdated content). Likewise, MediaWiki (or Apache) needs to be aware of any intervening trusted proxies to ensure that the user's IP address (and not that of Cloudflare's servers) is listed in Special:RecentChanges or logged by Extension:CheckUser.

Advantages and limitations

Cloudflare does not currently charge per megabit/second or per gigabyte or terabyte of data transferred; (as of 2014) the basic service, with some limitations, is free. For sites receiving high-volume requests for the same unchanged content (such as images, which typically account for more than 80% of bandwidth costs) placing a busy domain behind a service like Cloudflare can substantially reduce its cost of operation. As fewer requests are being made to the origin servers, in some cases the site may run faster.

Individual page view counters are broken under Cloudflare, as most requests never reach the origin servers and the individual wiki site owner does not have access to Cloudflare's logs. Cloudflare does provide analytics to tell how many visit your site, but not on a per-page basis.

The free version of Cloudflare is limited in its SSL capability. A paid version ($20/month per domain, no limit on number of subdomains) provides somewhat better SSL support, but content would still be decrypted and re-encrypted at Cloudflare's servers - a potential 'man in the middle' vulnerability.

Cloudflare takes control over DNS for your entire domain; this may be a problem if you use domains in which individual subdomains are shared between users with a service like freedns.afraid.org or if you use other services which expect to be your DNS provider. Cloudflare provides DDoS protection for web, but not for e-mail or other services; unless you remove any MX records pointing to your own domain, the addresses in these records can still be used to find your underlying origin server and target it for DDoS or abuse.

Cloudflare places your site behind multiple anycast servers in multiple countries. This can be an advantage if being geographically closer to your users makes your site appear faster, but can be a legal disadvantage for sites which are targets of libel tourism or are dealing in politically-sensitive matters. While censorship of content by Cloudflare is uncommon, it is not completely insulated from its home country's political climate; incidents in which Cloudflare has removed sites because of their content include Switter (a microblogging site for sex workers) and the Daily Stormer (an extreme-right political site). Likewise, a Wikileaks-like site might not want a US server in the data path if discussing sensitive information about US intelligence agencies; as Cloudflare expands infrastructure into other countries (including Russia) the number of governments to which it is potentially exposed only increases. A site discussing activities lawful in its own country but illegal in one or more of the Cloudflare server locations might wish to avoid using the service and keep their content distribution network at home.

Cloudflare does not perform well in China as the Great Firewall often blocks traffic from Cloudflare servers.

Like any third-party free or paid service, there is also the risk that what's free today becomes an expensive paid service tomorrow, becomes slow or unstable or simply disappears. Be prepared to switch your domain registration's DNS entries back to your original service (or another provider) if the 'freemium' Cloudflare service goes away in the future.

Integration with MediaWiki

MediaWiki's caching strategy is designed to work with open-source reverse proxy servers (see Caching proxy). On a single-server configuration, this looked like:

Outside world

⇔

Server
Squid/Varnish accelerator `w.x.y.z:80`	⇔	Apache webserver `127.0.0.1:80`

When an anonymous user requested a page, Squid/Varnish checked whether it had a copy already stored. If one was available, it was served directly without even querying the origin server. If not, the request would pass through to Apache and MediaWiki.

This saved significant amounts of processor time (as MediaWiki was no longer repeatedly regenerating the same HTML for frequently-read pages) but had various impacts on MediaWiki server configuration:

The address of the Squid/Varnish proxy had to be kept out of Special:RecentChanges, instead using the user's IP address from the X-Forwarded-For header.
The proxy servers had to be notified when a page was changed, so they could discard the outdated version. This was done from LocalSettings.php with entries like:
```
$wgUseCdn = true;
$wgCdnServers = [ '<your IPv4 address>' ];
$wgCdnServersNoPurge = [ '127.0.0.1' ];
```
For setups with MediaWiki 1.33 or older, replace Cdn with Squid.
The proxy had to avoid caching dynamic content which frequently or pseudo-randomly changes, such as special pages or output from some extensions such as Extension:DynamicPageList or Extension:RandomSelection.
Pages displaying a "You have new messages" banner or similar flags could not be cached
Pages for logged-in users (as they contain login names or identifying info) could not be cached
Protection against image hotlinking had to move out of Apache and onto whichever server was facing the user
IPv6 and HTTPS support also moved onto whichever server was facing the user.

MediaWiki sends HTTP headers (such as "cache-control: private" if something should not be cached, or an expiry time if something can be cached for a limited time) which are recognised by Squid/Varnish. Squid/Varnish sends "X-forwarded-for:" to indicate the IP of a user to a MediaWiki installation behind a Squid/Varnish proxy. MediaWiki also sends HTTP PURGE requests to notify Squid/Varnish that an item has changed and needs to be discarded.

As MediaWiki was designed to work with Squid/Varnish, the MediaWiki installation and the cache are tightly integrated.

Cloudflare in some cases provides a function which could serve a similar role, but implements it in a non-standard or incompatible manner. MediaWiki can operate behind Cloudflare, but there are some configuration issues to be addressed.

Anonymous IP user identification

If a user connects directly to a MediaWiki installation (on Apache), the user's IP address is reported by PHP in $_SERVER['REMOTE_ADDR'] and no further configuration is required to get the information into Special:RecentChanges

If a MediaWiki installation is behind Squid/Varnish, MediaWiki must be configured to use the X-Forwarded-For header and a list of trusted proxies configured in $wgCdnServers or $wgCdnServersNoPurge.

Cloudflare's proposal to patch MW 1.18 (not recommended)

If a MediaWiki installation is behind Cloudflare, that company's servers will present the user's IP in a non-standard format with CF-Connecting-IP and CF-IPCountry alongside Forwarded (or de-facto standard X-Forwarded-For).

Cloudflare at one point recommended changing core MediaWiki code to use $_SERVER['HTTP_CF_CONNECTING_IP'] but their proposed patch is, by their own admission, only suitable for "MediaWiki versions 1.18.0 and older". This is obsolete.

Installing mod_cloudflare in Apache

This leaves mod_cloudflare as a relatively easily-deployed option. This module, downloadable pre-compiled for various distributions from Cloudflare's site, is installed directly into Apache (not MediaWiki). It restores the original user's IPv4 or IPv6 address if Cloudflare is connecting to Apache directly. (If the MediaWiki/Apache server is sitting behind Squid, and that in turn is behind Cloudflare, the IP addresses in recentchanges will break).

A list of (originally) fourteen IPv4 and IPv6 ranges used by Cloudflare's servers was hard-coded in the module's source code. Cloudflare added additional IPv4 blocks to this list in late 2016 for new servers; sites with the outdated list still installed are now seeing Cloudflare's address in place of the user's address on a very intermittent basis.

Two recent RHEL/CentOS versions (6,7) are supported, as are a couple of Debian versions (7,8) and a few versions of Ubuntu (12,14,16). For these, Cloudflare distributes an .rpm or .deb file which installs mod_cloudflare as a precompiled binary. Cloudflare is no longer updating the RHEL/CentOS 5 .rpm's and the server list in these .rpm's is outdated.

Older OS versions may no longer be receiving updates to the mod_cloudflare packages; new major releases of distributions (such as CentOS 7, when it was initially released in 2015) are sometimes slow to be supported. Install an earlier version of mod_cloudflare into the Apache server bundled with a newer OS distribution and the web server likely will fail to start, throwing errors.

Cloudflare has no plans to offer .deb packages for Ubuntu18/Debian9, the current versions (as of 2018). Affected sites will need to compile mod-cloudflare from source files.

If you're running a compatible distribution, but the package manager refuses to install mod-cloudflare with "incorrect GPG key", reinstall the key and remove the old key using (for .rpm distributions):

rpm --import http://pkg.cloudflare.com/pubkey.gpg
rpm -e gpg-pubkey-8e5f9a5d-*

Configure Cloudflare IP addresses directly on MediaWiki

Since version 1.34, MediaWiki supports providing the list of trusted proxies in CIDR notation, so the list of Cloudflare ranges can be passed directly to MediaWiki for validation. As Cloudflare provides the client IP in the standard X-Forwarded-For, simply filling these values makes MediaWiki recognize them correctly. This is the recommended configuration for MediaWiki >= 1.34.

$wgUseCdn = true;
$wgCdnServersNoPurge = [ <list of Cloudflare ranges> ];

As the list of Cloudflare ranges varies over time, you will need to check periodically in order to update this setting.

Using the TrustedXFF MediaWiki extension

Another option is to use Extension:TrustedXFF and list every one of Cloudflare's IP addresses manually. The list is a long one and may change with time. See [1] for more information on recovering the original user address.

Add the list of Cloudflare's IPv4 and IPv6 addresses at the end of trusted-hosts.txt:

# Cloudflare
# https://www.cloudflare.com/en-in/ips/
103.21.244.0/22
103.21.244.0/22
103.22.200.0/22
103.31.4.0/22
104.16.0.0/13
104.24.0.0/14
108.162.192.0/18
131.0.72.0/22
141.101.64.0/18
162.158.0.0/15
172.64.0.0/13
173.245.48.0/20
188.114.96.0/20
190.93.240.0/20
197.234.240.0/22
198.41.128.0/17
2400:cb00::/32
2606:4700::/32
2803:f800::/32
2405:b500::/32
2405:8100::/32
2a06:98c0::/29
2c0f:f248::/32

Not a clean solution, but (until a proper Cloudflare extension is created for MediaWiki) this will get valid IPs into Special:Recentchanges and keep the addresses of the Cloudflare servers out of the logs... at least until the next time Cloudflare adds to its list of servers.

Removing old varnish

Any existing local caching servers (such as Squid or Varnish) will need to be removed when using mod_cloudflare; otherwise, mod_cloudflare will see Squid's address (which it doesn't recognise as a trusted proxy) and pass the request unchanged, at which point MediaWiki replaces the address of Squid with the next address upstream... Cloudflare. You can also try adding Squid to the list of trusted proxies by using the CloudflareRemoteIPTrustedProxy directive.

As MediaWiki typically blocks abuse by IP address, showing Cloudflare as the address in Special:Recentchanges will interfere with efforts to selectively block spam and vandalism.

Cache control

Cache Control Basics

By default, MediaWiki sets certain HTTP headers (such as Cache-Control: private for logged-in users, or Expires for cacheable pages) to control whether its dynamically-generated content is cached and when it should expire. For purely static content, such as image files, the headers are generated by the web server.

Cloudflare respects these by default [2].

You can check whether a given page is being cached by Cloudflare by looking at the headers returned from the server (i.e. in your browser's developer mode) and looking for the CF-Cache-Status header; CF-Cache-Status: DYNAMIC means "your webserver told me not to cache this".

MediaWiki's defaults in this regard are extremely conservative, however. By default, it basically says "no don't cache anything at all except images and CSS files" to upstream cache providers. If your goal is to use Cloudflare's cache to serve your site when your server goes down, this is not at all helpful.

To have MediaWiki allow Cloudflare to cache most pages (i.e. most stuff that isn't *fully* dynamic), you need to set the following in LocalSettings.php:

$wgUseCdn = true;  // used to be $wgUseSquid = true;

If you do that, then a Cloudflare "Cache Everything" page rule will actually cache most stuff; without that LocalSettings change, it'll just be ignore unless you also override Origin Cache Control, which is not a great idea because you'll cache too much.

Cloudflare Cache Control Overrides

From My Websites → settings → Page rules it is possible to create three rules per domain (more for paid users). Each matches a pattern (such as *example.org/wiki/* for all wiki pages on example.org and its subdomains, in standard view) and allows configuration of:

Custom caching - allows the cache control headers to be overridden, "Cache everything" is the most aggressive and may be OK for static image files but certainly not wanted for anything else. Special:Recentchanges or other dynamic content should use the least aggressive settings.
Edge cache expire TTL - in "cache everything" mode, overrides the time after which Cloudflare requests a new copy of a page or file from the origin server. If used on anything but static content (images), this will cache things which shouldn't be cached as it can override cache control directives from the origin server.[3]
Browser cache expire TTL - indicates the time after which the user's browser should request a new copy of a page from Cloudflare

There's also an "Always Online" (which causes Cloudflare to serve cached copies of your pages if the origin server is down) and an "Origin Cache Control" (which tells Cloudflare to honour cache control headers sent by the origin server, such as the "no-cache" on pages served to logged-in users).

It's best to cache static image files aggressively ("cache everything" would be valid for images, were a means provided to purge them when a new version is uploaded). Ordinary wiki pages should use settings which respect cache-control headers ("standard, origin cache control on" is OK, "cache everything" is problematic) and special: pages should not be cached or cached only briefly (so that they remain current).

For example, these settings:

/load.php?*
Always Online: On, Cache Level: Cache Everything, Browser Cache TTL: a year, Edge Cache TTL: a month
/wiki/*:*
Cache Level: Bypass
/wiki/*
Always Online: On, Cache Level: Standard, Origin Cache Control: On

would cause:

load.php to always be cached (which could easily reduce load on the origin server by 25-50%, as load.php serves CSS/JS which is constantly loaded on every page). Default settings are overridden to aggressively keep these pages in cache for the maximum amount of time.
wiki pages outside mainspace (such as Special: Talk: Project:) to never be cached, so that Special:RecentChanges and the like show current data
lastly, wiki pages in mainspace (which weren't matched by the previous rule) are cached as directed by the origin server (MediaWiki gives them a long expiry time with a "must-revalidate" status; pages served to logged-in users or displaying "you have new messages" banners must honour "no-cache" so they won't be served to subsequent visitors).

These settings presume that the standard/default behaviour for Cloudflare will cache obvious static files (.jpg, .png, .gif) but that Cloudflare needs to be explicitly told to cache output from load.php (which it otherwise would have mistaken for highly dynamic, variable content).

They are fairly aggressive, and may exhibit a couple of flaws:

Logged-in users may still be served cached mainspace pages in normal view as if they were logged out. While Cloudflare mentions a Bypass Cache on Cookie setting which could detect MediaWiki's login cookies and serve a fresh page to all logged-in users, that setting is only available on the most expensive paid plans, which (at hundreds of dollars per domain per month) are not a reasonable alternative for most small sites. If you do have the necessary plan though, make sure to set $wgULSAnonCanChangeLanguage and $wgULSLanguageDetection to false if you have Extension:UniversalLanguageSelector installed, to prevent Cloudflare from caching translated interfaces.
Uploads which replace an image by overwriting with another of the same name will not be detected; Cloudflare presumes this to be static content and caches aggressively. The standard MediaWiki+cache proxy setup allows MediaWiki to send a HTTP PURGE message to discard the outdated content; extensions such as CloudflarePurge allow MediaWiki to send an equivalent purge message to Cloudflare's proprietary API.

Effectively, it's possible to reduce load on origin servers by up to 75% by caching content with Cloudflare, but this comes with a risk of serving outdated pages or images.

Splitting Your Site

If you really want Cloudflare to cache most everything (i.e. you're relying on Cloudflare's Always Online feature, which means you need to cache aggressively as Cloudflare can't show a page to the world unless it's cached), but you also want editors to be able to do their job, a somewhat extravagant option is to make a second website, using the same back-end database, and give it a different URL, and have Cloudflare only cache the main site.

You can then use redirects to push editors to the "live" site, the one that Cloudflare isn't caching. The following works reasonably well for this in Apache:

On the normal site:

  # Send editors and people looking for special pages to the live site
  RewriteCond %{QUERY_STRING}           Special:
  RewriteRule .*                        https://live.mysite.com$0 [R=302,L]

On the "live" site:

  # Push non-editors to the cached site; don't send people on
  # Special pages back, though, because we send people from the
  # other site to here on Special pages and we don't want a loop.
  RewriteCond %{HTTP_COOKIE}            !mediawiki_session
  RewriteCond %{QUERY_STRING}           !Special:
  RewriteRule .*                        https://www.mysite.com$0 [R=302,L]

Example page rules on the Cloudflare side to go with such a configuration:

*mysite.com/*Special:*
Cache Level: Bypass

*mysite.com/*Talk:*
Cache Level: Bypass

*mysite.com/*
Always Online: On, Cache Level: Cache Everything

HTTP purge

When new content is uploaded to a MediaWiki, the wiki software will request that each cache server (as configured in $wgCdnServers ) discard the outdated version of the page or image. This notification is not sent to third-party web proxies (such as those listed in Extension:TrustedXFF) and appears to be hard-coded in SquidPurgeClient.php in core MediaWiki code with no hooks to allow an extension to change this behaviour.

The message, as sent by MediaWiki, looks like: PURGE http://wiki.example.org/images/0/01/Some_image_recently_replaced.jpg

Squid will honour this request if it comes from a trusted IP address. Cloudflare will likely not.

There is a means to manually request one outdated file be discarded (from 'My Websites' settings for one domain → Cloudflare Settings → Cache Purge, click "purge single file" and enter URL of file to purge) but this is cumbersome and wildcards are not supported.[4]

Cloudflare's API provides a function which can purge an individual file,[5] but there is currently nothing available to invoke this API automatically from MediaWiki. As a result, a wiki behind Cloudflare will continue to display outdated versions of content until the data expires from cache normally. See bugzilla:62356.

Cloudflare does not offer a direct equivalent to Squid's negative_ttl parameter, which controls how long an error code returned for a URL from an origin server remains in cache (negative caching) until the request is retried. If Cloudflare does "cache everything" with a long time-to-live, it may be necessary to manually purge URL's where Cloudflare has stored an error message but the error has since been corrected upstream.

Mobile pages

Extension:MobileFrontend provides a configuration in which mobile browsers are autodetected. This can play havoc with external caching schemes; while Varnish may be configured to auto-detect a mobile browser and serve a different version of the page, the Cloudflare cache may be blissfully unaware that this is happening – or even serve the wrong version. It's best to turn off auto-detection and serve the mobile version from a different subdomain (ie: mobile.www.example.org for the mobile version, if the desktop site is www.example.org).

To turn off Mediawiki's mobile browser auto-detection, try:

$wgServer = "//www.example.org";
$wgCanonicalServer = "https://www.example.org";
$wgMobileUrlCallback = fn ( $domain ) => "mobile.$domain";
$wgMFAutodetectMobileView = false;

This puts the mobile and desktop versions on separate subdomains.

Cloudflare can then be configured to provide mobile browser autodetection on their servers. Unfortunately, Cloudflare only provides this autodetection for the base domain (example.org) and the www. subdomain (www.example.org); any handheld mobile devices which visit these two portions of the site will be automatically redirected to some other subdomain (such as mobile.www.example.org) of the same domain. Any other subdomains (such as wiki.example.org or en.wiki.example.org) can be given mobile versions (m.wiki.example.org or m.en.wiki.example.org) but mobile browsers will not be autodetected by Cloudflare on anything but the base domain or the www subdomain. Visitors to en.wiki.example.org will instead need to click on the "mobile version" or "desktop version" links at the bottom of each webpage to select the desired version (an awkward limitation if you're serving multiple language wikis (or an entire wiki family) as subdomains of one main domain).

From the performance ("speed") tab on Cloudflare's web dashboard for each individual site, go to "Mobile Redirect", select a subdomain which contains the mobile version (for example, mobile.www.example.org), select "keep path" and turn the feature "on".

In the DNS and the Apache webserver configurations, the desktop and mobile sites point to the same MediaWiki installation; MediaWiki then examines the URL to determine whether Cloudflare is requesting the mobile version.

If MediaWiki doesn't correctly force the mobile subdomain into always-mobile mode, modifying ./extensions/MobileFrontend/includes/MobileContext.php to insert this additional check at the beginning of the usingMobileDomain() function may help:

        public function usingMobileDomain() {
                if (isset($_SERVER['SERVER_NAME']))
                   if ((substr($_SERVER['SERVER_NAME'],0,2)=='m.') || (substr($_SERVER['SERVER_NAME'],0,7)=='mobile.'))
                        return true;

...with the rest of the code unchanged. This should only be needed if traffic is reaching your m. or mobile. subdomain from clients which aren't being detected as mobile browsers.

If mobile users can no longer visit the normal desktop view through the desktop view link at the bottom of the page, modifying ./extensions/MobileFrontend/includes/MobileContext.php to change the cookies name and value may help:

At the top of the MobileContext class:

public const STOP_MOBILE_REDIRECT_COOKIE_NAME = '__cf_mob_redir';

At the middle of the setStopMobileRedirectCookie() function:

		$this->getRequest()->response()->setCookie(
			self::STOP_MOBILE_REDIRECT_COOKIE_NAME, '0', $expiry,
			[
				'domain' => $this->getStopMobileRedirectCookieDomain(),
				'prefix' => '',
				'secure' => (bool)$stopMobileRedirectCookieSecureValue,
			]
		);

At the middle of the shouldDisplayMobileViewInternal() function:

		$stopMobileRedirect = $this->getStopMobileRedirectCookie();
		if ( $stopMobileRedirect == '0' ) {
			return false;
		}

This method would require you to click the link twice to actually redirect to the desktop view, though.

Error 404 handling

By default, Cloudflare used to enable a "feature" which it brands as SmartErrors. This replaced the originating site's 'error 404' pages with a Cloudflare search page which listed other, related pages which do exist on the same site and allowed the user to search the web. This "smart error" page, which displayed in US English instead of the site's local language, may have contained advertising or directed users to obscure, little-known external web search engines.

MediaWiki, when given a request for a page which doesn't exist https://en.wikipedia.org/wiki/Like_this_one will display a custom error page. This page has the 404 code but invites the user to create the page (or indicates the page was deleted, protected, or "no such special page"). MediaWiki sites need this page left as-is so that clicking on a red link invites a user to create a new article. SmartErrors on every 404 page broke this.

Before 2017, SmartErrors could be disabled for an entire domain on initial import of the domain to Cloudflare or deactivated later on a per-domain basis from the list of domains by clicking 'Apps', scrolling down to "SmartErrors" and turning the app 'off'. They could also be turned off for specific paths using "page rules" in the settings for each domain. As of 2018, SmartErrors appears to have been removed from Cloudflare, resolving this issue.

Image hotlinking

Many web-based forums are infamous for encouraging users to link directly to images hosted on other sites. Their site appears to be displaying the image, but is using bandwidth that the 'hot-linked' target site's operators must pay for. A common defense is for Apache webmasters to deploy mod_rewrite to look at the Referer (sic) line of each request and reject any that hotlink images (.jpg, .png, .gif et al.) for use on pages on some other site.

With any form of caching server, the origin servers are no longer user-facing. Any anti-hotlinking code needs to be removed from the origin webserver (such as Apache); anti-hotlink protection is available on Cloudflare if needed.

HTML document comments

MediaWiki will normally add comments such as < Served by wonky-server.example.org in 666 seconds --> to HTML output for troubleshooting purposes; if a site has multiple Apache's delivering the same content, this is invaluable in determining which server is causing a specific issue. Cloudflare is prone to "optimising" delivered webpages by stripping these comments, a function which may need to be turned off at times for debugging purposes.

HTTPS

Cloudflare may be used as a transition mechanism to convert between (insecure) http: and (secure) https: – while this does not provide end-to-end encryption (as it leaves Cloudflare's server in a "man-in-the-middle as a service" position) it's better than no https: at all.

All versions of MediaWiki from 1.18 onward support protocol-relative URLs, so that replacing a link like http://wiki.example.org with merely //wiki.example.org when loading images and resources will work properly in both HTTP and HTTPS. This change is not specific to Cloudflare, but affects any MediaWiki site available in both HTTP and HTTPS.

IPv6

All current MediaWiki versions are able to log an IPv6 user address to Special:RecentChanges if one is provided. No changes to MediaWiki's LocalSettings.php is required, although MediaWiki (and any extensions which rely on user IPs, such as CheckUser) needs to be a currently-supported version as some older (pre-MW1.19) revisions were buggy in logging IPv6 user addresses to Special:RecentChanges

IPv6 support is enabled on whichever server is facing the user; Apache for a stand-alone install, Squid/Varnish for sites using these as a reverse proxy, Cloudflare's server for sites where Cloudflare is user-facing. There is no requirement that the communication from the user-facing server back to Apache support both IPv4 and IPv6 and no benefit to enabling both for any back-end link.

My websites → settings → Cloudflare settings → Automatic IPv6 may be set to "full" for each domain to enable IPv6 for your site.

No other configuration is required for IPv6.