Mobile web/MobileFrontend deployment
 | This page is obsolete. It is being retained for archival purposes. It may document extensions or features that are obsolete and/or no longer supported. Do not rely on the information here being up-to-date. |
Status
[edit]| Target | Ruby Gateway | Mobile Frontend |
|---|---|---|
| Deployed | Yes | Yes |
| Default for all m. views | No | Yes |
| Serving "Mobile View" link | No | Yes |
| Squid redirect target | No | Yes |
| Default for all non Wikipedia projects | No | No (Unless you add the mobile tag) |
- Mobile frontend end has been deployed to production for in house testing and is available through both a link in the footer and with the useFormat=mobile url parameter.
- SQUID acls have been deployed (minus nokia) and were serving 302's to 100% of our web traffic. We ran a blog post earlier in the week to warn people about the change.
- Varnish cluster is in final prep scheduled to finish Mon/Tues.
- Opt in launch blog post TBD.
Schedule
[edit]- Finish backlog tasks
- Trial Deployment
- Production ramp up
Current Requirements for Trial Deployment
[edit]Backlog
[edit]| Task # | Description | Owner | Status |
|---|---|---|---|
| 1 | WURFL support in MW Memcache interface | Patrick | Done |
| 2 | Squid mobile redirector | Patrick | Done |
| 3 | Trial Rollout | Asher | Done |
| 4 | Mobile Varnish cluster | Asher | Done |
| 5 | Varnish multicast cache purge support | Asher | Done |
| 6 | Varnish udp2log patch | Patrick | Done |
| 7 | Install Mobile frontend in production | Patrick | Done |
| 8 | Proxy frontend on Ruby-based mobile servers | Asher | Done |
| 9 | Create deployment packages for #2, #6 | Asher | Done |
| 11 | Enable ACLs on production Squids | Asher | Done (note: Nokia ACL pending resolution of 29505 |
| 12 | Enable Mobile frontend in production | Patrick | Done |
Current Requirements for Prod Deployment
[edit]| Task # | Description | Owner | Status |
|---|---|---|---|
| 1 | Order LVS hardware | Asher | Done |
| 2 | Rack and release hardware | Asher | Done |
| 3 | Open Bugs | Patrick | Bugs |
Detail
[edit]WURFL support in MW Memcache interface
[edit]Need to update php wurfl api to use mediawiki's memcache interface -- Blocker- Converting from using the php memcache extension to the built-in mediawiki memcache interface for the WURFL php api.
- Special:Code/MediaWiki/91768
Squid mobile redirector
[edit]Squid mobile redirector - the squid config (acl's for mobile device and opt-out cookie detection) is finalized and tested with the redirect logic implemented in perl. The perl needs to be ported to C and ensured to be fast, ram efficient, and leak free. Simple but due to risk involved in impacting nearly all wikipedia requests, may need a full day or more of testing.
Trial Rollout
[edit]- How do we want a trial version of the new mobile site to be accessed? x% of requests to the current mobile vip, or new vips with test names (i.e. en.newm.wikipedia.org)
- If we want host names to stay the same and run a cookie based trial (i.e. users transparently get the new site when going to en.m.wikipedia.org if I have the test cookie set) :
- We need a proxy sitting in front of the ruby mobile servers (such as nginx or haproxy) on port 80
- Reconfigure ruby apaches to port 81
- Proxy configured with two backend pools - one is just localhost:81, the other is the individual new-mobile varnish hosts
- Dispatch based on cookie
- This isn't complex but if the new mobile varnish cluster is in eqiad, it might add up to 120ms of latency from multiple back and forth hops across colos
- New vip - what do we name the base? *.nm.wikipedia.org?
- LVS config for the new vip and varnish hosts behind it (need Ryan's help)
- DNS for test versions of all wikipedia sites
- Deploy the new mobile redirect logic to the squids, with the C redirector modified to use the test domain name
- Add a squid acl to only invoke the redirector if the cookie is set (this should be possible but need to verify)
- In either case, need to update mobile vcl to rewrite chosen hostnames back to $lang.wikipedia.org to be supported transparently by apache / mediawiki.
Mobile Varnish cluster
[edit]- Mobile Varnish cluster - waiting to hear from Mark if we can piggyback on and expand the bits cluster
- If Yes: Need to reconfigure varnish to use all available ram, likely need to put nginx in front, mobile vcl needs to be updated to play nicely with the current version (don't execute at all for bits requests)
- If No (this might actually be better): we can use the 5 mobile servers ordered in Ashburn for Varnish (may need RobH to complete server builds)
- Need nginx with consistent uri hashing - that module probably needs to be built from source and packaged into a dpkg for deployment
- Puppet rules to build mobile cache servers (nginx w/hash module on 80 and varnish on 81 with/mobile vcl)
- Nagios monitoring for varnish on a non-standard port?
Varnish multicast cache purge support
[edit]- Varnish multicast cache purge support - requires running a daemon written by wikia on each host
https://svn.wikia-code.com/utils/varnishhtcpd/htcpd-stomp-injectused https://svn.wikia-code.com/utils/varnishhtcpd/varnishhtcpd instead.- Needs to be installed via puppet with a startup script / config piece written
- Is network configuration needed to get multicast purge messages to varnish hosts? Most likely if using servers in Ashburn. Needs Mark
Varnish udp2log patch
[edit]Logging - we still need Varnish log2udp support via a logging daemon- What would be a good TTL setting for multicast? (Right now I defaulted to 1 second which is the same as it is in squid.)
- Implemented and is in testing by Patrick. Tfinc 19:17, 7 July 2011 (UTC)
- TTL is based on hops, not time - might need to be > 1 for running in the remote colo.
- Date/time format needs to be changed. It's currently in the apache standard, split over two fields. Needs to be like our squids.
- Example: 2011-08-08T23:51:35.203
- Hopefully a one line change, but requires making a new patch, building a new varnish package, etc.
Install Mobile frontend in production
[edit]- Install of mobile extension to production mediawiki cluster - this needs to be done as part of the mediawiki app install process
- Need help of someone familiar with deploying new mediawiki extensions to prod
- Patrick can own this since he has deployment experience Tfinc 19:17, 7 July 2011 (UTC)
- Need help of someone familiar with deploying new mediawiki extensions to prod
Enable Mobile frontend in production
[edit]- It's installed and testable via a query param but now needs to be enabled based on the presence of the X-Device header.
- Need a page to set the opt-in cookie. Does this need to go on the RoR mobile site? And an opt-out page via the extension?