Git/Conversion/Fundraising

The Fundraising team deploys on a different schedule and to different hardware than the rest of the cluster. As such, they've got their own sets of concerns. This page will serve to outline the current practice and the way forward when moving fundraising infrastructure to Git.

Code we deploy

• Core, but our own modified (probably older and patched up for things we need) version (See the payments deploy branch of 1.17 in svn). (payments cluster - need a project for this somewhere)
  • In the future, we may have to lobotomize parts of core for the payments cluster to lock down things like unnecessary API functionality, in the name of PCI compliance
• CentralNotice (cluster - No special access)
• ContactPageFundraiser (cluster - No special access)
• ContributionTracking (payments cluster - owned)
• DonationInterface (payments cluster - owned)
• DonationInterfaceLangOnly (cluster - probably need to establish a deployment branch for the cluster)
• FundraiserLandingPage (cluster - No special access)
• LandingCheck (cluster - No special access)
• ParserFunctions (payments cluster - copied from 1.17. Probably don't need special access for this)
• cldr (cluster - No special access)
  • Is this cluster, payments cluster, or both?
    • Just the regular cluster.

How we currently deploy

Since moving to git, we don't.

Back in svn, we had trunk and a production branch. Once commits were code reviewed and ready, we would MFT (via cherry-picking or what-not) to the production branch and then svn up.

Since you guys are deploying on a different set of hardware from the cluster, I think it would be pretty easy to set up a branch for you guys to deploy from. Gerrit's permissions are very fine-grained, so we can lock down the fundraising branch so only you guys can review/submit there. Then you guys could cherry pick from master the changes you need.

In talking with Jeff, it turns out Gerrit's not so great for this. PCI compliance requires review on a system that "can't be tampered with." Gerrit does not qualify. We suggested perhaps replicating to a payments-owned server and having you guys maintain the branching there. Still a matter to discuss.

We also have language-only versions of some of our extensions that do get deployed on the cluster.

Things we like about the current system

• We can cherry pick revisions to deploy
  • Git has cherry picking. You can easily cherry pick a change from master to your branch :)
• We can omit directories (e.g. */tests/*) from deployments
  • This is likely to create merge conflicts in the long run--is there a particular reason for omitting this?
    • Yes. In fact, this is totally necessary, even if I have to go in and rm -R these directories by hand every time (I'd really rather not). Deploying payments tests opens up all sorts of serious security vulnerabilities, not to mention the possibility of accidentally creating hundreds of accidental real payments on a production machine, which would be Bad. As we never want to run this code on a production machine, omitting the code from the production branch is by far the easiest way to keep us all safe.
• It works
  • We'll make git work too ;-)
• The deployment branch is all in one place, so we only have to svn update one location before we blast it out to the payments cluster. Not sure how this is going to work with all the extensions being separate projects from both eachother, and core.
  • You add the extensions you need as submodules to the core repo on your deployment branch. Then updating goes from `svn up` to `git pull && git submodule update`

Things that we wish we could change about the current system

• Probably because of our out-of-control cherry-picking habits, svn merge on directories seems to regularly mess with file properties we don't need to mess with. For that reason, we tend to have to merge file-by-file, which takes a lot more time to roll up.
  • This is partially due to bad habits, but also in the absolutely braindead way that SVN handles properties. This should be a non-issue entirely in Git.
• I worry about git giving some engineers the impression that it would be a good idea to roll up what should be a zillion commits in one giant mega-commit, and regularly finding that half of it is usable and immediately deployable, and the other half should probably be immediately reverted. I heard a rumour, though, that git patches can be split so you can deal with partials individually. I'd definitely like to know more about that.

Hacks in the existing branch

Migration is slowed down by hacks that don't exist in trunk/master. Does the current branch have any such live hacks? If so, what are they?

No. I slap people for live-hacking, right before I revert and update. :D

Deployment script idea

For creating the deployment branches on the cluster, we use a tool called make-wmf-branch. We could fairly trivially write a tool that lives safe on the payments cluster and does essentially the same thing:

1. Give it a name for your deployment branch (optionally a specific commit hash if you don't want HEAD)
2. Give it a list of extensions you want to include (optionally a specific commit hash if you don't want HEAD)
3. Tool then creates your new checkout of core (at the point you want it) adds the extensions as submodules (at the point you want them)
4. You can then push this branch out to the payments cluster.

Thoughts?