SecurePoll 2014 Redesign

Group:	Wikimedia Platform Engineering
Start:	2014-02
End:	2015-04
Team members:	James Alexander, Brandon Harris, Dan Garry, Brad Jorsch
Management:	Rob Lanphier

This page documents a 2014 project relating to software design changes to improve the SecurePoll extension. General documentation about the extension can be found at Extension:SecurePoll.

Rationale

Extension:SecurePoll is the voting software used by the Wikimedia community for important elections, such as the Wikimedia Foundation Board elections and the English Wikipedia Arbitration Committee elections. Unfortunately, the extension is incomplete, and technical difficulties with it have necessitated restarting elections and discarding votes.

This project aims to make incremental improvements to SecurePoll so that it can be used more easily and with fewer difficulties.

The SecurePoll extension is an extremely difficult to use system that has been known to (unintentionally) encourage errors and considerable stress, tedium, and difficulty to poll designers, administrators, and publishers (though not with voters).

Most of the more difficult or tedious tasks using SecurePoll are handled manually: command line execution of scripts, hand-building of machine-readable XML files, manual database manipulation, and off-line trawling of spreadsheets are common artifacts of the process. At no point does the software do anything to prohibit errors from creeping in nor does the software encourage or aid the poll designers and administrators.

In the wake of a series of errors in polls caused by the above and because the Wikimedia Foundation Board of Trustees has expressed a desire to improve these processes, several parts of the process are being redesigned (or, "actually designed") in order to improve the overall experience of those involved.

Hypothesis

A more robust and usable SecurePoll software system will make movement elections easier to manage, maintain, tally, and publish, resulting in an overall increase in satisfaction by all participants of a movement election.

Work Phases and Project Scope

During an initial meeting about the project the team was treated to a walkthrough of how SecurePoll works and how elections are created and managed. Discussion was also given to the overall problems surrounding SecurePoll and a brainstorming session about how to improve the software was held. During this time, three areas that were ripe for improvement were identified:

Poll Design System: The current system used for designing and creating polls relies heavily on manual creation and manipulation of XML files. This is prone to error as there are no error checks. Further, the process is innately user-hostile.
Voter List Preparation: The list of eligible voters in any election is pre-determined before a poll opens. This list is generated through a series of expensive database calls (sometimes taking over 48 hours to complete). Once done, the list is then manually processed and in some cases eligible voters are added (e.g., Wikimedia Foundation employees are added to the list of eligible voters in Trustee elections).
Post-election Scrutinization: After the voting period of a poll has ended, a group of poll administrators examine all votes cast and attempt to remove duplicates. The software provides some help here but not enough, and the work is done through a tedious process that mixes off-line spreadsheet work and on-line vote striking/unstriking.

Work was determined to be split into phases, with phase 1 (Poll Design) entering design and implementation first (this was determined to be the point where the most errors were generated, with some of the errors leading to the need to restart polls from scratch during the voting period.).

Additional work to do in a parallel track to the above:

Logging: Any admin-related actions taken on a poll (e.g. poll creation, changes to poll configuration, vote striking, translations) must be logged. This log should be viewable by everyone.

Workflow of SecurePoll

Each poll exists in one of a finite number of states:

Configured: An initial configuration has been entered, including vote eligibility criteria.
Creating list of eligible voters: SecurePoll is comparing the vote eligibility criteria against the set of all users to create the list of eligible voters. This can take quite some time to run.
Ready to commence: Once the list of eligible voters has been created, SecurePoll waits until the specified start date.
Poll deleted If, before the voting period has started, the poll was found to be set up incorrectly, election administrators may delete the poll from the interface and recreate it from scratch.
Voting period: Voting has commenced. Voting runs from the specified start date for the specified number of days.
Poll aborted: If there were found to be grave, irreparable errors in the poll, the poll can aborted during the voting period. The poll will need to be recreated from scratch.
Voting period ended aka Vote scrutinisation period: The voting period has ended. In this state, votes can be manually struck by the election administrators if they are determined to be illegitimate (e.g. duplicate votes).
Results: After the election administrators have finished the vote scrutinisation, the results of the poll are announced.

A graph illustrating the transition between these states is available in this image.

This workflow represents the steps that SecurePoll must go through to successfully work through a poll, and these states may or may not be explicitly visible to end users depending on the decisions of the team.

Design

Phase 1: Poll Design Wizard

The design of an poll will now be managed within a "wizard" (Special:SecurePollWizard). Upon initially going to Special:SecurePollWizard, the poll administrator will be presented with five elements:

A list of currently running polls (have met start date but not end date)
A list of pending polls (have not met start date)
A list of polls that were aborted during voting
A list of previous, expired polls
A big button saying "Create new poll"

Clicking the "Create new poll" button will take the user to the Poll Design Wizard (see "Poll Design Wizard", below).

Clicking on the links of any pending poll will take the user to the design wizard as well, but in this case it will be pre-populated with the current information (e.g., title, candidates, etc.). The admin will be able to edit the parameters of the poll and/or add new candidates.

Links to expired poll will display tallied results (if already tallied and published). If the poll has not been tallied, it will go to the (currently existing) scrutineer interface.

Poll Design Wizard

(Note that the mockup may not accurately reflect what will be in the final product as it was built by examining poll XML files, which may or may not reflect accurate data structures.)

For the most part, the wizard is self-explanatory (or described in the mockup itself). Some notes:

Poll Run Dates: Rather than provide two date pickers ("start date" and "end date") which would require a additional error logic (e.g., "is end date before start date") and to reduce confusion ("is this date crossing the UTC dateline?"), a single "start date" picker is used with a pull down to select the number of days to run the poll.
Admins: The names of the poll admins are stored in the XML with pipe ("|") separation. Commas are indicated in the field; pipes may be preferred based on allowable MediaWiki usernames. Ideally, this would be a smarter interface (search for username, add to list) to reduce error. However, this interface (the Design Wizard) will only be used by a handful of people (probably less than 10) so a bit of "living dangerously" can be allowed, given the low amount this project has been resourced.
Column Labels: The number and type of column labels are actually dependent on the Poll Type selected. These fields should swap out display based on the type of poll selected (the mockup assumes a three-point radio range poll). Column labels should be editable but start with a set of sane defaults.
Options: Ideally, we can allow for additional fields per option (e.g., "Real Name", "User Name", "Home Wiki", etc.). Given that the project has a low resource allocation, however, this is considered out-of-scope at this time. Clicking the "+ New Option" button will insert a new, blank option into the list. Blank options, when submitted, should be discarded.

Saving the poll will:

In the case of a new poll:
1. Write the poll to the database
2. Generate applicable cryptography keys
3. Display to the admin the correct private key that will unlock and tally the poll results. This is the key that is sent to the publishing official (a 3rd party) (Currently, this key is generated on the command line and then inserted into the XML).
4. Create a composite log entry that:
  1. Indicates that the poll was created and by whom (single line)
  2. Has additional lines for each element created (for later analysis against changes)
In the case of an existing poll:
1. Write changes to the database, possibly adding or removing candidates.
2. Create a composite log entry that:
  1. Indicates that the poll was changed, what was changed, and by whom (single line)
  2. Has additional lines for each change that was made

Phase 2: Voter Eligibility

The following criteria are routinely used in polls in the Wikimedia community and must be supported in a vote eligibility interface:

User must not be blocked on p or more wikis.
User must have at least n edits before a specified date.
User must have at least m edits between two specified dates.
User must not have any of the following global groups.
User is eligible to vote if they have any of the following global groups.

These criteria have the following intent, respectively:

User is in good standing in the Wikimedia community as a whole.
User has a history of activity at some point in the past.
User is active in the present timeframe.
User is not a bot.
User is a member of a specific set of users who are allowed to vote irrespective of other criteria (e.g. system administrators or staff).

The election administrator must have the option to choose which of the criteria they wish to use in his election.

The logic for the criteria, if they are all used, is 1 ∧ 2 ∧ 3 ∧ 4 ∨ 5; if this formula evaluates to true for a particular user then they are eligible to vote, if it evaluates to false then they are not. This essentially means that in order to be eligible to vote, you must either meet all of the first four criteria or meet the fifth.

The software must also support the ability for election administrators to manually include and exclude voters in the eligible voters list. This will supported through separate forms; the exclude form will always override the include form.

Timeline

December 2013 - Scoping of project. Determine requirements. Put together list of improvements that can be made. Triage list (prioritise, assess difficulty).
21st January 2014 - MediaWiki Core quarterly review. During this review, the engineer time allocation to the project should be decided.

Documents

Brainstorming

References