Progress reports for Evaluating and Improving MediaWiki web API client libraries

Final report

I wrote a new standard for third-party MediaWiki web API client libraries, reviewed and evaluated the existing Perl, Python, Ruby, and Java client libraries, and filed bug reports and feature requests with the results. I also improved the documentation for the Java Wiki Bot Framework and am working on adding the ability to use the MediaWiki API's search module via JWBF. Additionally, I spoke at WikiConference USA 2014 on using the MediaWiki API and its client libraries, and gave an "APIs 101" talk at AdaCamp Portland that used MediaWiki's API as an example.

Output summary

Wrote a standard for third-party MediaWiki web API client libraries after soliciting community input: API:Client code/Gold standard
Evaluated libraries according to the standard: API:Client code/Evaluations
Sorted API:Client code by level of maintenance
Made substantial improvements to the JWBF README (https://github.com/eldur/jwbf)
Spoke about the MediaWiki API at WikiConference USA 2014 and AdaCamp Portland

Ongoing work

Finish writing search function for JWBF [1]
Revisit evaluations in Oct/Nov 2014

Week 12/August 4-8

Changes to README.md are merged
Started drafting supplemental documentation (developerDoc.md)
Figured out code organization, inheritance, utility classes, what extends what

Week 11/July 28-August 1

Selected JWBF to contribute to for the final phase of my internship
Asked for more information on what information is actually necessary in the user-agent: http://lists.wikimedia.org/pipermail/mediawiki-api/2014-July/003304.html (need to follow up)
Got 2nd-4th opinions on how JWBF looks and its documentation quality
Got a project with JWBF as a dependency started in Maven
Substantially reorganized and added to the README for JWBF, including a "Getting Started" section: https://github.com/fhocutt/jwbf/blob/master/README.md, https://github.com/eldur/jwbf/pull/26
Figured out the basic structure of JWBF and proposed additional documentation: https://github.com/eldur/jwbf/issues/22#issuecomment-50694332

Week 10/July 21-25

Finished evaluation/TODOs for MediaWiki::Gateway and JWBF
Submitted issues for JWBF to Github issue tracker
Added comments and links to Pywikibot bugs in API:Client code/Evaluations/Pywikibot
Added Wikidata Toolkit to the list of Java libraries available: https://www.mediawiki.org/w/index.php?title=API:Client_code&curid=16467&diff=1073647&oldid=1062265
Decided on mwclient and JWBF as the shortlist for libraries to contribute to in the third phase of the internship

Week 9/July 14-18

Finished evaluation/TODOs for MediaWiki::Bot
Submitted issues for MediaWiki::Bot to Github issue tracker
Emailed MW::Bot and MW::API maintainers
Started evaluating MediaWiki::Gateway
Paired with Nik to start evaluating JWBF's code (verdict: pretty solidly written Java)

Week 8/July 7-11

Finished evaluation/TODOs for MediaWiki::API
Started evaluating MediaWiki::Bot
Started learning Ruby/talking with Rubyists to evaluate MediaWiki::Gateway
Added TODOs for simplemediawiki, wikitools, and mwclient to the projects' issue trackers

Week 7/June 30-July 4

Finished writing TODOs for Python libraries: API:Client code/Evaluations
Emailed TODOs to all Python library maintainers, and gave clarification when requested (API talk:Client code/Evaluations/Pywikibot‎)
Added links with my work to Data & Developer Hub: https://www.mediawiki.org/w/index.php?title=Data_%26_Developer_Hub&curid=170842&diff=1056865&oldid=1055396
Started looking at Perl libraries: API:Client_code/Evaluations/MediaWiki-API
Posted progress to mediawiki-api-l: http://lists.wikimedia.org/pipermail/mediawiki-api/2014-July/003292.html

Week 6/June 23-27

Traveled to Portland to participate in and present at AdaCamp (Introduction to APIs, using the MediaWiki API as an example) and give the Thursday keynote address for Open Source Bridge.
Met with Sumana in person to discuss status and plans, arranged for daily check-ins and agreed to write more/smaller deliverables for the rest of the internship period

Week 5/June 16-20

Finished up mwclient evaluation
Started pywikibot evaluation: https://www.mediawiki.org/w/index.php?title=API:Client_code/Evaluations&oldid=1039850
Finished up responsiveness evaluations for all of the Python libraries except for pywikibot (last missing piece for all the evaluations).
Emailed Merlijn with pywikibot/pywikibot community questions (need adding to the evaluation)
https://github.com/ianweller/python-simplemediawiki/issues/12
https://github.com/mwclient/mwclient/pull/42
https://github.com/mwclient/mwclient/pull/43
https://github.com/mwclient/mwclient/pull/44

Week 4/June 9-13

Continued evaluating simplemediawiki and wikitools, completed except for "Easy to Improve" sections.
Started evaluating mwclient (https://www.mediawiki.org/w/index.php?title=API:Client_code/Evaluations&oldid=1033313) and asked for clarification on library workflow structure: https://www.mediawiki.org/w/index.php?title=Talk:Evaluating_and_Improving_MediaWiki_web_API_client_libraries&curid=202499&diff=1037131&oldid=1035529

Week 3/June 2-6

Lots of reading on software development, programming psychology, API usability, engineering community norms
API:Client code/Gold standard is out of draft; minor changes may be made as needed, but the bulk of it is in final form
Started API:Client code/Evaluations as a draft; began evaluating simplemediawiki and wikitools
https://github.com/alexz-enwp/wikitools/pull/11
https://github.com/ianweller/python-simplemediawiki/issues/11; https://bugs.launchpad.net/ubuntu/+source/python-simplemediawiki/+bug/1328303
https://github.com/ianweller/python-simplemediawiki/pull/10

Week 2/May 26-30

Revised API talk:Client code#Proposed "gold standard" criteria and solicited feedback; incorporated feedback into second (final?) draft.
Traveled to NYC to present at WikiConference USA 2014: http://franceshocutt.wordpress.com/2014/06/02/wikiconference-usa-2014-rundown
Shared slides on Commons
Met other developers

Week 1/May 19-23

Select the best library/libraries in Python, Perl, JavaScript, Ruby, and Java. (required)
- Done: API talk:Client code
Research and decide on criteria to evaluate these libraries in more depth.
- Done: API talk:Client code#Proposed "gold standard" criteria
Also went through API:Client code and separated all libraries that have not been maintained in the last year
https://github.com/mwclient/mwclient/issues/39

Community bonding report

Minimum viable product and goals

Search for the existing API client libraries, add these to API:Client code.
Select the best library/libraries in Python, Perl, JavaScript, Ruby, and Java using preliminary criteria.
Research, select, and write up criteria to evaluate these libraries in more depth. Write a library "gold standard."
Post "gold standard" and useful resources found while writing it to Data & Developer Hub.
Evaluate the best libraries in three languages. To the library maintainers submit documentation of tests and results, praise for where they get things right, and bug reports for areas that can use improvement.

Communication plan

Check in daily with Sumana over IRC and/or vidchat
Communicate with other mentors over IRC and email with questions, to look over code, etc.

Lessons learned since 21 April

Wireless drivers on Linux are a challenge, even for ThinkPads!
How to replace (if not how to compile) a kernel in Ubuntu
How to set up a virtual environment in Python and install packages to it
How various MediaWiki IRC channels are used
How to write a twitter-bot, the structure of Wikidata, the Wikidata API, a lot about writing Python programs:
- https://twitter.com/AutoWikiFacts
- source at https://github.com/fhocutt/obscure-enwiki-fact
How git and github work, including add/commit/rebase
- submitted patches for Dreamwidth and had them merged: https://github.com/dreamwidth/dw-free/pulls/fhocutt?direction=desc&page=1&sort=created&state=closed