Jump to content

Selenium/Ruby/Workshops/Search features

From mediawiki.org
10min video with basic instructions: youtu.be/Kad3EUM4GdM
Etherpad to document progress: http://etherpad.wmflabs.org/pad/p/Browser-automation

WHAT: Writing Wikipedia search scenarios in plain English for our automated testing process. Let's feed the Test backlog with descriptions like this: 

Feature: Search

  Scenario: Search suggestions
    Given I am at random page
    When I search for: main
    Then a list of suggested pages should appear
      And Main Page should be the first result

WHEN: We will start with 30min demo streamed on Wednesday, March 13, 2013 at 17h UTC and we will be helping volunteers during the rest of the week. This is an ongoing activity: you can arrive / leave at any time.

WHERE: #wikimedia-dev connect IRC channel. After joining on IRC, say Hello browser testers! and we will welcome you with instructions and a simple task. chrismcmahon, zeljkof and qgil will do their best covering timezones. You can also use the Discussion page to ask any questions.

WHO: Anybody interested, including you! The only requirement is basic level of plain English in order to describe the features to be tested automatically. It is that simple.

WHY: The Wikimedia Foundation engineering team is working on several Search improvements. Automated browser tests do find bugs!

See also How to contribute.

This is a weekly QA activity organized by the MediaWiki Group Browser testing. Join us!

Focus

[edit]

Here is a list of open bugs we are likely to fix soon that we want to test automatically:

We could use regression tests for Search features also:

  • Umlauts and accents are disregarded in search
  • Some ligatures match the separate letters. For example, a search for aeroskobing will find pages containing Ærøskøbing (ae = Æ)
  • It is not possible to search for the string |LT| (letters "LT" between two vertical bars); all articles with "lt" will be returned.
  • Ignore parentheses except for exact matches: Credit (finance)" will return articles with the words "credit" and "finance", ignoring the parentheses, unless an article with exact title "Credit (finance)" exists
  • A phrase in double quotes returns exact matches
  • Etc.


There are more examples in Bugzilla.

Both desktop and mobile browsers need Search tests.

Evaluation

[edit]

★★★☆☆

  • Summary: It was our first Browser testing activity. The preparation work was significant but the external participation was small: 3 people. Still, we broke the ice and organizing a second edition will be a lot easier. Promoting the event to our interns and group members was useful.
  • Results: 9 new scenarios related with Search in the Test backlog (the goal was 5).
  • Participants: 7 in total. 4 WMF employees (Chris, Zeljko, Quim, Sumana), 2 OPW interns (Valerie, Sucheta), 1 volunteer (Rachel99). 3 external when the goal was 5. All women!
    • New: 4 people wrote their first scenario ever: Valerie, Sucheta, Sumana, Quim.
    • Repeating: Rachel99 had volunteered previously.
    • Mood: people seemed to have fun.
  • Documentation: we created a highly reusable weekly goal page (this one), a 10min video (youtu.be/Kad3EUM4GdM, move to Commons pending), one new page (Examples) and significant improvements improvements to Browser testing, Group Browser testing and Test backlog.
  • Promotion: it wasn't easy because the concept is quite novel for most people. We did broadcast through social media, wikitech-l, wikitech-ambassadors, en.wiki tech Village Pump, Chris & Zelko's blog posts. We sent calendar invitations to 9 WMF employees and 6 OPW interns. We contacted the 4 members of the Browser testing group (ecp declined because of travel).
  • Developer team: Munagala "xyvram" Ramanath is the only Search developer currently. He was available during the activity, pointing to details about bug reports.
  • Organizers: It took a significant amount of time and work to get Chris, Zeljko and Quim in the same page and to prepare the documentation. But we did it and next time it should be a lot simpler.
  • Lessons learned:
    • There must be one topic well defined in advance.
    • Pointers to bug reports or scenarios are a must.
    • Organizing a synchronous activity is good and fun, but we need a clear workflow for DIY contributors landing at other times.
    • Encouraging pair programming was interesting for the couples formed, but maybe that left some individuals out? Needs more thought.
Retrieved from "https://www.mediawiki.org/w/index.php?title=Selenium/Ruby/Workshops/Search_features&oldid=3688201"