Jump to content

Wikimedia Discovery/Meetings/Portal retrospective 2016-04-22

From mediawiki.org

Discovery Portal Retrospective

2016-04-22

Covering whatever has happened related to the team since the last retro (2016-03-23)

  • Quarter ended; new quarter started
    • Process for deciding on new quarterly goals
  • Full month without Julien
  • Deb and Jan traveling
  • Fixed a bunch of bugs
  • Ran another test
  • Brainstorm/roadmap meeting
  • Moiz begins his transition out
  • Weeklong code freeze due to codfw migration
  • Started using new portal backlog workboard

Review action items from before

[edit]
  • Jan: Document "production quality", beta site, demo sites, etc.
  • Kevin: Try to unstick quick surveys
    • Yes Done, although I'm not sure if they are re-stuck or just on hold. Either way, they are no longer the responsibility of the portal team.
    • Agreed - the work done by the Portal team is done for the time being. All survey tickets have been moved to the Portal Backlog board

What went well?

[edit]
  • Ran the localized top-links A/B test
    • Some delays turning it on and turning it off
    • Showed actual user impact
    • Went smoother than earlier tests
  • Working with Julien on code review is going well
  • Jan has done very well with the tasks assigned and delving into upcoming tasks and research +1+1
  • Great analysis by Mikhail - he's been busy with lots of other tests and work +1+1
  • Good conversations (that Deb had) with Legal, Comms, and User Testing teams about upcoming work +1
    • Also, Edward was very helpful with qualtrics
  • Portal work (small bug fixes) in Tech News, weekly status updates, mailing lists. (good communication) +1+1
    • Reminder: Work that seems small to you might still be worth publicizing
  • Good collaboration with User Research team
  • Portal Brainstorming meeting was good - lots of new ideas that can be acted on in the future+1+1
  • Communication between Portal engineers and analysis team+1
  • Team has gelled and supported each other well through some organizational turbulence

What could we improve?

[edit]
  • A/B Test roll out process
    • We have improved, but could still be improved more
    • Checklist for all the things needed to complete a test
    • Look at how to avoid past problems in the future (retro for each test?)
      • Would be cool to know how long each step took, so we would know where to optimize
    • Maybe sit down with FR team, because they run tests very quickly (e.g. 30 minutes)
      • They are less busy right now
    • Maybe talk with search team about their decision to build relevance forge, as a way of putting things in front of users to get feedback
  • The time it takes us to run an A/B Test
    • If all our work is going to be tested, we need to get really good at doing it
    • Some delays were due to developers being new; some was rework of hasty prototype quality code
    • Last test didn't run into those problems, but still seemed to take a while. Not sure why.
    • Portal doesn't have a rich framework like mediawiki does, so have to invent more
    • Reviews (code, security) does take a while. Seems to be getting shorter.
    • Need to be careful because any change affects so many people
    • Perhaps we could get people to review mocks in some cases rather than running a full test
  • Survey should've been out by now, disconnect between user research/design and product
    • UR/D came up with a survey they felt good about; when Deb saw it, it didn't align with her expectations
    • We should include product manager in meetings of what we're trying to do with a survey (earlier in the process)
    • The email thread has helped; we will probably do multiple surveys
    • Now facing some unexpected technical issues to implement what UR wants to do
  • Future portal meeting was underwhelming (i can provide more context)
    • Was hoping for more excitement about big ideas and ongoing work
    • We need to apply resources where the benefit/cost is highest, so regularly re-evaluating each area
    • Portal team lost months just getting the code into git
    • Team is hitting its stride this quarter, but we still have a lot to learn about what we should be doing
  • Design stuff has been slow last two weeks, sorryyy.. New mocks coming today :)
  • Moiz shouldn't leave!

What else should be noted?

[edit]
  • Two surveys will probably be run on the Portal in the coming month(s)
    • requires close coordination with User Testing team, Legal and Portal teams
    • two surveys might be done - one to hit up all users after a period of inactivity on the page, the other one to talk directly with users that bounced previously from the portal and engage with a survey.

Action Items

[edit]
  • Jan: Create phab task to wrap up doc of beta/demo/production sites
  • Jan: Do a post-mortem on what technical tasks actually took the longest time to complete.
  • Deb: create rough mocks of some of the ideas for the future work and add to page on wiki ro further discussion
  • Deb: set up meeting with Creative Tech (Megan) to go over their rapid testing process (Deb, Jan, Mikhail)
  • Deb: Create ticket for pushing successful lang links / localized phrase to production on portal
  • Kevin: Consider some light form of retro after next A/B test
Retrieved from "https://www.mediawiki.org/w/index.php?title=Wikimedia_Discovery/Meetings/Portal_retrospective_2016-04-22&oldid=3944623"