Wikimedia Discovery/Meetings/Search retrospective 2017-03-28

The Retrospective Prime Directive: “Regardless of what we discover, we understand and truly believe that everyone did the best job they could, given what they knew at the time, their skills and abilities, the resources available, and the situation at hand.” — Norm Kerth

Action items from previous retro

Erika: Follow up on feedback mechanisms DONE (conversations; planned session at offsite)
Erika: Follow up on knowledge sharing
- Have had some conversations
- Hope to address more at the offsite
Kevin: Follow up on not having meetings too efficient DONE -> Dan has also been good at making sure we address off topic stuff
Kevin: Send list of little things around to everyone in email DONE

What has happened (since March 1)

(Note: These were pulled from status updates, so there may be duplication and/or out-of-date entries)

Finished analysis of potential new Chinese-language analyzers. See T158203. We expect improved word segmentation accuracy, which should lead to better search results, as well as support for searching across Traditional/Simplified character sets. A test index in labs will be available soon.
Posted on French, Arabic, German and Russian Wikipedia Village Pumps to ask if there are any issues with A/B testing the sister project search results on them
ElasticSearch 5.1.2 is now serving all wikis from the eqiad (again!) datacenter,
Took a wrong turn on some updates to the Swedish analysis chain (T155822), but got set right with some helpful input from native speakers (T160562) and completed the analysis of enabling ICU folding on Swedish, which we are in the process of deploying. Actual updates to the Swedish wikis will follow.
updated several minor UI bugs for second A/B test for sister project search results: (task T158937), (task T158935)
Started and finished a second A/B test for sister project search results on selected wikis, (turned on second A/B test for sister project search results (task T160005) for a week of testing on 8 wikis)
Updates from work done in previous weeks (apologies for the delay!):
- Fixed an issue where subpages were mistakenly preferred over parent pages if you didn't type the whole title (T156840)
- Fixed multiple issues with the how Elasticsearch integrates with the Translate extension (T132076, T132315, T158070)
Fixed regressions which seem to be related to Elasticsearch 5 upgrade:
- Fixed an issue causing language-converted suggestions to not show in the search box (T160896)
Fixed an issue where using certain special characters in advanced search syntax would unexpectedly give zero results (T147636)
Special:Undelete search is almost done (we may be done with it by this year's Hackathon!)
Experimenting with a new agenda for weekly sprint planning meetings

Format: "Tiny Retrospective"

This retro focuses on the really small things that might make a difference
- http://retrospectivewiki.org/index.php?title=Tiny_Retrospective

People

It is great to see Erik march through the upgrades minefields and go through mostly unharmed+1
- +1 it went surprisingly smooth (at least for an external observer :) +1+1
It's great to see how often everyone pitches in to conversations about directions the team should take (ie. on the machine learning to rank stuff)
*There are so many things that this team wants to do but just not enough time to do it...but everyone is excited about doing what they can, it's refreshing!

Technology

Internet access is elusive. Having it would make a difference.
[1] * Some discussion / experience sharing around Maven might make sense (I have been having quite a few discussions around it already)
[3] ***Fix how we do tests in our elastic plugins
- Since switching to ES5, there are two ways to send queries, but all tests go through the binary protocol, not the json protocol
Should we have more discussions on some of the more etheral things we'd like to do with search (machine learning, etc)?
finish migration of elasticsearch plugins to gradle (seems it has been in the work for some time and David probably need some help on it - or more time in the week...)
ELK stack has never ending problems with multiple log sources into same index...but hopefully not really our problem
- We receive problem reports, but nothing clean we can do about it
[7] *******Product team (Discovery) owning non-product infrastructure (Logstash) feels weird and out of scope, although it makes perfect sense from a technological perspective
[1] * It's sad that live index updates for completion suggester got stalled and didn't make it out with ES5
- THEME: Reliable updates to ES5 is hard
- Should we have any postmortem writeup on this?
- FIguring out what to update is hard
- CirrusSearch document model is really different from mediawiki doc model

Process

Facilitator should take an extra 5 minutes during retro prep to dedupe the "what has happened" list :)
New/more formalised agenda for the sprint planning meetings as a result of a previous retrospective is working well from my (Dan's) perspective +1
- In particular, new agenda blends the strengths of both Dan's and Deb's approaches to sprint planning meetings, which has been great.+1
How to bring in a new contractor process has been interesting
- Costs evolved over time; negotiating start dates
[2] **How do we feel about quarterly planning process? Current process is ad-hoc and light, and works.
[7] *******Should we more formally split frontend/UX and backend/relevance into separate planning meetings? How do people feel about the current arrangement of things?
Tiny retro is hard, but it's a good idea to consider small things with potential big impact. Maybe add it on as a single section in any other retro format.

Other

Referring to people not on our team by first name only is ambiguous/confusing (e.g. Bryan/Brion/Brian)

Discussion

[7] *******Product team (Discovery) owning non-product infrastructure (Logstash) feels weird and out of scope, although it makes perfect sense from a technological perspective

Erik: For history, Bryan (bd808) set this up to avoid grepping logs. Set up ELK cluster. He has moved to other teams and positions, so doesn't fit his JD. Have heard that there might be a request to hire a log wrangler.
DC: It really requires someone full-time. It's not working well with decisions by Elastic. It's a huge mess.
KH: Any idea what team might be requesting log wrangler? [Ops]
GL: We're not bad at maintaining ELK cluster...upgrades, looking at it when things go wrong. But we're not in a position to move it forward
DC: It's fragile.
Erik: I have spent at least 50% of my last 2 weeks to get it working with ES5 upgrade.
GL: Anything we can do about it, if nobody else does? Better us than having it just die.
Erik: There is nobody to hand it off to.
GL: Definitely would make sense in ops.
KH: There is a pervasive game of "hot potato", so if nobody else takes it, we keep it. [7] *******Should we more formally split frontend/UX and backend/relevance into separate planning meetings? How do people feel about the current arrangement of things?
Erik: I feel that should be up to Jan
JD: I rarely attend sprint planning meetings, because I have a conflict with a weekly design meeting. Not sure if it should be more formally split, but could use more communication from me about what I'm working on. I have an earlier Tuesday mtg w/Deb, where we discuss search stuff. Maybe that meeting could be formalized, and she could give updates during sprint planning meeting.
DT: That's what I do now. I act as a conduit. Don't know that it needs to be formalized. We're kind of making sure it works. I don't see it as being a problem, unless Jan and I are not giving the team enough info about what is happening on the front end.
DG: I added this, and don't feel like there is a problem. Could we optimize? Sounds like we could make some improvements.
JD: It's hard for me to think that search relevance or other backend work affects my work. Not sure we can improve that.
DG: It's good that the backend changes don't affect the front end much--shows that the APIs are working.
DT: Affects front end but not the code. Both sides knowing what the other is doing is important. [3] ***Fix how we do tests in our elastic plugins
DC: We need to write more tests. The nice features are not easy to use with maven. I started to port to gradle, but not sure it will solve everything. Not sure we should do it because gradle is new tech for the team. Elastic's direction is gradle.
DG: Android moved from maven to gradle. Maybe they could help.
DC: One concern is CI, and how we would adapt jenkins if we switched. Mobile team might
DG: They made a lot of CI improvements. They have added for example building the apps for a patch. I'll send an email.
DC: I started it in 10% time. We should make the decision as a team.
Erik: Not clear what gradle would buy us.
DC: You can reuse REST testing framework. Writing queries as you would inside elastic. Visible what you send, instead of writing in java. The .pom file has a lot of hacks, so build.gradle seems to have a lot less hacks. The real problem is fixing tests. If we can do that in maven, I might be ok delaying. But the upgrad to ES6 might be hard in maven.
Erik: Makes sense. ES6 RC should be within 6-12 months. Nothing announced yet.
GL: Make sense to follow ES. Will make our live easier in the long run. We should not increase dependency on ES in tests. Might not make sense to use their test features if it causes too many dependencies.

Big ideas parking lot (captured here for future consideration)

Consider dependencies between quarterly goals (e.g., language analyzers needed to wait for ES 5 update)

Retro of the retro

The things we talked about weren't tiny (but discussion was good)
Would we have had these discussions without this format?
- Probably, at least if we had the topic area prompts

Action items

DG: Ask android team about gradle
DG: Consider dependencies between quarterly goals (e.g., language analyzers needed to wait for ES 5 update)
GL: Set up gradle (maven) meeting within search
KS: Start email thread about frontend/backend meetings
KS: Include a "tiny things" section in the next retro, but switch the format
KS: Send email about items that got votes but weren't discussed:
- [2] **How do we feel about quarterly planning process? Current process is ad-hoc and light, and works.
KS: Facilitator should take an extra 5 minutes during retro prep to dedupe the "what has happened" list :)