Early planning for Portal "Trending Articles" feature

2016-03-10

Deb, Julien, Jan, Moiz, Tomasz, Kevin, Mikhail, Dan

On mobile, if you switch languages, the old language will stick around for a day

Currently, articles are updated daily

Moiz: Before we do trending, we need to do page cleanup

Collapse/hide/remove bottom links (trending articles would fill that space)

Collapse/hide/remove globe (to avoid articles being below the fold)

T: Need to include Chris in this discussion

Moiz: We could either do that cleanup incrementally, or we could do a "big bang" one step.

Deb (D): We had originally thought about A/B testing the globe and language removal. Arrangement convention was set in 2008. There seem to be a number of people who like the current arrangement. I'm nervous about making those changes without testing them first. But maybe we should do the big thing as an A/B test.

Moiz: I think we should do the big thing, and run it as a test. If that test is successful (need to define "success"), then we should deploy in a staggered 3-step approach.

D: Would testing everything be good analysis practice? Need to hear from Mikhail

Jan: Changing the page visually might cause a fairly large community reaction. Doing it at once, maybe we would collapse any outrage into a single event, rather than having multiple conflicts. Running as an A/B test of the whole thing would be challenging technically because it's not merely hiding/showing elements. Not sure it's feasible.

Julien: 1. It's a lot of changes, and the tests should test more isolated parts. We would be testing multiple things. 2. Doing everything in one page is a big technical concern.

K: Technically, could we do a redirect? (Devs can answer that later.) For the test, testing it all at once kind of makes sense because testing increments without the benefits might be misleading.

Moiz: We'll have to solve the tech issue eventually anyway. Collapsing the bottom section will only affect 1%, and collapsing the globe will affect 10%.

K: We're not testing the gamified scoreboard use of this page.

Moiz: We should link to a gamified scoreboard from the portal. A small number of vocal people use it this way.

T: We can't lose that. It's the way they broadcast it. The portal isn't a great spot for it. A separate page would probably be better.

D: Need to account for mobile use. And it would be great to keep them on the portal page when they check out the stats.

Julien: Don't agree with the 1%. We should make the bottom links better. Adding a link would be like an About button that never gets clicked. People are very interested in these numbers, and it's the only place online where you have a list of all the wikis. Small wikis value this. There is competition for positions in the top-10, and we have already had complaints when that's not updated often enough. It's a great marketing page for the wp brand. The front page of a web site is typically a marketing tool, and not all that functional. We don't have metrics for the marketing aspect.

Moiz: Should we not remove the links?

Julien: It's a difficult issue for the community. That's why I think we should be in an exploratory phase, using prototypes. To inspire people, and engage them. Get them to say "This part is awesome, but I really still want that missing thing too".

Moiz: So you're arguing for portal labs?

Julien: Basically, yes.

Moiz: I thought we had all agreed to focus on trending next quarter, not portal labs, so we need to have that discussion.

T: We need to figure out how to work with community to visualize their pride in their work. They're probably not locked into this specific design.

D: You have raised good points, and I have changed my mind. We have been doing small steps. People are used to this page. What if we look at the bigger picture, where we'll do a complete redesign. We want to keep the aspects of gamification, etc. But display it differently.

Moiz: We need to make sure we involve the community in this redesign. Maybe we should post step-by-step mocks, with benefits and rationale, and make it part of an RfC.

D: That has been done

Moiz: But not as an RfC. On signpost. Explain what we plan to do, and ask for opinions. That could be a 1-2 month process. We don't have to build a portal labs infrastructure. Just do it in a wiki way.

Julien: The wiki way doesn't allow much space for innovation.

Dan: Depends on how the consultation is arranged.

Julien: As a RfC, it wouldn't

Dan: Really depends on the framing. We can triage input to focus on the useful comments.

D: For the mobile redesign, did they ask for community input?

Dan: That was a small number of users (.8%).

Julien: As long as you don't touch what they have been using for 10 years, people will be OK.

D: If we still have all the features people come to the portal for, why can't we do something fun visually. Usage is relatively low. Might be interesting. As long as it does the things it does now, but better.

Julien: Curious about the history before we joined (say pre-2015)

Moiz: We need to make an effort to involve relevant portions of the community before anything is deployed. To do that, we could do A/B tests, and use data to prove that what we did was good. Or we could do something more like an RfC/consultation. In either case, we need to talk to people. Not everyone, but give interested people an opportunity to speak up.

D: We have a beta page already

Moiz: That's for mediawiki.

Dan: There is a portal beta page

Moiz: Oh, I thought she meant beta features, but that's mw, so not available to the portal. So we could link from the RfC to working prototypes. But I want the community to be involved early and often. I don't want to spend a lot of resources before we get some feedback.

D: Can be WIP. We may or may not get any feedback at all. By having the consultation on some kind of labs/beta, the existing page continues to work.

Moiz: We need feedback.

Dan: Let's not go too far with CE issues without Chris being here.

Julien: I would like to target a broad/large audience, not just the small group of vocal people that normally participate.

Dan: "The community" does not exist. Individuals have different opinions. Would prefer that we talk about "users" rather than "community". We should ask the people who go to the page, which is not us, and not (just) the vocal contributors.

D: We could add a link to the portal itself to let people go to the new experimental stuff

Dan: There need to be feedback channels, which is a question for Chris. This is a solved issue.

Julien: RfC's are really targeted at serious wp people, not the broader user base

Dan: Don't get too caught up on the term RfC. There are different types of RfC. The important thing is to get user feedback.

Julien: A handful of vocal people shouldn't be able to veto something that would be useful to millions.

Dan: If one person says "no", we need to know why. How is one person from community saying no any different from a staff member saying no? It's not. If we have data proving something, that should override any one person who simply objects. If the person explains why the data is wrong or misleading, then that is helpful and worth considering.

D: This team was formed to make the portal better. We shouldn't be stopped based on 10 people saying "I don't like it". We can show them with data that there are major improvements.

Julien: I want the conversation to focus around the page itself, not on separate mock-ups.

Dan: We want to encourage helpful feedback, and discourage unhelpful feedback. The product owner (Deb) clearly owns the look of the portal. So I'm not too concerned. We want broad input.

Mikhail: Why wasn't Chris invited?

Dan/Deb: Simple oversight. As long as we don't make decisions without him, it's fine.

Mikhail: When should we involve user research?

Dan: Later, when we have something more concrete. They can do guerrilla testing. In January, that was larger scale, but they can do lighter tests.

Jan: There are a lot of possible channels for feedback. The question is at what points we want the feedback: Mock-up? Functioning prototype? Ideally, at all steps.

D: That's why I suggested using the beta. Tell people: Here's what we're working on.

Jan: The beta is late in the process (e.g. varnish cache debugging). The main page there should be reserved for that. We should have somewhere else for earlier feedback.

Dan: Don't worry about that. We can generate another copy easily, and use that for other phases. It's a valid point, but not a worry.

Julien: The beta cluster is not designed for getting feedback on early designs.

Jan: I just want to make sure we have a place to deploy just before production.

Julien: Jan and I should be involved with the UX design.

Jan: From analysis perspective, we need to set up new systems to judge success

D: Let's do it. Let's create a design that has all 3 things on a page (search, language clicks/ranking, trending). Let's create a new publicly accessible preview portal page and get feedback on it. It doesn't matter where it is hosted, but it should be functional. Regularly update it as we add features. Invite community to view it, and provide feedback somewhere (else).

Dan: There are several issues that need to be resolved, but not in this meeting. Deb should write up a plan based on this meeting, and then we can discuss in email.