Talk:Reading/Strategy/Strategy Process/Testing

Please feel free to list comments, questions and feedback? Would you like to add further tests? Would you like to adjust testing conditions? Share your thoughts!--Melamrawy (WMF) (talk) 22:48, 20 October 2015 (UTC)

Makes more sense to me

Hey M, this looks a lot clearer. Thanks for your work on this! The "About" section works. One thing: In the first bullet point, it says "beautiful" twice. -Rdicerb (WMF) (talk) 21:31, 21 October 2015 (UTC)

Re: Allow readers to interact with content

"Allow" is misleading, all users can alter our wikis. "Activate more users" is what you seem to say. Nemo 07:58, 5 November 2015 (UTC)

Hey Nemo, could be re-worded. All users can change our wikis, but given that this is Reading strategy, so talking about readers, it is not expected that all readers will become editors. The idea is to allow ways to interact with our content also a reader. Maybe allow readers isn't the best way to describe it, but activate more users doesn't make it clear, imho. Other suggestions? --Melamrawy (WMF) (talk) 14:11, 5 November 2015 (UTC)

Re: Traffic from design

You should also monitor the opposite, i.e. what redesigns killed the traffic. I still have to see a good analysis of the effects of the Flickr redesigns, for instance, which upset so many pro users and wikimedians. Would be interesting indeed to find out. Nemo 07:58, 5 November 2015 (UTC)

Re: Deliver a radical reading feature

WMF has often delivered "radical features" quickly, the problem is that they were catastrophes. I don't think this point is useful for anything other than proving WMF's irresponsibility (no need; users are already convinced). Nemo 07:58, 5 November 2015 (UTC)

Hi Nemo, I agree with your point about radical features and reworded it to 'substantive'. The point of the test is 'can we deliver on meaningful, substantive changes' and the test is of our ability to identify and develop such a change, as well as to successfully roll it out. If we can't, then it calls this strategy into question. As to your parenthetical about WMF's approach, while the substance was meaningful, the tone is offensive and does not move the dialogue forward. The same sentiment could have been expressed as "Radical changes might further a reputation the WMF has in some circles for being irresponsible", by posing such a concern as a joke, it changes from being a constructive comment to an insult, corrodes our good will, damaging our relationship and making it harder for us to communicate. I encourage you, next time you have some criticism in mind, to keep this in mind. Jkatz (WMF) (talk) 17:54, 5 November 2015 (UTC)

It's not an insult nor a joke, I'm serious; if you don't recognise it, you won't go anywhere (luckily, there is some acknowledgement). No need to be offended, I'm convinced the machine as a whole is broken but I know individuals are well meaning. :) Nemo 07:19, 6 November 2015 (UTC)

We are certainly hoping that we can partner with the community in a more substantive way this time around. We're being as open as possible with what we are thinking about and how we test our assumptions. In any case, radical features aren't an end in themselves but an avenue to happier users and that's unequivocally the goal. TNegrin (WMF) (talk) 03:00, 11 November 2015 (UTC)

Re: interactive content

Please stop using this term, which you presumably borrowed from some non-free knowledge industry. All our content is interactive, we host wikis. (Well, except the WMF marketing sites in HTML.) Nemo 07:58, 5 November 2015 (UTC)

Is there a term you are more comfortable with? TNegrin (WMF) (talk) 03:01, 11 November 2015 (UTC)

Re: think we have forums/comments

w:Wikipedia:What_Wikipedia_is_not#FORUM. I hope you won't be spreading false information about Wikipedia. The community doesn't like its careful and long-running communication efforts to be reversed by cannon-style incorrect propaganda by WMF. Nemo 07:58, 5 November 2015 (UTC)

Wikipedia is not a forum, but we already have talk pages, Village pumps and even a forum. The test is arguing the point that "If readers already see Wikipedia interactive, but do not want to interact, then there is no point in building more tools". How does this spread false info? Nemo, this is an open work in progress document, on wiki, the point is to discuss and edit as we move forward. In the scope of this assumption that the test wants to examine, what needs to change from your point of view? Thanks!--Melamrawy (WMF) (talk) 14:30, 5 November 2015 (UTC)

Perhaps the strategy could be expressed more clearly, in plain language, with less jargon. This might make it easier for us to understand what you are trying to communicate. Pbsouthwood (talk) 05:49, 6 November 2015 (UTC)

First of all, forums are so 2000... Compare 33% of American Internet users read or posted material in chat rooms, bulletin boards, or other online forums and 15 % of internet users read or comment in discussion forums such as reddit, Digg or Slashdot. We don't want to be associated with this idea of "forum", now less than ever. Don't mention forums or comments to any person, please. (BTW, there is hope in the universe: Facebook lost between 30 and 50 % of activity last year.)

Thanks for clarifying that you want to discover whether users "already see Wikipedia interactive": but what's the relationship to forums and comments? That's the kind of "interactivity" that was common in the 1990s, AFAIK; we are a wiki. To measure whether users "already see Wikipedia interactive", I'd ask whether they know there is an edit button and whether they ever tried to click it (this would be very useful to know). Nemo 07:16, 6 November 2015 (UTC)

I think the simplest way to understand the potential that might come from this feature is to consider that for many sites, search and social referrals are about equal. However for Wikipedia and sister sites, search refers far more page views than social (I don't have the citation handy, but it's something like an order of magnitude difference). If we were to have the same ratio as other sites, we would be reaching a lot more readers. However, as many people have pointed out, these types of features are very controversial. Clearly forums, comments and likes are typical methods of doing this on the internet but I would hope we could find a more appropriate way of doing this, perhaps by introducing readers to the interactive features already built in. TNegrin (WMF) (talk) 03:23, 11 November 2015 (UTC)

Abbreviations

Please don't use "WP". Nemo 07:58, 5 November 2015 (UTC)

Please, feel free to edit it. Thanks! :-)--Melamrawy (WMF) (talk) 14:25, 5 November 2015 (UTC)

What is UGC? Pbsouthwood (talk) 10:12, 5 November 2015 (UTC)

Also I am not familiar with UX. How much effort does it cost to completely eliminate abbreviations, or at least define them at first use? DO NOT ASSUME that your readers are all familiar with the abbreviations/jargon you use. Pbsouthwood (talk) 10:17, 5 November 2015 (UTC)

Good point, I wrote the full terms now. --Melamrawy (WMF) (talk) 14:25, 5 November 2015 (UTC)

Re: Comparison of ourselves against top 3 performers

A comparison is not a proof of anything. Nemo 07:58, 5 November 2015 (UTC)

Comparison, as in analysis, isn't proof of anything? How so? :-)--Melamrawy (WMF) (talk) 14:16, 5 November 2015 (UTC)

Re: hold for the top 5 language Wikipedias

This sounds like something that should involve Wikidata. If you tight yourself to 5 wikis only, the solution may not scale. Nemo 07:58, 5 November 2015 (UTC)

Agreed -- I think we need to think about what top 5 actually means in this context. Wikidata is certainly critical to test. TNegrin (WMF) (talk) 03:25, 11 November 2015 (UTC)

Re: readers preferring/requiring content in their local languages

This sentence should be the test and is interesting. The currently proposed condition «There is relevant content available in local languages» is instead our final goal. What we need to break is the vicious circle by which users don't find things on a local wiki (or never reach it because of Google) then stop searching there altogether then reduce contribution even further. Useful tests could be whether

• a set of users (a focus group?) can be made to use local search more (knowing they'll also be provided results from other languages where missing locally);

Interesting, would you like to elaborate more on how you see this test implemented? for instance, it will be 100% random, or will they use preset words for search, certain wikis to target, etc..--Melamrawy (WMF) (talk) 14:54, 5 November 2015 (UTC)

• promoting localised content can successfully drive away some (useless) traffic from the English Wikipedia;

Am not sure what is (useless) traffic, but how do you imagine the process of promoting localised content? --Melamrawy (WMF) (talk) 14:54, 5 November 2015 (UTC)

• visits moved away from the English Wikipedia (or other non-local language Wikipedia) result in higher activation in the mid term (most English Wikipedia users don't have sufficient language competency to contribute there, inter alia; contributing to wikis in their native languages is supposed to be easier and should be more likely).

There as an assumption that English Wikipedia users don't have sufficient language competency, regardless of the ^{[citation needed]}, do you mean that when more people find content in their local languages, it will eventually drive more traffic to those wikis instead of English? Makes sense, but the assumption around contributing to local language is always easier, isn't always true, for example, RTL editing has tremendously improved, but is still more challenging that LTR. Maybe you can elaborate more on this point. Thank you!--Melamrawy (WMF) (talk) 14:54, 5 November 2015 (UTC)

Nemo 07:58, 5 November 2015 (UTC)

Thank you for the time you put in reading and commenting Nemo, it is helpful and interesting altogether. :-) --Melamrawy (WMF) (talk) 14:54, 5 November 2015 (UTC)

Re: Identify trustworthiness

We never tell people to trust Wikipedia, please choose another term. (I saw such a measure tracked e.g. by Wikimedia Sweden, but that's because it's an externally provided official statistic and they can work with what they have.) Nemo 07:58, 5 November 2015 (UTC)

Re: Verify whether advertising improves odds

What sort of advertising do you have in mind? Something like m:Grants:IEG/What is about - C'est quoi. A series of communication tools about Wikipedia. Cameroon pilot project? Does that suffice as a test or do you have something broader in mind? Nemo 07:58, 5 November 2015 (UTC)

That's a great link; we also have some data from Wikipedia Zero that is relevant.TNegrin (WMF) (talk) 03:27, 11 November 2015 (UTC)

Gauging community interest or resistance

Create deep-dive (guided) educational experience
Objective: Gauge interest or resistance
Test: Email wikitech-l and wikimedia-l describing the simple-moderate-complex content tagging concept
If/Then Hypothesis: If feedback is positive, then the community will welcome it
Standard of Proof: The ratio of feedback is 10:1 (positive+neutral:negative)

There is a problem with the test and the hypothesis. Community decisions and views are determined at on-wiki RFC's. Trying to use a mailing list for this purpose is generally viewed as illegitimate. It may even result in a negative community reaction that spills over as hostility against the proposal itself. Try this instead:

Objective: Gauge interest or resistance
Test: Post an RFC at (at minimum) the largest Wiki Village Pump describing the simple-moderate-complex content tagging concept
If/Then Hypothesis: If feedback is positive, then the community will welcome it
Standard of Proof: The community closes the RFC with an affirmative consensus. 

Note: The community routinely returns consensus results at far closer ratios than 10:1. An affirmative consensus RFC close is a minimum standard. The WMF could examine an affirmative RFC close and apply any higher ratio it wants before proceeding. Alsee (talk) 07:01, 7 November 2015 (UTC)

Thank you for the feedback and guidance. I think this makes sense for this test.TNegrin (WMF) (talk) 03:28, 11 November 2015 (UTC)

TNegrin (WMF), cool. I'm not sure if this project is more about hoping to engage readers as new contributors, or about having simplified summaries for articles. For what it's worth, I predict targeting non-editors will obtain content of (at best) extremely erratic quality. Article Feedback Tool obtained submissions of such poor average quality that there was literally negative value in expending editor-time to read them. If there is ever a deployed-feature to have "article summary for elementary students", the existing editor community would fill them in extremely rapidly, for any article of moderate significance. Alsee (talk) 16:28, 12 November 2015 (UTC)