Growth/Personalized first day/Newcomer experience marketing pilot

Video used on the campaign-specific create account page

The newcomer experience marketing pilot is an experimental Wikimedia Foundation pilot project that is running in May 2022. We are testing the efficacy of social media ads (bringing users into our projects) and welcoming emails (retaining new users). Once these users have signed up, we are testing the effectiveness of the "personalized first day" newcomer homepage.

Background and plan

The Wikimedia Foundation is experimenting with tactics that we believe would increase the number of new editors if we expanded them into a full-scale project.

This experiment, which is part of a larger series of FY21–22 pilots that are exploring different ways to drive Wikimedia editor growth, focuses on Latin America and the Spanish Wikipedia. It involves the following initiatives:

We will bring in potential new editors through social media ads (Facebook)
We will enhance the account creation user experience creating a campaign-specific create account page
Once inside, we will measure the effectiveness of the newcomer experience features
Finally, we will send a welcome email to new editors who opt in, featuring a video with interviews with Spanish Wikipedia community members

The social media ads will be targeted at users with interests like writing, editing, learning, research, and academic writing—the sorts of interests that you might expect would be correlated with a Wikipedia editor. Our plan is to run these ads for no more than four weeks, and probably less than that. The welcome emails will go out about one to two weeks after the ads. As this is a small pilot project, we do not believe that it will have any major negative effects on the Spanish Wikipedia and its editors. We previously announced the project on the encyclopedia's cafe/village pump.

Example content

Video used in the welcome emails

We are running both standard and "Canvas" Facebook ads. For an idea of what the Canvas ads look like, see this information page.

What are our goals?

To demonstrate that these tactics would, if scaled up, achieve the following benchmarks:

Increase new account creation by 5%
20% of new accounts are "constructively activated" (i.e. making one or more unreverted article edits)

Beyond those goals, we are hoping to find that these tactics have the potential to offer real benefits to the Wikimedia movement, regardless of whether that's across entire wikis or in more limited scenarios.

While we have significant doubts around whether social media ads could be a realistic everyday option, we would, for example, be happy to discover they are effective enough to use during events.

What metrics are we using to measure success/failure?

We will be collecting standard anonymized metrics from Facebook, such as reach, engagement, click rates, and similar. On-wiki, we will measure how many people land on the campaign-specific create account page, end up creating an account, and are constructively activated.

We're using Facebook to reach people on a non-wiki platform that is heavily used in Latin America, and for its ability to show ads to people based on interests they've demonstrated. The equivalent Wikimedia option (central notice banners) does not have similar capabilities and would bring in an audience that has too many motivations and interests for one campaign to handle.

The use of Facebook has been vetted and approved by the Wikimedia Foundation's Legal team. We are only using features and information that Facebook itself generates and provides, like user interests. No personal information from our side will be shared with the platform.

Who is involved in the project?

The Wikimedia Foundation's Marketing, Community Relations, and Growth teams.

What if I have questions?

We are watching the talk page!

What kind of results did we get?

Newcomer experience marketing pilot final report – Analysis of this pilot's effectiveness
Growth/Personalized first day/Newcomer experience marketing pilot/Welcome emails experiment, September 2022 – a Wikimedia-side analysis of the email part of this pilot