Growth/Personalized first day

Wikimedia Foundation projects Personalized first day Provide experiences to new editors that help them achieve their goals Group: Growth Start: 2018-10-01 Team members: Gergő Tisza (software engineer), Benoît Evellin (movement communications specialist), [User:Etonkovidova (WMF) Lead: Roan Kattouw Management: Marcella Florence (engineering), Marshall Miller (product), Jazmin Tanner (program) Updates: Most updates posted here.

This page describes the Growth team's work on the "Personalized first day" initiatives, and contains the strategic thinking and links to the specific projects related to the initiatve. Most incremental updates on progress will be posted on the general Growth team updates page, with some large or detailed updates posted here. These are links to the three projects under the "Personalized first day" initiative:

• Welcome survey
• Engagement emails
• Newcomer homepage
• Newcomer tasks
• Structured tasks

When new editors create accounts, they are asked for minimal sign-up information and then are redirected back to the context from which they initiated account creation. This is a missed opportunity to learn what the new editor is attempting to accomplish and to provide them direction or content that helps them achieve their goals. Therefore, the "Personalized first day" project contains two parts:

1) Ask new editors additional questions during their account creation process, such as their reason for creating an account, what they are trying accomplish, topics that they are interested in, or whether they want to meet a mentor. This part of the project is called the "welcome survey".

2) After the login is created, we can direct the new editors to help content that is relevant for their goals, or to WikiProjects that match their interests, or potentially match them with a mentor who shares their interests. There are two projects underway to reach this second objective:

• Engagement emails
• Newcomer homepage

The Growth team is building Part 1 (welcome survey) first during 2018/19 Q2 (October 2018 - December 2018). After analyzing the results, we started pursuing Part 2 in 2018/19 Q3 (engagement emails and newcomer homepage).

Research has shown us that new editors have their own specific objectives when they begin to edit, and if they are not able to pursue those objectives, they are not retained. We also know that new editors have to learn many technical, conceptual, and cultural skills in order to be successful editors of the wikis, and that it is best to teach these skills through [[Special:MyLanguage/Growth#Solutions|"in-context" and "human-to-human" help]]. The "Personalized first day" project is meant to discover a new editor's goals by asking them, which gives us the ability to give them the "in-context" or "human-to-human" help that they need most to be successful. We prioritized this project because:

• It helps us address the issues brought up in the New Editors Experiences research.
• Community members were positive in their feedback on the idea.
• It helps us increase our learning at this early stage of our team's work with new editors.
• This sort of feature is almost universally common among other successful software platforms.
• The feature can be built and adjusted incrementally, without a major investment before it starts to show results.
• It will be easy to translate and apply to other wikis in the future.

The welcome survey asks newcomers a set of basic questions about why they've created their account and what they are interested in. It was first deployed in Czech and Korean Wikipedias on November 20, 2018, and on Vietnamese Wikipedia on January 24, 2019. The survey has successfully collected actionable data. For details and results, see this page.

In the Summary section above, we talk about how this project is divided in two parts. First, we are collecting information from new users about what they are trying to achieve and what they're interested in. We're doing that through the welcome survey. Second, we will use that information to personalize the experience they have on their first day, so that they can accomplish their goals and keep editing. After collecting about a month of data from the welcome survey, our team started to talk about how to take action on that data. Our central question for Part 2 is: How might we personalize a newcomer's experience to help them achieve their goals?

In this section of this page, we've recorded our thoughts and community discussion points around how to do this. Please participate here in the discussion.

The New Editor Experiences research identified six personas that most newcomers fit into. Those different personas have different skills and different goals, and we can increase retention if we develop an experience that is attuned to the persona of each editor. For instance, if we think a newcomer is a "Reactive Corrector", we might recommend a set of corrections to make. Our welcome survey questions are not granular enough to precisely classify newcomers into personas (that would require a much longer survey), but they give us some good information to work with. The table below shows an example of how we're thinking about this. Each row shows one of the six personas, along with a response from the welcome survey that might help us identify them, and a treatment that might go with that persona to help them be retained.

To think through the potential ways to take action on the welcome survey responses, we went back through our team's original lists of potential interventions, several of which the community discussed on this page. These are three main approaches we could think of around how to personalize the first day using the welcome survey.

Approach 1: use responses to surface relevant content somewhere on wiki

There are four separate ideas inside this approach:

• Task recommendations: surface tasks relevant to the reported goals, experience level, and topic interests of the newcomers.
• Relevant help materials: surface help documents that are specific to the task the newcomer said they're trying to achieve, and the experience level they said they have.
• Community visibility and topic routing: give newcomers visibility into activity and editors in the topics they care about, perhaps by allowing them to filter the Recent Changes feed by category, or by connecting them to WikiProjects. Perhaps by seeing that there is a vibrant community, they will be inspired to plug in and be active.
• Newbie forum: create a place where newcomers can have visibility into the work done by other newcomers, along with help materials specifically geared toward being new.

Looking at these together, we realized that if we built any of these things, we would not have a good place to "put" them -- there isn't a page that a newcomer can go to to get resources specifically around being new. Perhaps pursuing this approach might mean building some kind of "newcomer homepage", where they can find and access these things. This idea actually came up in the user testing for the welcome survey, in which a couple users said they would expect to be taken to "their dashboard" after completing the survey.

Approach 2: mentorship program

If substantial numbers of newcomers respond affirmatively to the question about whether they would like to receive help from an experienced editor, that might be evidence that we should consider how to connect experienced and new editors in some sort of mentorship program. This is an idea that has been attempted multiple times in the Wikimedia movement, with varied success (e.g. Adopt-a-user in English Wikipedia). The idea of "human-to-human help" is also one of the strongest recommendations from the New Editors Experiences research. The challenge would be how to do this right, and how to rally a community around it.

Approach 3: engagement emails

This is an idea that was discussed specifically by the community in September 2018: send emails to new editors about their impact, or to remind them to make their first edits, or to otherwise encourage them to engage. The results from the welcome survey could help us make such emails more relevant to the user, and speak to the goals, experience level, or topics they indicated in the survey.

After taking in thoughts from community members and discussing amongst our team, we decided in January on a path forward here that we believe will take the best parts of several of the approaches, maximize our chances of finding something that works, and minimize risk.

We have decided to begin two new projects:

• Engagement emails: begin a minimal email campaign containing only 2 or 3 different treatments.
• Newcomer homepage: develop a minimal homepage corresponding to the minimal email campaigns, that is also available for newcomers that are not emailable.

This is a “breadth-first” approach, allowing us to identify potential in two ideas instead of one, while also increasing our ability to reach people with the email project. For each project, we will make the minimum investment to discover whether something is working before we invest further. We also hypothesize that the projects can be most effective when a newcomer is experiencing them both. It is also worth noting that our approach will capture the "mentorship" aspect discussed above without endeavoring to build an explicit "mentorship program". What follows is the high-level reasoning behind our decision. There are many detailed considerations and potential pitfalls for both the projects. See the linked project pages above for those details.

Engagement emails

We believe that after new users close their Wikipedia tab on that first day when they create their account, they’ll generally not hear from Wikipedia again.  That means they won’t feel appreciated, won’t get new ideas of how to contribute, won’t find out about the vibrant community, and won’t get helped if they had gotten stuck.  We can use email as a proactive communication vector to reconnect with new users off-wiki to encourage them to make their first edits or to continue making edits.  Email is an effective engagement channel for most successful contribution platforms, and the conventional wisdom is that emails get the best results when they are relevant to a user’s interests and are actionable.

Our team’s work on the welcome survey allows us to capture information about a newcomer’s goals and interests, giving us a way to create emails that are relevant and actionable.  Given a combination of responses on the welcome survey and the newcomer’s edit history, we can choose the right email to send. Here are some of the calls to action we could send through emails:

• Impact: page view counts for the user’s contributions.
• Nurture materials to encourage editing: links to help and introduction materials to appeal to those least interested/familiar with editing on Wikipedia.
• Task recommendations: suggestions for contribution around topics of interest.
• Neighborhood: information and links for discovering activity around the user’s topics of interest.
• Mentorship: information on how to reach out to an experienced editor for help.

There are many open questions around emails, such as how often to send them and how to design them.  But one of the biggest questions is: when people click something in the emails, where will that take them?  Leading to...

Newcomer homepage

We know from our research that newcomers feel disoriented in the Wikipedia editing world.  While experienced editors know how to stitch together disjointed pieces of the wiki into a cohesive experience (talk pages, user pages, WikiProjects, watchlist, categories, page history, etc.), newcomers do not know how to use them together.  They wonder, “where do I start?” We can build an entryway to surface the things newcomers need to know and do first — a centralized location to orient them.  Such a page could contain the same kinds of content that we want to send via engagement emails, but in more actionable depth:

• Impact: page view counts for the user’s contributions.
• Nurture materials to encourage editing: links to help and introduction materials to appeal to those least interested/familiar with editing on Wikipedia.
• Task recommendations: suggestions for contribution around topics of interest.
• Neighborhood: information and links for discovering activity around the user’s topics of interest.
• Mentorship: information on how to reach out to an experienced editor for help.

Though we would want at first to build a newcomer homepage with sufficient content to be useful, such a page can be modular, with separate components added and subtracted over time. The layout and content of modules may also be further personalized based on difference in newcomer motivations.

There are many open questions around a newcomer homepage, but one of the biggest questions is: how to surface this to newcomers so that they visit and benefit from it?  Leading to...

Putting the two together

This newcomer homepage can be the landing place for engagement emails.  In other words, they are two sides of the same coin: the emails are our way of proactively engaging newcomers, and the homepage is the place where we bring them.  We can send a small amount of actionable information via email, and then have the full set of information available when the user clicks through the email. For instance: