Nothing Special   »   [go: up one dir, main page]

Page MenuHomePhabricator

Conduct a Gather extension postmortem
Closed, DeclinedPublic

Description

The Gather extension fell apart on the English Wikipedia and has now been disabled.

I'd like to fully document what went wrong in a postmortem as I think the underlying idea is totally valid and needed, but the implementation was (somewhat predictably) problematic.

This task is a follow-up to T127509: Uninstall the Gather extension from en.wikipedia.org.

Event Timeline

In terms of page lists, we have categories and watchlists built in to MediaWiki core. And users can make lists of regular page links and track those using Special pages such as Special:RecentChangesLinked or Special:WhatLinksHere.

My sense is that we should have improved one of these features rather than creating an unintegrated and separate extension.

MassMessage and its use of ContentHandler may be a good example to look at. It may provide lessons for a better path forward.

I remember comments from early phases in the order of: "Yeah, but we just want to try out this simple thing on mobile and we don't even know if it works yet. If we do this by changing core, we will be bogged down in development of core for a gazillion years."

I'm honestly not sure what this would accomplish now. The Reading team has abode by community wishes communicated by an RFC and have disabled the feature. All of the people who were responsible for Gather on a decision making level have been gone for more than a year.

In spirit of looking forwards, not backwards, we are asking for community suggestions and feedback on features that keep readers more engaged. There's some very useful discussion there already.

In my opinion the lack of documenting 'lessons learned' amongst teams, is one of the reasons we kept seeing the same mistakes by multiple teams all the time. From a Team-Practices point of view, I think it is worthwhile to document what has been learned, before moving on, so that all teams, and future employees can make use of those lessons.

I'd naively hope that processes on a decision making level were documented more than a year ago, hence I'd also be personally interested in seeing "Lessons learned" identified, which might be another source of input for a future WMF product development process.

I'm honestly not sure what this would accomplish now. The Reading team has abode by community wishes communicated by an RFC and have disabled the feature. All of the people who were responsible for Gather on a decision making level have been gone for more than a year.

Respectfully, I don't accept this. I don't believe that all of the people involved in Gather no longer work at the Wikimedia Foundation. But even if that were the case, as long as you and the rest of the Reading team are going to stick around at the Wikimedia Foundation, I think it makes sense to clearly figure out what went wrong with Gather's implementation in order to learn from the experience for future projects. You and the Reading team will be involved in these future projects and the lessons of what went wrong (and right!) with Gather are almost certainly applicable to the projects ahead.

In spirit of looking forwards, not backwards, we are asking for community suggestions and feedback on features that keep readers more engaged. There's some very useful discussion there already.

Sure, a postmortem is looking backward. That's not a problem, that's its purpose. :-)

KLans_WMF renamed this task from Gather extension postmortem to Conduct a Gather extension postmortem.Mar 23 2016, 6:06 PM
KLans_WMF moved this task from To Triage to Team radar on the Team-Practices board.
KLans_WMF added a subscriber: MBinder_WMF.

Removing Team Practices Group. This task is in the Reading-Web-Backlog with @MBinder_WMF
subscribed to it.

I'll copy what relevant things I wrote in the discussion Nemo bis linked:

Gather was introduced to enwiki. In this case indeed, actionable blockers were proposed: collections could not be deleted, collections automatically used non-free content in violation of the EDP, the moderation system was very hard to use, and enwiki admins were being expected to patrol it with no benefit to serious contributors of content. Only one of these things was addressed. Complaints continued to be raised about the rest of the problems, but Moushira Elamrawy, the liasion for Gather, instead of understanding and passing on the gravity of these complaints, tried to pacify the community without taking any action about the problems. Eventually an RfC demanded Gather be removed. Then Toby Negrin needed a lot of convincing that yes, we actually meant it, and we wouldn't allow him to kick it into the long grass (link). Yes, the proposal looked reasonable on the face of it, but all trust had broken down.

(Here endeth the quote.)

It's also worth noting that the lack of trust was a result not only of the WMF's disastrous handling of Gather, but also of VE, MV, Flow etc. IMO Gather has only increased distrust and bitterness.

I'm honestly not sure what this would accomplish now. The Reading team has abode by community wishes communicated by an RFC and have disabled the feature. All of the people who were responsible for Gather on a decision making level have been gone for more than a year.

In spirit of looking forwards, not backwards, we are asking for community suggestions and feedback on features that keep readers more engaged. There's some very useful discussion there already.

As I wrote, the WMF did not abide by community wishes, in that instead of listening to valid concerns and fixing them in 2015, it had to be flamed and dragged through the mud to take action in 2016. If you think failing to respond to serious concerns about BLP and fair use is not something that needs to be learned from, you are gravely mistaken.

[offtopic] @BethNaught: Please criticize ideas, not people. Thanks for keeping Phabricator a respectful place.

[offtopic] @BethNaught: Please criticize ideas, not people. Thanks for keeping Phabricator a respectful place.

This task is about conducting a Gather postmortem. It is not possible to do so without discussing the actions of people involved. I do not mean to attack anyone nor to criticise unduly. I described the situation as it appeared (appears) to me and many others on enwiki.

Oh, that's just a tick of his; you can safely ignore this trope.

My guess is that one of @Moushira, @JKatzWMF, or @Tnegrin would be the person to resolve this task. I guess technically anyone could do it (including me), but it seems really unreasonable to ask others to write a report on this mess. If a few more months go by without any substantive response, it's probably fine to mark this task as declined. This forming albatross need not stay open indefinitely.

I found https://meta.wikimedia.org/wiki/Wikimedia_Foundation_metrics_and_activities_meetings/Quarterly_reviews/Reading,_July_2015 interesting, linked from https://www.mediawiki.org/wiki/Gather.

Lila: it hurts to suspend Gather right now
Toby: there are learnings here [...]

Part of the reason this task was filed, as noted in the task description, is that the underlying idea was and is sound. But the implementation and execution was so poor that this idea failed, perhaps even hurting future efforts.

If it matters, my impression was that Gather was insufficiently socialized and that the maintenance burden for the communities involved was underestimated.

While I think it's sad that no postmortem happened, there isn't much point in trying to force one, especially not a year later. This task should probably just be declined now.

I'll also note that the way this was brought up now (assumptions of bad faith and thinly veiled personal attacks by people with very little actual knowledge of how decisions were made) is deeply counterproductive and will probably make others even less inclined to participate in a postmortem.

While I think it's sad that no postmortem happened, there isn't much point in trying to force one, especially not a year later. This task should probably just be declined now.

A year later? This task was filed in March 2016, a few weeks after T127509: Uninstall the Gather extension from en.wikipedia.org was resolved. What else should have been done here? Shortly after this task was filed, with relevant people subscribed, Toby chimed on this task to say that everyone involved had left the Wikimedia Foundation, which is flatly not true. Then throughout 2016, this task got shuffled around as various people unsubscribed themselves from it.

I'll also note that the way this was brought up now (assumptions of bad faith and thinly veiled personal attacks by people with very little actual knowledge of how decisions were made) is deeply counterproductive and will probably make others even less inclined to participate in a postmortem.

You and others can keep deflecting here by pointing to tone and "assumptions of bad faith" and whatever other trivial excuse, but Maniphest, true to its name, provides a very clear record of what actually happened. A very failed software deployment was followed by nobody involved taking responsibility to even document what went wrong, much less commit to not repeating the same mistakes in the future.

When you or any of your Wikimedia Foundation colleagues attempt a similar future deployment, I can guarantee you that the lack of response here will not be forgotten. It is insane to think that this failed deployment lives in a vacuum, not affecting the reputation of and trust placed with the Wikimedia Foundation.

This task was filed in March 2016, a few weeks after T127509: Uninstall the Gather extension from en.wikipedia.org was resolved.

And it is 2017 now. I did not say the passage of time is somehow your fault, but it does make a postmortem less useful, as recollections of events become less accurate.

You and others can keep deflecting here by pointing to tone and "assumptions of bad faith" and whatever other trivial excuse, but Maniphest, true to its name, provides a very clear record of what actually happened. A very failed software deployment was followed by nobody involved taking responsibility to even document what went wrong, much less commit to not repeating the same mistakes in the future.

You'll have to make up your mind on whether you want a documentation of lessons learned or a public shaming of WMF staff who didn't act the way you wanted them to act at the time. If it's the first, in my opinion you and BethNaught are giving off the wrong signals here. If it's the second, I doubt you'll find people interested in working with you.

A very failed software deployment was followed by nobody involved taking responsibility to even document what went wrong, much less commit to not repeating the same mistakes in the future.

In my limited perspective (I only got involved towards the end) the most important software design lessons have been captured in Risker's checklist for content-creation extensions, and the people who had to did take it to heart. And for outreach and consensus-building there is the (draft) WMF product development process and Technical Collaboration Guideline which aim to capture a lot of repeating lessions. (Which is not to say a Gather retrospective would not have any added value, but IMO the low-hanging fruit is elsewhere.)

You'll have to make up your mind on whether you want a documentation of lessons learned or a public shaming of WMF staff who didn't act the way you wanted them to act at the time. If it's the first, in my opinion you and BethNaught are giving off the wrong signals here. If it's the second, I doubt you'll find people interested in working with you.

This is a classic false dichotomy.

You're also trying to equate people taking responsibility for a failed deployment—with its resulting waste of time, money, reputation, and goodwill—with public shaming.

And in between you're pointing at volunteers who showed up to ask why this reasonable request was never acted upon, some nine months after it was made and given ample time to be resolved, and you're suggesting that the volunteers themselves are to blame for the lack of response. Wow.

Do you really think it's difficult to see what you're doing here?

In my limited perspective (I only got involved towards the end) the most important software design lessons have been captured in Risker's checklist for content-creation extensions

I don't see how that's relevant. Those problems were already known in January 2015: https://www.mediawiki.org/wiki/Talk:Gather

MZMcBride, I agree with Tgr that it's likely a post-mortem here would result into fingerpointing, though there is a slim chance that public shaming would be avoided (that would be possible only if each responsible individual made a true apology per https://meta.wikimedia.org/wiki/Apology#Which_elements_should_be_included_in_an_apology , which sounds unlikely).

Unlike Tgr, though, I think that some kind of post-mortem or incident documentation is possible where individuals could collaborate adding facts to clarify the chain of events, without assigning blame. In particular, we could use a mere chronological list of events, public and private, which led to three specific events:

  • 2015-01-08: a DB schema and API documentation for a new extension are posted, before even writing down use cases
  • 2015-01-27/28: a justificatory project page is created which gives for granted the implementation already decided, despite the comments
  • 2015-02-05/10: the extension is created on and GitHub and gerrit (174c07cde101cef1a3adc462266c6d86e6e7a55c), despite the comments to the project page

It would really help to fill the gaps, because a lot must have happened, but all of it behind the scenes. Some examples.

  • What happened between April 2014 and January 2015 to the RfC? Why did it stall and what were the proponents doing? https://www.mediawiki.org/wiki/Talk:Requests_for_comment/Support_for_user-specific_page_lists_in_core#April_9th_update
  • Did the implementation proposal of early January 2015 really come out of the blue, or was there some preparatory document and discussion, maybe some dissatisfaction with the RfC process which made people suddenly abandon it and start something else from scratch?
  • How were the ideas, use cases and implementation proposals produced which ended up on the project page at the end of January 2015?
  • What private discussions happened, that brought people to ignore the universally negative feedback and proceed at full steam, instead of rethinking the project from scratch? (In particular before 2015-02-05, but also in subsequent remarkable checkpoints like deployments which should never have happened.)

Once the train is running downhill at full steam towards a precipice (as it was on 2015-02-10 already), it's rather pointless to debate who was pulling which levers and who moved which tiny pieces of coal where.

I believe I have avoided a "blame" tone in my list below, but I welcome any attempt to improve tone or clarity.

  • Pre-development input: This is an old lesson. Early community input has previously been identified as a key factor in the success or failure of a project. For some reason it is still hard to put this lesson into practice. The Gather project was effectively invisible to the community, up to the point it was announced for deployment. Community input during the concept and design phase could have identified significant issues in advance. The idea could have been re-thought, or cheaply canceled.
  • Moderation tools: The community has fairly high expectations that any user-generated content is tracked in contribution history, that all of the usual functionality be available for cleaning it up, and that cleanup actions be logged for accountability. Integrating any new non-wikipage user content into our existing systems is a significant undertaking. It should not be done lightly. The community may (uncomfortably) accept bugs or limitations in theses systems in a limited beta-test. However a project team must be prepared to either roll back the product, or to take responsibility for implementing full tracking&moderation functionality if content is left live for an extended period. This issue was addressed by Risker's checklist for content-creation extensions and T126952: Integrate Risker's checklist for content-creation extensions to the WMF product development process.
  • Was the maintenance burden underestimated? (Suggested in comment by @JEumerus.) I think it is important to note that this was NOT a significant focus in community discussions. Yes, there would be a lot to review, and yes AFD style keep/delete debates would be ugly, but the critical community concerns were not the what the WMF expected.
  • Nature of the content and nature of the work: The content goes against many community policies, and against the very reason that we volunteer. Editors volunteer their unpaid labor to generate publicly-owned content, as a public service to the world. Even user space pages are considered community owned, they are (loosely) expected to serve our global-good mission. Gather content is viewed as essentially "privately owned" social-network style random junk. Any labor greater than zero is like being an unpaid employee policing Facebook content. The community isn't a free labor force to police John Doe's personal junk, for John Doe's personal benefit. The work might seem similar, but community sees a significant difference.
  • Terminating a project must be explicitly in-scope for discussion: This issue lead to communication-breakdown and collaboration-breakdown on multiple levels. There were multiple requests from the community to the WMF for a collaborative process to resolve the issue. These requests were made to the liaison, to the project manager, and even to the executive director. All of those requests received evasive non-responses. (Is there some internal WMF policy or culture against responding to those requests?) Multiple requests were made for the WMF acknowledge that ending the project was in-scope for discussion. All of those requests received evasive non-responses. (Is there some internal WMF policy or culture against responding to those requests?) This issue eroded community trust that the WMF was willing or able to participate in good-faith collaborative resolution. It's hard to discuss fixing a project when the WMF can't acknowledge that ending the project might end up being the most appropriate outcome.
  • "Community doesn't like/want a product" needs to be considered valid and important feedback. If the community reasons are unclear or confusing, the actionable-response is to investigate and engage. The WMF shouldn't have missed the huge red flag when the Administrator's Noticeboard announcement turned into outright revolt. The community may have communicated its position poorly. The community tends to cite policy-jargon in its reasons. And in this case the community discussion took an odd turn... the community didn't want to engage in an uphill battle arguing against the project. The community basically said "You can keep your project but you'll have to pay staff to do 100% of the labor". This was basically a sarcastic position. The community did not expect the WMF to pay staff to police the content. The WMF was expected to realize that the project was non-viable without community&admin support. In any case, the WMF should have taken the situation seriously.
  • Avoid telling the community not to run an RFC. A number of editors wanted to pursue collaborative-resolution with the WMF. Morale for a collaborative process broke down for a number of reasons. Telling the community not to run an RFC is a sore point. This was a "last straw" that caused the final crumble in morale. At that point the community started the RFC, rather than wait for the WMF to announce its internal decision on Gather.

Well, it wasn't quite accurate that the maintenance burden was not a factor - in fact, the first RfC on enwiki was started in part because some collections were questionable.

What I mean is that the community wasn't quibbling about the exact quantity of work. If you could wave a magic wand and reduce the labor by 90%, it wouldn't have made a difference. People who opposed it still would have opposed it.

Jdlrobson subscribed.

3 years later with most of the people involved moved on I think we can say this is not going to happen.