Talk:Project management tools/Review/Options

Phabricator

---

• Help

---

Edit

Other criteria

Possible other criteria :

• Cost: difficult to assess. Hosted tools for which we pay a fee are easy, but self-hosted tools that we need to maintain also have a cost. Tools scattered across the Wikimultiverse also have a cost.
• Programming language: For FOSS tools, the language(s) they use will influence our ability to fix / tweak / expand functionality to serve our needs.

guillom 11:56, 24 February 2014 (UTC)

I've added the languages I could figure out. guillom 13:20, 24 February 2014 (UTC)

Some criteria I'd like to flesh out more on our major options is venture an evaluation of the health of the upstream project. Active development? Responsive to input? That sort of thing. These things are going to be important since we want to make sure that whatever we choose both meets our needs now and has prospects for becoming much more awesome. -- RobLa-WMF (talk) 21:28, 28 March 2014 (UTC)

Active developement vs. Scrumbugs: For the records, on April 4th, I asked on #scrum on Bugzilla IRC "Are there any plans to extend Scrumbugs functionaity to provide more than a non-interactive board for teams, like card view per assignee which supports drag&drop or such?" and pmac (maintainer) answered "I don't personally have such plans, but I'm very open to merging patches to take it in that direction". --AKlapper (WMF) (talk) 15:45, 7 April 2014 (UTC)

Discuss options

Status quo

Status quo is use of too many tools that overlap and aren't well-integrated:

1. Gerrit
2. Bugzilla
3. Mingle
4. Trello
5. Other stuff?

This means that volunteer contributors have to look in three places (wiki, gerrit, bugzilla) to get information about MediaWiki. WMF staff/contractors have it worse, potentially looking in four places (wiki, gerrit, bugzilla, trello/mingle). Is anyone really happy with the inflexibility and arcane UI of Bugzilla? Is anyone really happy to look in three different discussion venues to talk about one issue/one patch? It costs us all countless hours, and increases the learning curve for new staff and new volunteer contributors. Also: gerrit + git-review makes me want kill myself, and is going to frustrate anyone used to GitHub, Phabricator, or other more usable systems that have decent devtools and where patches are closely tied to easy issuing tracking. TL;DR: what we've got works, barely. We can do better. Steven Walling (WMF) • talk 13:00, 6 March 2014 (UTC)

Pivotal Tracker

Pivotal is extremely popular, but would only replace our project management tools like Mingle and Trello. It is not a suitable replacement for a public bug tracker or code reivew tool, and is designed to instead integrate with those. This is basically 1/2 way between the usability of a tool like Trello and the Agile-centric way Mingle works (with reporting tools, etc.). Despite the fact that it's proprietary, they give indefinite free licenses to public projects and FOSS projects, so we would not have to pay for it like Trello or customize it to death like Mingle. It also has a good API and integrations with tools like Bugzilla out of the box. In light of that, I think we should consider this as a backup option for centralizing on a single WMF project management tool, if an effort to integrate everything on top of Phabricator fails to take root. This would not improve the situation with reliance on non-FOSS tools for WMF-related team management. But at least it would be free as in beer, highly usable, well-integrated with Bugzilla, and centered on Agile practices. Steven Walling (WMF) • talk 12:49, 6 March 2014 (UTC)

Fulcrum

This one looks kinda interesting from the screenshots at least. This might be worth setting up a test instance for. Has anyone seen this in action? -- RobLa-WMF (talk) 05:37, 5 March 2014 (UTC)

I think the things we like about Fulcrum (i.e. the project view with swimlanes) could be/is developed within Phabricator. Steven Walling (WMF) • talk 12:45, 6 March 2014 (UTC)

Is this all what we have discussed and about Fulcrum during this long review? If so, I would just remove it from the list. If someone comes with stronger arguments during the RfC we can always bring it back to the table.--Qgil (talk) 05:22, 27 March 2014 (UTC)

Scrumbugz

This one has all of the outward appearances of a poor choice. It's difficult to evaluate the user interface based on the website, and the project hasn't seen a commit in four months. I somehow doubt that's because they're "done".  :-) Does anyone find this option at all interesting? -- RobLa-WMF (talk) 16:31, 4 March 2014 (UTC)

I think it was just mentioned in passing on the list. I don't think it's viable either. Will remove. Steven Walling (WMF) • talk 17:36, 4 March 2014 (UTC)

AFAIK, Wikimedia Germany is currently experimenting with Scrumbugz, so they hopefully should be able to comment on it. --AKlapper (WMF) (talk) 19:10, 5 March 2014 (UTC)

For what it's worth, I have been looking at the options presented and consider this one to be the most interesting so far. I think regardless of what we pick, we need integration with bugzilla. Since Scrumbugz is built around this requirement and seems quite flexible, it's worth at least a look. I was able to get a running instance in about 30 minutes on my local machine, would anyone be interested in me setting this up in labs? --DAndreescu (talk) 20:01, 6 March 2014 (UTC)

For testing: A (temporary) Scrumbugs instance to test is already available on Labs at [1] - people could play with it and get a better impression on its functionality and UI. --AKlapper (WMF) (talk) 15:01, 6 March 2014 (UTC)

Whoops, my previous comment might be misleading. That Labs instance is in production use by WMDE so it's nothing to "play with" data-wise, but you can totally check out its UI. --AKlapper (WMF) (talk) 15:51, 6 March 2014 (UTC)

Looking at the UI... I think this is not good. It basically looks like just a bunch of Agile reporting on top of bugzilla tickets? Not sure that meets our needs, and the UI needs a ton of work to be decent. Steven Walling (WMF) • talk 00:31, 11 March 2014 (UTC)

Having fully evaluated and implemented Scrumbugz, I can provide some light as to what it does and does not do. It is first and foremost a 'stop gap' measure that performs a very specific function. WMDE needs a way to better manage the existing data stream coming from Bugzilla. And, for this purpose, Scrumbugz is not bad. The UI is not where Scrumbugz excels, obviously. Yet, the code is quite excellent in the way it is able to continuously synchronize with Bugzilla. It may be the only tool that can do this, so this is its primary selling point and greatest advantage. --Christopher Johnson (WMDE) (talk) 11:53, 14 March 2014 (UTC)

Thank you for your first-hand opinion, Christopher Johnson (WMDE). Let me also ask this question to Lydia Pintscher (WMDE), have you tried or discussed the possibility of using Phabricator? Are you following the discussion in the team-practices mailing list? What is your opinion on all this? If the main reason for using Scrumbugz is its complementarity with Bugzilla, and if Scrumbugz is the only relevant candidate in this debate apart from Phabricator, then the acual options we have are: keeping the status quo (Bugzilla, Gerrit, Trello/Mingle/Scrumbugz, because I don't see the Trello/Mingle teams moving to Scrumbugz just because) or moving everything to Phabricator (because Scrumbugz becomes between useless and redundant with Phabricator, since Bugzilla would be gone and there is already a project management component).--Qgil (talk) 05:33, 27 March 2014 (UTC)

We have not at the time since Phabricator would not have made sense to use just for us. Scrumbugz isn't perfect but it has seriously helped us over the past two sprints since we started using it. Since we've been managing tickets in bugzilla already anyway this was a nice addition. --Lydia Pintscher (WMDE) (talk) 07:53, 27 March 2014 (UTC)

iceScrum

This is FOSS but is ugly as sin. We could do better. Steven Walling (WMF) • talk 17:37, 4 March 2014 (UTC)

It's also partly proprietary. As noted in the table, they have an "iceScrum Pro", which includes many key features (e.g. connecting commits and issues, continuous integration, LDAP). The very fact of the Pro separation means the community is divided, and there's an inherent tension. E.g. if we implemented linking commits and issues in the open source version, would they accept it to merge upstream? Superm401 - Talk 05:33, 5 March 2014 (UTC)

GitHub

While I've expressed interest in Phabricator for having enjoyed working with it in the past, I didn't realize that GitHub covered our needs in terms of continuous integration, etc. With that in mind, and since working with GitHub on open source projects has always been a pleasant experience, I think that GitHub has an undeniable advantage: we would have more 3rd-party contributions on GitHub than we would with any solution we host ourselves.--GDubuc (WMF) (talk) 08:49, 4 March 2014 (UTC)

Operations is not likely to be excited about deploying to the prod cluster from a repository that is not entirely under our control. I talked to GitHub a couple of years ago about auditability of their hosting and was told that the SaaS hosted version did not support security audits. They recommended their "Enterprise" product which is a VM image that is hosted internally by an implementing organization. GitHub being non-free (libre) seems like a large drawback. When we find things that don't work well for us there (when, not if), we will be at the mercy of the GitHub team to correct the problems. --BDavis (WMF) (talk) 15:58, 4 March 2014 (UTC)

I think GitHub should be out of scope for this conversation. It doesn't really address the project management issue, and its issue tracker requires a non-WMF account. It *might* be ok to ask our developers to sign up for an account on an outside service, but asking site readers and editors to sign up for a GitHub account to file a bug is a non-starter. Whether we use it for repo management is a separate conversation (probably still not likely, but a more interesting conversation nonetheless). -- RobLa-WMF (talk) 16:19, 4 March 2014 (UTC)

I agree that this is a non-starter; I think that we should use it as a benchmark for comparison, rather than evaluate it as an actual target for use. Jdforrester (WMF) (talk) 18:17, 4 March 2014 (UTC)

What James said. Legoktm (talk) 18:04, 5 March 2014 (UTC)

Phabricator

I think that in terms of FOSS solution, this is as good as it gets at the moment. I'd lean towards Phabricator over GitLab because the foundation has much more PHP expertise than Rails. After all the foundation could have improved gerrit to make it more tolerable, but didn't, probably because of unfamiliarity with Java. We would definitely have to contribute code to Phabricator, because some features in beta need improvement, but their team is very helpful and responsive to contributions. Taken as-is, though, it's definitely much more basic and rougher around the edges than GitHub for some features.--GDubuc (WMF) (talk) 09:00, 4 March 2014 (UTC)

+1 to this. After seeing how far Phabricator has come in the past couple of years, I'm pretty excited about it. Regardless of whether we migrate to it for repo management, it would seem to work well as an issue tracker. The project management tools are still a bit basic, but seem to be heading in the right direction, and I think we can realistically expect that they will be solid in a year or two (especially if we help them along). Of the FOSS options out there, this seems the most viable. -- RobLa-WMF (talk) 16:10, 4 March 2014 (UTC)

As Gilles has mentioned, this is really the standout solution at this point. While GitHub vastly outperforms all others in its ability to give us global reach to the open-source community, it does not give us any sort of reasonable project management options for products of our scale. I can see our people making many additions to Phabricator, as there is quite a bit that we could still improve on; but hey, that's where this whole FOSS thing becomes useful. As-is, it does suit our needs, and it will definitely speed up my team's development process. In addition, it makes cross-team changes more visible, and more manageable.

But most importantly: it really makes the process of reviewing code simple and painless, and actually lets you have local git revision history for changesets! This is huge compared to the inane mess that is git review, which requires you to repeatedly amend. Being able to easily apply patches for testing, revert, squash, and merge is a particularly useful feature of this software stack. SG (talk) 09:33, 5 March 2014 (UTC)

This means you can have more than one commit in a changeset (ala GitHub pull requests)? I agree that would be quite nice. Superm401 - Talk 05:46, 6 March 2014 (UTC)

I've played with Phabricator for about 5 minutes and like it better than gerrit. The only thing I don't like about it is that it seems to require a login before you can view anything? I'm assuming that's a configuration setting we can not set, but it is annoying that to view a bug in Phabricator itself, I need to create an account on their site. Legoktm (talk) 18:08, 5 March 2014 (UTC)

Yeah, I don't get why it requires accounts for views (oddly, when I went to retest this, it no longer required me to login, but maybe I was looking at a different page type). Anyway, this should be relatively simple to fix and/or configure. Superm401 - Talk 05:46, 6 March 2014 (UTC)

I believe it was originally designed that way because it assumed two things: 1. that the content was probably private (original use case was Facebook) 2. even if it's not private, anyone who is viewing should also probably be contributing, so contributions need to be attributed to an account. Steven Walling (WMF) • talk 12:43, 6 March 2014 (UTC)

For testing: A (temporary) Phabricator instance to test is already available on Labs at [2] - people could play with it and get a better impression on its functionality and UI. --AKlapper (WMF) (talk) 15:01, 6 March 2014 (UTC)

Can that test instance be configured to auto-approve new accounts? --Waldir (talk) 02:32, 18 March 2014 (UTC)

Still needing an answer to Waldir's question; I added a #Reporters engagement ideal section just in case. --Nemo 10:42, 27 March 2014 (UTC)

 Done (I think) by flipping the switch at http://fab.wmflabs.org/config/edit/auth.require-approval/ (you probably can't see that page, but I'm linking it so other admins can revert if needed. guillom 14:24, 28 March 2014 (UTC)

I like Phabricator. It's come a long way since we talked about it a couple of years ago. It's nice that we could roll more of our tools (issue tracking, project management) into the same home as our code review & management. It's all very slick and seamless. Plus, there's something to be said how *freaking easy* it was to setup and manage. No JVM! You just grab the code and configure the Apache vhost. ^demon^{[omg plz]} 18:19, 6 March 2014 (UTC)

Along with Gilles I have been using Phabricator for the past few years (?). Seems like just about two. In Operations we used it for all the same things as the development group. We pushed Puppet code through review, tracked tickets, and used the Wiki. We were using it in combination with Gitolite. This allowed people to pull down and view an Ops controlled repo, submit a diff, and get it into the pipeline while still needing ops to land it into production. However, everything short of landing it was possible through Phabricator. We also built a Phabricator 'app' that showed the results of our cruisecontrol tests, which was a terrible hack at the time since it wasn't quite as modular as it is now. But it worked. We were able to integrate with our chat bot to show new tickets, query tickets, and eventually we hooked into their basic IRC history log interface. Phabricator as a Web UI is good. It's slightly better than most. Phabricator as a suite of tools for deployment lifecycle using Arc is damn good. Maybe even fantastic. Using ARC you can put 'fixes T1111' in your summary and have a diff that closes it's own ticket linking them automatically. You can say 'depends on D1111' and link diff's as dependencies easily w/o touching the UI. You can use a branch to track a change and keep your history, instead of a mindset where each commit needs review. This allows back and forth inside of the differential process in a more natural way for Git. We also easily created a new Oauth provider for in-house use in December when I was still at my last job. This could almost certainly be done here as well for mediawiki auth integration. I have used Jira, Github, Trac, and Magic in the past. Now briefly, I have used Gerrit and RT. Phabricator wins that fight. It has got some warts, doing branch dependencies on each other can get messy. That's mostly just Git showing through. It is also under pretty active development so depending on how bleeding edge you want to be there is an occasional bug, but the #phabricator people have been helpful in the past. CPettet (WMF) (talk) 21:25, 14 March 2014 (UTC)

Bloodhound

Apache Bloodhound is basically a fork of Trac, to solve some its nastier usability problems. Not well-supported and it looks a bit too much like traditional project management packages to me. Steven Walling (WMF) • talk 12:55, 6 March 2014 (UTC)

GitLab

This basically looks like a FOSS GitHub clone? It also looks to be another "open core" type of project, where they push you to use their hosted version and the free/open version is more limited. Not sure it's suitable. Steven Walling (WMF) • talk 12:55, 6 March 2014 (UTC)

Just moved GitLab to unlikely, since it seems to miss the PM part, which is the core part of this review.--Qgil (talk) 05:20, 27 March 2014 (UTC)

I don't think that alone warrants demoting the option: neither the current ecosystem nor any of the current proposals replaces the full stack, so use of multiple tools is unavoidable. In fact, GitLab seems to complement the PM-only tools Fulcrum and Scrumbugz quite nicely, so we could choose to adopt a combination of them. There may be even tools out there that integrate with it (or plugins) to provide the missing functionality (e.g. I know there's http://waffle.io, which provides Trello-like boards for Github). I'll move Gitlab back up until there's more discussion on this. --Waldir (talk) 21:27, 27 March 2014 (UTC)

Would that proposed combination reduce the current complexity (having half a dozen of tools trying to somehow interact with each other) plus integrate better than our current tools? I don't know the answer myself, but that should be the main question to ask. --AKlapper (WMF) (talk) 12:29, 28 March 2014 (UTC)

Reporters engagement ideal

Context: this Project management tools process so far only involved WMF (and WMDE?) teams, which is fine as long as it only affects them. If they reach a conclusion and are so convinced as to involve gerrit, I suppose a wide discussion as for SVN->Git will be held on wikitech-l etc. For bugzilla we don't have precedents, but I give for granted that it won't be affected without an actuall discussion about it involving all its users: after all, Requirements don't have a single line on reporters, the bugzilla side of the evaluation is only at very early stages.

• Assumption: we need more bug reporters. Too often, bugs go unnoticed, or get noticed but go unreported, because people don't bother or because they're dropped on one of hundreds' Wikimedia wikis, or a talk page, or IRC, or a mailing list, or a non-Wikimedia site; too many misplaced bug reports and enhancement requests need to be manually filed in bugzilla by too few people. (The number of reports we have is not immane but rather small, compared to RedHat's or Mozilla's bugzilla; Confluence has a similar amount but way less users, even though the main org's budget is similar to WMF i.e. in the tens of M$.)
• Ideal:
  1. filing a bug should be very easy and not require registration, as in our CLDR friends' trac tracker, or allow authentication with an account we're sure the reporters have (ideally, any MediaWiki instance);
  2. and it should be so for everyone in all languages: the issue tracker should be available in as many languages as MediaWiki (e.g. localised on translatewiki.net); users should be able to write and discuss their issues in any language and let someone translate.

We currently have no idea how Phabricator or others would bring us any closer of farther from the ideal. If/when a bugzilla replacement evaluation starts, this and other things will need to be considered and summarised. --Nemo 10:35, 27 March 2014 (UTC)

I only support letting anybody file tickets without registration if the ticket system provides good spam handling (throttle comment, report and account creation). "Too easy" also means that it could be intentionally (ab)used to get attention for stuff that aren't bugs so content-wise we'd end up with hundreds+1 talk pages. --AKlapper (WMF) (talk) 11:21, 27 March 2014 (UTC)

Sure, it's an ideal because some things make it difficult. :) Spam is the first. CLDR's trac seems to manage, so it's not an unresolved problem of computer science. --Nemo 11:30, 27 March 2014 (UTC)

I think our baseline should still be that bug reports, tasks, and patches can be created by identified users only. Wikimedia userID (OAuth to the rescue) would be great (required?), associated with a unique email address.Users can be as anonymous as they want, but we are talking about collaboration tools, and replying to anonymous that won't even receive a notification is not inviting.--Qgil (talk) 23:10, 27 March 2014 (UTC)

I am sometimes talking to "IP"s and are often surprised to get a useful response because I didn't expect it. Consider people who are too lazy creating an account raising issues on twitter like recently happened after the typographical-refresh-update. And (going beyond the scope of the review) what about a "report a problem" link in the sidebar, or better next to "developers" at the bottom of the page? -- Rillke (talk) 15:13, 15 April 2014 (UTC)

There is a "Bug tracker" link in the sidebar under "Development". --AKlapper (WMF) (talk) 15:45, 15 April 2014 (UTC)