Jump to content

Growth/Growth 2014/Retrospectives

From mediawiki.org

This document records for the Growth team in Wikimedia Engineering. These are rough notes for the most part. Growth retrospectives typically happen every two weeks the day after an iteration of work (i.e. a sprint) ends.

September 22 2014

[edit]
Things to start doing
Things to stop doing
  • Destroying the most awesome team ever
Things to keep doing
  • Nothing
  • Releasing awesome stuff
  • Hanging out
  • Not having feature creep

September 5 2014

[edit]

Observation: Wikimania distrupted the development of the Task Recommendations experiment.

Keep doing (TOKEEP)
  • Usability testing at Wikimania
  • Hypothesis generation between design, research & engineering. (do it more and do it better)
    • Make this an explicit task in spec'ing out a project
Stop doing (TODONT)
  • Putting off coming to a conclusion about experimental design
  • Losing all of the video from your usability tests
  • Don't schedule building an experiment around a conference
  • Don't feature creep
    • e.g. bullet clicks -- necessary vs. nice
    • e.g. flyout's initial work
Start doing (TODO)
  • Be strict about committing to an experiment design before starting to build it
    • Both features side (UI, etc.) and research side (hypotheses, duration, etc.)
  • Explicit blockers list for the deployment of major features
    • Bring up new potential blockers during CATSOUP!
    • Identify blockers early
    • If it's related to the feature, but not a blocker, don't do it.
      • Instead, do the blockers.
      • If a blocker is externally blocked, so we have nothing to do, do something else from the backlog.
    • Embrace feature freeze
  • When in doubt, over-spec (what's a new user?).
  • Try and have at least one experiment specified before a sprint-planning meeting

July 28 2014

[edit]
Things to keep doing
  • Deadlines on as many cards as possible, even if the deadline is just for review of designs etc.
  • Team meetups all in the same location! This was a very productive meetup.
  • Being willing to iterate a lot on mocks/ideas and even throw them out.
Things to start doing
  • Discuss theory (shared ideas between research, design, engineering and product) before the mocks come out
    • Goal: Do experimental design and UI design at the same time, or in general much earlier. Research and design should happen simultaneously, so that the we have clear coupling between research questions and the UI.
    • Means: Roadmap as a list of questions we want to answer, before start design phase.
    • Means: Make sure that there are preliminary designs for discussion ready by the time we want to make something a sprint goal. This also means making sure there is appropriate feedback.
    • Means: Make a card as early as possible and make everyone a member
Things to stop doing

N/A this time

July 11 2014

[edit]
Things to start doing
  • Deadlines on design cards and set up a followup meeting to review results on the deadline
  • Ping people if you think something is ready for commenting
  • Making a list of the cards we want to bring up in Scrum of Scrums
  • Before we start work on implementing a (large, not minor) mediawiki.ui upstream patch in core, bring it up to other teams using the library (Flow, Mobile) and to the UX team earlier, possibly on design list, Bugzilla, or mediawiki.ui Trello board
Things to stop doing
  • Tracking of “sprint velocity” using number of cards assigned/completed. Instead just talk about whether we completed the goal or not. We should pick it up back if we start doing points estimation of tasks.
  • Making Kaity go to retrospectives and sprint planning, as secondary.
Things to keep doing
  • Canning implementation discussion in standup.

June 30 2014

[edit]
Things to stop doing
  • There was a 30 day gap in between this and last retro. That's too long.
  • Overflow meetings, where we don't get things done in the hour so we add another hour. We need to be stricter about not talking about details of technical stuff and design in standup, sprint planning, etc.
  • If we're going to have one sprint planning meeting, then we need to have the core set of cards for the iteration specified and prioritized beforehand. Otherwise, we have to have two meetings, one for planning and one for estimation.
  • [Still under discussion] Scheduled design checkins on a biweekly basis. Instead we should schedule Hangouts when designers have mocks or sketches, instead of only discussing ideas in the abstract.
Things to start doing
  • Start calling this meeting "spretro" ;)
  • Start spretro by reviewing the to-do items from the previous spretro.
  • If someone is gone for a sprint, consider taking out cards or not assigning at all
    • If we know that they'll be gone before hand, then don't commit to them at all.
  • To-do items that have lingered for more than one previous sprint, we should consider deprioritizing or not assigning new work.
  • Actually standing up in standup!
  • The team should be more vigilant about butting in when we're talking about implementation details during standup etc.
Things to keep doing
  • Quick planning meetings, if we're still iterating on the major goals from last sprint.
  • Standups every day.

May 30, 2014

[edit]
Things to keep doing
  • Shipping! We got the experiment out, we fixed bugs quickly, and we did 70-75% or more of cards in the last two sprints.
  • Continue to use bucket o' mess and prioritized backlog. We made this mid-sprint and it's a good idea.
  • People emailing team list with logistical stuff, like when you're going to be late or away etc.
  • Keep including technical debt in the sprint commitment.
  • Trying to have sprints have a theme or relation between cards, not just having a big mess of cards assigned to each person.
  • Organizing a team summit. We need to do this.
Things to start doing
  • Make sure there's plenty of user-facing work lined up to do once we've pushed a feature/experiment. (This is leftover from last retrospective 10 days ago.)
  • More user stories in cards that are user-facing (i.e. As a buffalo, I want to buffalo, so that I can buffalo buffalo.)
  • (Start again) regular design checkins including everyone. We've been meeting intermittently with Steven/Kaity or Moiz/Steven.
  • Consider alternative ways to do design brainstorming and idea collection with the remote group. It might have helped to have a longer idea generation and refinement time, either in one long meeting or broken up in to two. What about tools other than EtherPad? We should explore using a spreadsheet, Trello, sharing sketches, digital whiteboards, etc. if we can, to help a more creative meeting be high energy.
  • We need a mascot! And t-shirts. Fun things make work fun.
Things to stop doing
  • Being late to meetings! We've all been guilty of this. This is wasting peoples time. It doesn't block progress but it's annoying.

May 20 2014

[edit]
What should we keep doing? What went well?
  • Pinging people outside of Trello about immediate blockers. (This should include the standup too.)
  • Having retrospectives. ;-)
What should we start doing? (New ideas and ways to improve.)
  • Accounting for flight time when noting which meetings we (Aaron) will miss.
  • Adding more stuff to the sprint, so we don't move things in from backlog.
  • Assign cards evenly to everyone at the start of the sprint.
  • When an A/B test is about to launch, must have a plan for what we're going to work on while it's running and in analysis.
  • When we're about to launch and we have any complications/questions/doubts, we should just do a Hangout to talk about it, rather than try to catch up asynchronously
  • Brief architecture meetings up front will help with the same issue above, i.e. talk about how we're going to tackle non-trivial items.
  • Aaron states his assumptions about instrumentation of schemas, e.g. client side versus server side.
  • Cards at the top of the backlog need to have clear definitions of done.
  • First thing: Go over past retrospective notes to say what we were gonna change.
What should we stop doing? (Things that blocked progress.)
  • Starting a sprint without a clear product goal/thing of value to deliver to users. This time we just pulled random stuff from the backlog. We're still getting good stuff done, but it's less clear what the priorities are.

May 7 2014

[edit]
What should we keep doing? What went well?
  • Adding cards for only essential items mid-sprint
  • Decomposing cards. There was a larger number but we actually got more done even though.
  • Strong levels of communication in the team. It's good having a smaller tight-knit team. We don't think adding more people would not necessarily add more Mythical Man Months. ;-) Not duplicating effort. Items are clearly assigned to one person.
  • Kept up momentum despite people being at conferences or on vacation.
What should we start doing? New ideas and things to improve.
  • Showing up to meetings on time
  • Take existing cards into account when adding new cards.
  • Strongly consider not adding new cards until the backlog is clear from things in Needs Review/QA. Alternative is to take things out of Current Sprint To Do.
What should we stop doing? What blocked progress?
  • Cards sitting around in Needs Review/QA list. What can we do about this?
    • There are things in our control fully. What blocks these is having the time in a sprint to get these reviewed/built, and we should finish them before we add more to the sprint.
    • There are things we wait for review outside the team, like core fixes. Solution is Scrum of Scrums and bugging individuals.
  • Rollback deployments due to bugs. :( We wouldn't have necessarily caught the bugs in code review, but it would have maybe been better to not have him deploying alone.

April 18, 2014

[edit]
What should we keep doing? What went well?
  • R&D talks design with designers
  • Design team plus R&D (Aaron, Pau, May, Leila, Oliver)
  • Splitting up research projects into background, analysis, and reporting tasks
  • Each card has a definition of done and a user story attached to it (when relevant)
  • Scheduling time to talk about investigating topics/how to solve a problem
What should we start doing? New ideas and things to improve.
  • Provide instructions on how to test a patch when it moves to the Review stage.
  • This was particularly true when it comes to testing the data pipeline which is more complicated.
  • When we want to deliver something that will span two sprints, being up-front about that
  • Never ignore a -1 from Siebrand and other i18n staff/translatewiki admins. This was bad.
  • Stuff is ugly and inconsistent all over Wikipedia. Moiz needs to get on Gerrit, start pushing krazy klean kode, create havoc for all frontend engineers.
  • Solution is for Pau/Moiz/Steven/Juliusz to work on git-review etc.
  • Be sure bring up things like Schema:PageRestoration
What should we stop doing? What blocked progress?
  • Interviews: these take time. How can we balance them with the needs of the sprint?
    • Matt did five interviews for Director of Community Engagement and Growth position.
    • Sam did one for Growth
    • Steven, Aaron, Moiz, and Pau did four interviews for the UX researcher position + reviewing candidate tasks
    • Aaron reviewed/screened ~30 applications for new Researcher rec. (Fundraising).
  • Spending so much time bringing Beta Features releases to production
    • Steven spent a good 40+ hours during this sprint on Typography Refresh
    • Solution is to say no more!

April 4, 2014

[edit]
Things to start doing
  • We should more fully decompose cards at the time of sprint planning. In plain English: that means breaking stuff down in to smaller tasks. We could do a better job of this during the sprint planning. A concrete example of this is the non-linear tours card, which really should have been at least six cards instead of one or two. This also applies to research cards, which right now tend to represent a whole cycle of measurement, preliminary reporting, review, and final reporting.
  • Split the "Doing" list in to "In Development" and "Needs Review/QA". We agreed that cards tend to sit in the Doing column a while. This isn't because they're blocked, but because "doing" really means either "I am working on this" or "I am waiting on someone else to look at/code review/QA/verify requirements" on this. Splitting these two up should help us differentiate between these two steps better. I've completed this already in Trello.
  • Track spikes and apply a label to them. We aren't really tracking these as work items right now, even though we do it fairly often.
Things to stop doing
  • Remove the "In Code Review" label and replace with the new Spike label. This works since we have a new Review column.
  • Stop adding cards mid-sprint unless you consult with the team. If you want to add a card to the Current To Do list, then email the team mailing list to confirm we think it's an immediate blocker/priority that we overlooked during sprint planning.

March 26, 2014

[edit]
To-do items we agreed on then
  • We have developed a tiny backlog in code review. Let's be diligent about bringing up blockers like this during standup.
  • We want to do retrospectives more frequently. Steven to set up a meeting for 30 minutes maximum, after every sprint.
  • We need to more clearly define sprint theme. That is answering the question: what value do we want deliver to users?
  • Steven will set up time to talk about product strategy for anonymous editor acquisition. Aaron and maybe others not yet feeling like they have sufficient visibility in to the reasoning behind the theory of impact and design decisions on this.
  • Steven work with the team and the product group on a quarterly Growth roadmap, with direction from Howie. Will send this out to the team mailing list.
  • We need to get back in the habit of doing a quarterly planning discussion before each quarterly review. The next one is Wednesday, May 21st, so we need to do another of these before that. Look for a calendar invite.
Other items discussed but still unclear
  • Should we have a funky naming scheme for sprints? Mobile uses animal names. Flow uses . I proposed we use names of film stars (Alec Baldwin, Charlie Chaplin, Natalie Portman). Suggestions?
Retrieved from "https://www.mediawiki.org/w/index.php?title=Growth/Growth_2014/Retrospectives&oldid=4775146"