Wikimedia Technical Conference/2018/LessonsLearned
The lessons learned from the Wikimedia Technical Conference 2018
Suggested changes for next time
- Session leaders have a minimum of 1 month to prepare their sessions
- Sessions identify pre-event readings / thought work and attendees do it. Program Committee to identify the priority pre-event preparations for attendees. / Create a concise overview of the previous efforts in the area of the conference. "everything you need to know about xyz"
- Identify clear ways for management and c-levels to be visibly/actually engaged / feel comfortable engaging
- Solicit feedback about specific sessions from attendees in advance. Maybe another survey with option questions directly from the lesson leaders.
- Be very clear with the session leaders about what is expected of them
- Pre-event calls for participation to non-attendees
The best
- The emphasis and clear effort by the group to determine clear goals, questions, actions, and decisions.
- The facilitation & participation (vs a presentation/discussion style) oriented sessions
- The flow of the event / the event's progression
- Productive / positive conversations
The worst
- The Portland location was a further away / harder travel location
- Session leaders did not get enough warning or time to prepare / thus sessions were not prepared adequately
- Everything around session formate and preparation was too late
- WMF upper management not participating / clearly engaged
Feedback Survey Background
[edit]- rfarrand created the feedback survey with using google forms based on feedback survey questions from past events and input from the program committee .
- The Feedback Survey was sent out to participants on December 10, 2018
- The survey received 22 responses (33% response rate)
Considerations for next year
[edit]This section is based on the fill-in-the-blank/comment sections of the feedback form. Some of these comments will be contradictory. We are mostly trying to include common themes and issues that were felt by groups of people. This section is a bit more subjective than the #Data section below and suggestions may be paraphrased and combined with each other or just pasted as-is.
What did you think the purpose of the event was in your own words?
[edit](copy paste)
To discuss big-picture topics around the future of our various technical platforms, and everything that interrelates to that.
To create a shared understanding of the main challenges Wikimedia's technical systems will need to overcome in the next few years, and to begin prioritizing them and planning for how to solve them.
To explore and partially map out specific goals, questions, and actions stemming out from the strategic goals for MediaWiki.
The purpose was to align developments on a technical level. There are different decisions to be made making the different WMF ecosystems future proof and the conference acted as a moment to move forward
To discuss and agree on the mid-long term roadmap of mediawiki between different stackholders (WMF/WMDE/3rd parties/volunteers)
Feedback to shape the platform evolution strategy
Make some decisions on Platform Evolution directions
To discuss and set the tone for how the core platform will evolve over the next years. It was about the broad strokes and not so much the details.
Helping Cory and his teams work on planning for the Platform Evolution CDP
To identify the most important technical problems we need to, the overall direction of our technology in the areas affected by those problems, and for a narrow subset of those problems to reach consensus on a specific solution.
To have a face to face discussion on various tech and product ideas that are critical for the future of MediaWiki and WMF. Come up with clear decisions to take them forward in our roadmap
Figure out the technical direction for Wikimedia in various defined areas.
Talk about the limitations we have, where can we go from here and how can we shape the future of our product and tech
Discuss and make decisions on topics related to the PE CDP
Discussing and finding agreements on the overall, high-level technical direction the MediaWiki development should go into. This includes decisions and discussions about technical areas MediaWiki should involve into as well as what high-level features should be done and what technical problems this might incur.
discussing high level technical architecture and strategy
Enable the technology team to talk about important upcoming work and get a better understanding of the product requirements and ideas.
Bringing together relevant people to discuss the course MediaWiki should take in the future also by reaching out to ideas and feedback from people outside WMF.
The official purpose was to plan the high-level 3-5-year strategy of the Wikimedia technology stack. Realistically, I think the purpose was to make a list of high-level current questions, discuss them and prioritize work for the next 1-2 years.
Gather feedback around technical realms in the Mediawiki/Wikimedia world. Exchange on problems and possible solutions in these realms with all stakeholders. Develop consolidated starting points for the platform evolution program.
To put together strategic documents that the C-levels at the Wikimedia Foundation understand as productive work.
The purpose appeared to be to drive focused discussions on key topics and recommend actionables to TechCom and the relevant engineering teams.
How could we have better prepared you to attend this event?
[edit]
- Pre event details / readings:
Slide decks shared
Notes to reach over while traveling
Suggested readings and videos
General disclosure around what would be discussed in advance
The order and structure of the event to be explained in advance
- Completely new people could be matched with someone who knows what they are doing and has the time to explain context
- These are community events, don't only focus on WMF projects and interest areas
- Have clear documentation on different participants and their roles and responsibilities.
- Being a session leader, I think I was pushed to be rather well prepared and I am pretty pleased about that.
- A small disconnect was this: I was leading a session, but the topic of the session was decided without me. Could not properly shape the topic since it was late.
- Some topics needs longer sessions
- Narrow the topic
- session leads were co-opted only days before the event, and did not have time to properly take ownership
- Announce sooner, select participants / sessions / session leaders sooner, make at least a vague guess of which people will be at which sessions well ahead of time, give way more time for people to participate in preparing the sessions on Phabricator, make active Phabricator participation a prerequisite for having that session.
- Hard to say, in opinion this event was a lot about getting started with these kind of discussions anyway. - For me as session leader I prepared myself by giving the questions of my session to a local audience already before hand, so I get a first feeling about opinions there. - So one idea could have been maybe, sending a sheet with one key question per session to every attendee, so everyone could have thought about things beforehand.
Based on what happened at the event, what (if anything) do you think will change?
[edit](copy paste)
Better coordination and communication between stakeholders in the future. A clearer shared understanding of the existing environment and our possible/desired futures.
As someone who doesn't work on 'Platform' full-time, it's hard for me to predict what will change, beyond hopefully improved collaboration and more 'pulling in the same direction' on the part of WMF engineers.
Ideally (since it was written into formal actions) a more formal interface will be created and maintained between the WMF and "3rd party people". This will bolster the improving relationship between the two communities and I think will help both parties.
higher priority / awarness for reviewing volunteers patches, decoupling mediawiki core classes. From Wikidata side, a plan to incremently add WD editing support from client wikis
Some ideas (Mediawiki-based Machine Translation) won't happen and some will (better APIs, more security)
As an outcome, MediaWiki distribution options will be modernized.
I 'd like to believe there is be some more clear rules about how and when we will proceed with introducing and deploy new services. I also think we will be able to change the way mediawiki purge events get processed.
The order of the event (identify problems, gather ideas, set direction, consensus on solutions) was really well done. Each of these four can and should happen back-to-back in my opinion and in that order. In past events we've tried to do all of these in one session for 1 topic and then again for another topic, which is more draining and less likely to succeed I believe. This year was great.
I am looking forward for seeing the decisions turning to actions. Something that I missed to see in past events.
The event seemed to have more actionable items result from it compared to previous events with a similar scope, so I hope we will get more done / change more as a result of this event.
I hope MediaWiki's software will be cut into smaller pieces, the framework part will be just a framework and the logic/product part will be a different tool built with said framework And also, new frontend tech...
I think the conference motivated and empowered people to take action in certain areas, which is refreshing to see.
The understanding of a good testable code and actual tested code will get better and will hopefully lead to a better code quality in the long term. Also I think that the understanding is better about what the user asks for is what should be implemented, instead of something which looks cooler to others, but has less value for the user.
communication between teams and departments will improve, platform evolution will go forward informed with input from many stakeholders.
I hope wmf product now has more appreciation for the technical challenges of our movement.
Well I hope so. I believe that reaching out to people outside WMF will strengthen MediaWiki. However, ideas and input should be considered even if there is no immediate benefit for WMF itself in the short run.
I think the event was a good place to speed up discussion of some high-level topics that already took place to some extent, such as the architecure principles, and to turn reasonably-well-understood problems into propspective roadmaps (such as the MediaWiki platform refactoring session). I don't think it hit the mark on the high strategic level decisionmaking that it was officially aimed at.
At first maybe not much yet, but hopes are high, that people in fitting positions can decide on next steps and that the outcome of the TechConf might be useful as input for followups.
Attendance will fall slightly. As another event fills the gap for volunteers and 3rd party developers, the Tech Conference will turn into the Tech managers and Product managers pre-annual plan planning meeting.
How could we have better engaged folks, that were not in attendance, both before and after the event?
[edit](copy paste, replaced some organization names with X for anonymity)
- More summarization emails. More summarization of notes - the skeleton tables of Q&As in each notes doc was a good idea, but didn't work out in most cases, due to time/expertise constraints - I.e. it takes at least an hour for a topic-expert to write a clear short summary.
- Hearing more from leadership on a) explicit goals before the event, and b) specific executive decisions made in response to the event, would defintely be the most impactful way to improve engagement, IMO. That would make the significance (or possibly lack of significance) of the event more transparent.
- I think the white-board idea of soliciting answers to a few questions each morning really helped to kick-start conversations in break-out sessions. I would have liked to open these up to everyone online a few days before the event. After the event is tough ... obviously the results are posted online, but I don't forsee getting much extra progress from the wild Internet.
- Personally I tried to continue some of the discussions started in X in the community wishlist. To engage folks that weren't in the event, it would be nice to have similar survey to community wishlist (or this survey, MW devs survey/3rd party survey) happen just before X, and then include some of the highest requests into the agenda
- Besides above-said skills/roles documentation, informing about how Platform Evolution can impact their work
- Public discussion on phabricator tasks, mailing lists, or wikis both before and after the event.
- Some simple data collection / thought collection, perhaps using simple surveys from session organizers as we know discussion on phab tickets prior to these events gets hard to follow.
- I don't know, the X part was wonderful in my opinion
- Perhaps the sessions could have been presented in advance to non-attendees (in a tech talk or a similar venue), which could have given them the opportunity to engage and give feedback before the event. Likewise, giving a resume/digest talk afterwards would allow people to be informed better about what happened there.
- Host moderated discussions beforehand about each "big" topic. Do the same afterwards.
- Well what I have already done in the meantime is giving a talk about the conference and the topics covered including the general scope as well as pointing to interesting and important topics. Moreover I talked or will talk about this conference with relevant and interested people who did not attend. This is an ongoing process for me and hopefully for the other attendees, too.
- Again, more time to prepare. The recommendations included that session leaders hold an IRC or Phab discussion before the event but no one did, probably because it did not realistically fit into the timeline.
- Before: As said above, having a pretty narrow and neat catalogue of key questions send around ( e.g. on Wikitech-L ). After: Hard to tell, the outcome is still a bit vague. But if there are actionable items that could use broader involvement it should be natural to spread this and one could refere to the TechConf then.
- Live streamed. Online participation. Have early voting on topics by a wider audience or something like that so people become invested in what gets discussed.
Given the need to limit the event's size, can you comment on how we might improve the participant selection process?
[edit](copy paste)
I'm sure this was tough. I wonder if you chose 56 other participants from roughly the same distribution of "roles" if you'd end up with the same results. It would be an interesting experiment.
Maybe invite one guest (representative) from each team/background - have them talk to others from the theme and bring feedback to the conference.
I think it highly depends on the theme of the event every year. The theme this time around was core platform and is pretty specific, so the limited size made sense. If less specific themes (say AI/Machine learning) are introduced in the next events it might make sense to include more people and reach out more to volunteers.
Narrow focus of the event, engage in more asynchronous discussions in months leading up to event, select participants for face-to-face meetings based on resolution of conflicting points of view in active discussions rather than who knows whom. This all presupposes that an event like this is meant to move forward stalled issues rather than to do something like new idea generation.
I like what was done now, define who are the stakeholders of the program, talk to people, invite relevant people
Since this is a WMF event, I think limiting the size is the problem to be solved, not how to select people
Communicating specific topics to be discussed and decisions to be made would allow people to have a better idea of whether they want to come, or who they want to ask to attend.
Hard to comment as it was not very transparent. I liked the DevSummit process of writing strategy proposals (but than that event failed to incorporate those in any form). Judging by the result, the selection of staff did not seem like a problem area (the low participant limit maybe more so). For non-Wikimedian-non-staff participants, the selection did seem somewhat arbitrary. E.g. systematically inviting non-rookie-level people from the largest third-party MediaWiki-based organizations would have been nice.
If we're going to turn this into a manager's meeting, then I think the size is fine and might even benefit from getting smaller.
What did you like most about the Wikimedia Technical Conference?
[edit]- Discussions with others
- Volunteers and their heavy participation
- Small group dinners
- Clear effort by the group to determine clear goals, questions, actions, and decisions.
- Learning other peoples concerns
- Good atmosphere
- The order and flow of the event
- The facilitation. The fact the sessions were organized in ways to optimize the participation (vs a presentation/discussion style) and output was great.
- Activity oriented sessions
- Collecting point of views about topics during the sessions and not making decisions on the spot! for that shows planning and a creation of a plan as opposed to hack things around.
- Longer event
- Good note taking
- We had important discussions with the right people in the room.
- That product and technology finally were forced to talk productively
- The no device rule
- The focus feeling of the event
- Hallway posters
- Organizing the conference topics into stages from strategy to product to architecture made the program much easier to navigate (although it also meant that often similar things happened in parallel and expert participation was split).
- It felt very productive. The feeling of "thing can get done" was really intense.
- Opportunity to highlight some long-term issues and set up a plan and responsibilities for addressing them
What did you like least about the Wikimedia Technical Conference?
[edit]- Long days
- C-levels not engaged
- Missing voices
- needed more understanding around if the decisions will be carried forward
- Multiple tracks
- Portland location (hard travel)
- Use less paper
- To many topics were covered, harder to come to decisions
- No unconference
- Not enough time to prepare
Anything else that needs to change to make a similar event more successful in the future?
[edit]- More preparation by session leaders / start earlier
- Glossary of terms for session, to be sure the audience is talking about the same thing, would be nice
- Make sure session leaders have full grasp of the topics
- One time slot per day for presentation sessions
- Better ways for people not attending to participate
- The program consisting of different stages (strategy, use cases, architecture) that build on each other was an interesting idea, but I don't think much building upon actually happened (nor was there much attempt to facilitate this - the documentation sessions seemed to be an attempt at that but I don't think that worked out).
Data
[edit]Your affiliation
[edit]| Volunteer | 5 | 22.7% |
|---|---|---|
| 3rd Party | 5 | 22.7% |
| WMDE | 4 | 18.2% |
| WMF | 11 | 50% |
How would you describe your overall experience at the Technical Conference?
[edit]| Very Positive | 11 | 50% |
|---|---|---|
| Positive | 11 | 50% |
| Neither negative or positive | 0 | |
| Negative | 0 | |
| very negative | 0 |
I learned something important at this event from a field that I am not an expert in...?
[edit]| Yes | 17 | 77.3% |
|---|---|---|
| No |
1 | 4.5% |
| I don't know / neither | 4 | 18.2% |
A long needed decision or agreement, that I (or others) have been struggling with, has been made because of the Wikimedia Technical Conference
[edit]
| Agree | 13 | 59.1% |
|---|---|---|
| Disagree | 0 | 0 |
| I don't know / neither | 9 | 40.9% |
Logistics
[edit]Amount of time for breaks?
[edit]| I want longer breaks | 2 | 9.1% |
|---|---|---|
| I want shorter breaks | 0 | 0 |
| The length of the breaks was OK | 20 | 90.9% |
| I don't have an oppnion | 0 | 0 |
Amount of schedule content time vs break and discussion time?
[edit]| I would have preferred more time for breaks / discussion | 3 | 13.6% |
|---|---|---|
| I would have preferred more time in scheduled sessions | 1 | 4.5% |
| The ratio of break to session time felt OK | 17 | 77.3% |
| I don't have an opinion | 1 | 4.5% |
Starting Time
[edit]| The event should start earlier | 1 | 4.5% |
|---|---|---|
| The event should start later | 2 | 9.1 |
| The event should start at the same time but the program should start earlier | 1 | 4.5% |
| The timing was OK | 15 | 68.2% |
| I don't have an opinion | 3 | 13.6% |
Ending Time
[edit]| The event should end earlier | 3 | 13.6% |
|---|---|---|
| The event should end later | 3 | 13.6% |
| The ending time was OK | 16 | 72.7% |
| I don't have an opinion | 0 | 0 |
Event size and productivity — what are your thoughts?
[edit]| 56 Participants felt about right | 15 | 68.2% |
|---|---|---|
| The event should be larger, more voices were needed to have a successful conversation | 4 | 18.2 |
| The event should be smaller as I was not able to interact with everyone and/or not everyone had the chance to speak up | 0 | 0 |
| I don't have an opinion | 3 | 13.6 |
Please tell us how much you agree or disagree with the following statements.
[edit]Attending this event was worth my time
[edit]| Strongly Agree | 14 |
|---|---|
| Agree | 8 |
| Neither agree or disagree | 0 |
| Disagree | 0 |
| Strongly disagree | 0 |
The opportunity to meet with follow developers and Wikimedians was valuable to me
[edit]| Strongly Agree | 18 |
|---|---|
| Agree | 1 |
| Neither agree or disagree | 3 |
| Disagree | 0 |
| Strongly disagree | 0 |
I would like to attend this event again next year
[edit]| Strongly Agree | 15 |
|---|---|
| Agree | 6 |
| Neither agree or disagree | 1 |
| Disagree | 0 |
| Strongly disagree | 0 |