Developer Satisfaction Survey/2023/Beta
📖 Developer Satisfaction 2023
The developer satisfaction survey is an annual survey of the Wikimedia developer community.
🧪 Beta cluster
[edit]88% of all comments we coded as negative mentioned at least 1 of 3 things (in order of decreasing frequency):
- Production-likeness
- Ownership/Sustainability
- Reliability
🔗 Beta cluster's use over the past year
We asked survey takers, “In the past year, have you used Beta Cluster?”
- The majority (59%) of respondents said they had used Beta Cluster
- 36% said they had not used Beta Cluster
- 5% were unsure
🔗 Beta cluster satisfaction
💬 We rely on beta to catch issues before they go live to production but we can't in turn often rely on beta!
We asked survey takers, “How satisfied are you with Beta Cluster?”
- 38% said they were satisfied with Beta Cluster
- 33% said they were dissatisfied
- 25% said they were neither satisfied nor dissatisfied
- 4% were unsure
🔗 Beta cluster satisfaction (by role)
- Test engineers had a majority of respondents indicate they were satisfied with Beta Cluster
- All but two role groups had more respondents indicate satisfaction than dissatisfaction
- Tool developers and SRE Infrastructure developers had more respondents indicate disatisfaction than satisfaction
🔗 Beta cluster features ranked by priority
💬 the beta cluster as it currently exists is trying to be too many things at once.
We asked respondents, “How would you rank the following use cases for Beta Cluster? Rank the following from 1 (most important) to 5 (least important).”
Respondents were asked to order each of the following use cases by priority:
- Exploratory testing/QA of recent patches
- Demoing/showcasing
- Test automation of end-to-end tests
- Test automation of integration tests
- Test automation of unit tests
- Other
These were presented in a random order to each respondent.
The chart below shows descending order of average rank. "Exploratory testing" has the highest average rank and received the most #1 rankings. The use cases with the average lowest rankings were "test automation of integration test" and "test automation of unit tests". The graph does not capture "other".
There was no clear consensus on "Other" priorities. People listed a wide range of priorities such as:
- developing infrastructure changes
- developing userscripts/gadgets
- understanding what's going out with the weekly train deployments
💬 What needs to be done is identify 1 or 2 useful use cases, implement them in some other way and then burn the thing to the ground.
🔗 Activities on the beta cluster
We asked respondents, “Do you engage in any of the following activities on Beta Cluster?”
🔗 Activities on the beta cluster (by role)
