Jump to content

Reading/Web/Projects/Invest in the MobileFrontend & MinervaNeue frontend architecture/Progress

From mediawiki.org

The purpose of this page is to measure progress of the Invest in the MobileFrontend & MinervaNeue frontend architecture project.

Goal: Automate asset bundling (completed) Yes Done

[edit]

Progress can be measured by looking at the amount of files and lines of code migrated to the src/ folder.

A table showing progress in asset migration inside the MobileFrontend repository
Date Number of files in src/ [1] Number of files in resources/ Number of files in tests/qunit [2] Number of files in tests/node-qunit [3] Number of ResourceLoader modules Reference point
All [4] JS[5] *[6] JS [7]
08/21/18 0 197 86 44 0 57 34 d518689
08/29/18 3 200 43 5
09/12/18 7 198 42 7
09/19/18 8 198 83 42 7 57 9fb7a95
09/24/18 12 194 79 40 9 57 5fca35e
10/01/18 12 194 79 40 12 57 4165229
10/10/18 19 186 71 38 14 56 95b413d
10/19/18 28 182 65 36 17 55 e5f8b03
10/22/18 32 177 61 36 19 55 32 6599140
11/06/18 32 177 60 30 22 54 31 5217102
11/13/18 32 177 60 30 27 53 31 1a632c
11/16/18 36 174 57 25 30 52 30 255209
11/20/18 36 174 57 25 31 52 30 72a27c
11/29/18 39 172 55 23 35 51 29 420040
12/11/18 40 171 54 23 37 50 30 2cf551b
12/17/18 45 166 49 18 40 46 28 4dbf7ea
01/07/19 49 164 49 16 42 46 28 1dd04ebdf
01/10/19 58 166 46 13 45 44 26 c587c43d
01/15/19 67 167 42 11 47 37 21 3e4e2cd
01/17/19 83 159 31 4 55 33 18 4b6b3ba5
01/23/19 90 153 27 3 56 33 18 f4c2f4f1
02/05/19 (victory) 105 150 19[8] 0 59 29 14 28f31914

Outcomes

[edit]

At the start of the project, MobileFrontend had 86 JavaScript files. This was reduced to 19 files built via Webpack from 101 source files (files inside src/ folder). The increase in source files reflects the teams ability to embrace Webpack and separate code responsibilities more effortlessly.

44 QUnit tests were updated so that the could be run from the command line. This allowed us for the first time to see code coverage of the entire MobileFrontend extension. 15 new files were added thanks to refactorings and improvements to test coverage.

At the start of the project (commit d518689) there were 57 ResourceLoader modules insiding MobileFrontend. 24 [9] ResourceLoader modules were removed in the project leaving 34. This is important as it reduces overhead in the startup module. Similar changes are planned in Minerva and various other opportunities exist in MobileFrontend.

Goal: Speed up unit test execution and install code coverage Yes Done

[edit]

Progress in speeding up unit testing can be measured by looking at the amount of files migrated to the src/ folder, like in the previous sub-project (automatic bundling), and the amount of test files moved or left from tests/qunit to tests/node-qunit.

Currently, we have low test coverage and are unable to accurately measure test coverage. Test coverage is measured using nyc as files are migrated to node-qunit and we aim for ~75% coverage.

The following table details progress in this area. Bold highlights total coverage (100%).

File Code coverage at start Current code coverage
Reference point % Stmts % Branch % Funcs % Line Reference point % Stmts % Branch % Funcs % Line
mobile.categories.overlay
CategoryGateway.js ea25835 88.89 50 100 88.89
CategoryAddOverlay.js ea25835 0 0 0 0
CategoryLookupInputWidget.js ea25835 0 0 0 0
CategoryTabs.js ea25835 0 0 0 0
categoryOverlay.js ea25835 0 0 0 0
mobile.categories.overlays.js ea25835 0 0 0 0
mobile.editor.overlay
AbuseFilterOverlay.js ea25835 66.67 100 0 66.67
AbuseFilterPanel.js ea25835 47.37 0 25 47.37
BlockMessage.js ea25835 57.14 0 25 57.14
EditorGateway.js ea25835 91.67 87.5 88.24 91.67
EditorOverlay.js ea25835 51.27 38.57 47.06 51.27
EditorOverlayBase.js ea25835 37.21 7.41 15.38 37.21
VisualEditorOverlay.js ea25835 15.91 0 0 15.91
parseSaveError.js ea25835 95.45 91.3 100 95.45
schemaEditAttemptStep.js ea25835 0 0 0 0
mobile.editor.ve
ve.init.mw.MobileFrontendArticleTarget.js ea25835 0 0 0 0
mobile.init
BetaOptinPanel.js ea25835 0 0 0 0
editor.js    ea25835 0 0 0 0
mobile.init.js ea25835 0 0 0 0
mobile.languages.structured
LanguageOverlay.js 1dd04ebdf 74.42 55.56 50 74.42 1dd04ebdf 74.42 55.56 50 74.42
rtlLanguages.js 1dd04ebdf 100 100 100 100 1dd04ebdf 100 100 100 100
util.js 1dd04ebdf 88.71 68.42 84.62 88.71 1dd04ebdf 88.71 68.42 84.62 88.71
mobile.mediaViewer
ImageGateway.js c587c43d 95.24 78.57 100 95.24
ImageOverlay.js c587c43d 77.66 41.67 72.22 77.66
LoadErrorMessage.js c587c43d 75 50 66.67 75
mobile.notifications.overlay
NotificationsFilterOverlay.js c587c43d 0 0 0 0
NotificationsOverlay.js c587c43d 0 0 0 0
mobile.special.mobilediff.scripts.js
mobile.special.mobilediff.scripts.js c587c43d 0 0 0 0
mobile.special.mobileoptions.scripts.js
mobile.special.mobileoptions.scripts.js c587c43d 0 0 0 0
mobile.special.nearby.scripts
Nearby.js 4b6b3ba 86.67 71.43 91.67 86.67
NearbyGateway.js 4b6b3ba 94.44 80 90 94.44
mobile.special.uploads.scripts
PhotoList 4b6b3ba 0 0 0 0
PhotoItem 4b6b3ba 0 0 0 0
PhotoListGateway.js 4b6b3ba 17.4 0 11.11 17.14
mobile.special.userlogin.scripts
mobile.special.userlogin.scripts.js c587c43d 0 0 0 0
mobile.special.watchlist.scripts
WatchList.js 3e4e2cd 74.19 50 44.44 74.19
WatchListGateway.js 3e4e2cd 96.77 87.5 100 96.3
mobile.startup
Anchor.js 0c3ecb33 [10] 100 100 100 100 1dd04ebdf 100 100 100 100
Browser.js ab9af61 59.52 46.67 38.46 59.52 1dd04ebdf 90.7 83.33 92.31 90.7
Button.js 5217102 [10] 57.14 0 0 0 1dd04ebdf 100 100 100 100
CtaDrawer.js 6599140[11] 0 0 0 0 1dd04ebdf 100 100 100 100
Drawer.js 6599140[11] 0 0 0 0 1dd04ebdf 100 100 100 100
Icon.js 0c3ecb33 [10] 58.33 33.33 100 58.33 1dd04ebdf 70.83 50 100 70.83
LoadingOverlay.js 0c3ecb33 [10] 100 100 100 100 1dd04ebdf 100 100 100 100
Overlay.js acfff63e5[12] 50 32.35 25 50 1dd04ebdf 50 32.35 25 50
OverlayManager.js 729a04c 98.63 97.22 100 98.63 1dd04ebdf 98.72 97.37 100 98.72
Page.js d065aee 83.65 86.57 56.52 83.65 1dd04ebdf 86.46 91.3 65.22 86.46
PageGateway.js 6599140 90 73.17 100 89.87 1dd04ebdf 90 73.17 100 89.87
PageList.js 5217102 [10] 100 50 100 100 1dd04ebdf 100 50 100 100
Panel.js 78db74e 0 0 0 0 1dd04ebdf 100 75[13] 100 100
Section.js d065aee 31.25 0 0 0 1dd04ebdf 100 100 100 100
Skin.js a959d2b 58.59  46.81 52.38 58.59 1dd04ebdf 59.23 46.81 52.38 59.23
Thumbnail.js d065aee 87.5 100 75 87.5 1dd04ebdf 88.89 100 75 88.89
Toggler 4dbf7ea 90.6 70.77 85.71 90.6 1dd04ebdf 90.6 70.77 85.71 90.6
View.js 104322e 91.53 76.92 86.67 91.53 1dd04ebdf 92.65 82.76 87.5 92.65
cache.js 9b45e1a 0 0 0 0 1dd04ebdf 100 100 66.67[14] 100
context.js 78db74e 0 0 0 0 1dd04ebdf 100 100 100 100
extendSearchParams.js   5217102 100 83.33 100 100 1dd04ebdf 100 83.33 100 100
icons.js 6599140 0 0 0 0 1dd04ebdf 100 100 100 100
mfExtend.js 9b45e1a 0 0 0 0 1dd04ebdf 100 100 100 100
moduleLoader.js 78db74e 72 50 66.67 72 1dd04ebdf 69.57 50 66.67 69.57
rlModuleLoader.js a959d2b 0 0 0 0 1dd04ebdf 100 100 100 100
time.js 5fca35e 24.24 10 16.67 24.24 1dd04ebdf 24.24 10 16.67 24.24
toast.js 0c3ecb33 [10] 52 25 50 52 1dd04ebdf 52 25 50 52
util.js ab9af61 18.18 0 7.14 19.05 1dd04ebdf 100 100 100 100
mobile.startup/lazyImages
lazyImageLoader.js N/A N/A N/A N/A N/A 4b6b3ba 40 100 40 40
mobile.startup/references
ReferencesDrawer.js bd03d2d 90 75 88.89 90 bd03d2d 90 75 88.89 90
ReferencesGateway.js ce9ef65 100 100 100 100 ce9ef65 100 100 100 100
ReferencesHtmlScraperGateway.js ce9ef65 100 100 100 100 ce9ef65 100 100 100 100
ReferencesMobileViewGateway.js ce9ef65 78.95 50 54.55 78.95 ce9ef65 78.95 50 54.55 78.95
mobile.startup/search
MobileWebSearchLogger.js 4dbf7ea 75 100 87.5 75
SearchGateway.js 4dbf7ea 91.3 58.33 84.62 91.3
mobile.startup/watchstar
Watchstar.js af81aa1 91.84 91.67 75 91.84 af81aa1 91.84 91.67 75 91.84
WatchstarGateway.js af81aa1 100 100 100 100 af81aa1 100 100 100 100
WatchstarPageList.js af81aa1 95.83 75 100 97.87 af81aa1 95.83 75 100 97.87
mobile.talk.overlays
TalkBoard.js 4b6b3ba 92.5 60 100 92.5
TalkSectionAddOverlay.js 4b6b3ba 44.93 0 36.36 44.93
TalkSectionOverlay.js 4b6b3ba 72.55 50 70 72.55
autosign.js 4b6b3ba 100 100 100 100 4b6b3ba 100 100 100 100
talkOverlay 4b6b3ba 0 0 0 0

Outcomes

[edit]

During the webpack porting project, we increased code coverage in a few mobile.startup modules. Despite this small effort, upon completing the Webpack migration code coverage was around the 53% mark, specifically:

  • 53.13% of our statements were covered
  • 46.84% of all branches were covered
  • 53.67% of all functions were covered
  • 53.13% of all lines were covered

Given code coverage was purposely increased slightly during this migration, the initial code coverage was likely much less, if we ignore file size and take the average % of all the columns in the table above, the average file featured code coverage with the following characteristics:

  • 50.38% of all our statements were covered
  • 39.2% of all branches were covered
  • 45.2% of all functions were covered
  • 49.4% of all our lines were covered

Most alarmingly of all, 45 of the 81 files had 0% coverage.

By the end of January 2019, we were able to enforce coverage in the repository. meaning that code coverage could only get better.

Goal: Increasing code coverage In progress In progress

[edit]

After speeding up unit test execution and adding test coverage monitoring, it became a lot easier to track progress in this area. Given our goal is code coverage, progress can be monitored as we work towards 100%.

How code coverage changed over time in MobileFrontend
Date Statements Branches Functions Lines Commit
02/05 53.13


46.84 53.67 53.13 28f31914
02/25 54.63 47.73 53.7 54.63 d40287e9
03/06 55.19 48.43 54.7 55.19 52ca30bf
03/27 56.7 49.47 56.4 56.7 385ec4b8


Goal: Review and refactor Minerva components In progress In progress

[edit]

Refactoring responsibilities / separating concerns

[edit]

The number of files in src is expected to grow while the number of template files in resources is expected to decline as we make components more reusable, thus reducing the need for templates and CSS files in the latter and separate concerns in the former.

Date Number of files in src/ [1] Number of template files [15] Number of LESS files in resources/[16] Number of files in resources/ [4] Commit
02/05 105 42 36 150 28f31914
02/25 109 41 37 150 d40287e9
03/06 109 41 39 152 52ca30bf
03/27 113 40 43 155 385ec4b8

Removing inheritance

[edit]

When both columns are equal numbers the goal of removing inheritance can be considered achieved. We hope the number of classes extending View will go up as we move away from large multi-concerned components to dumb components.

Use of inheritance in MobileFrontend
Date Number of classes extending View[17] extending anything [18] Commit
02/05 21 44 28f31914
02/25 21 43 d40287e9
03/06 21 43 52ca30bf
03/27 24 43 385ec4b8

All: Impact on bytes shipped

[edit]

It's hard to measure this as during the MobileFrontend refactor project several new projects were simultaneously worked on and shipped - including Page Issues.

At the start of the project, in August 1st we shipped 175kb of JavaScript.

https://grafana.wikimedia.org/dashboard/db/mobile-2g?panelId=69&fullscreen&edit&orgId=1&from=now-1y&to=now&tab=time%20range [Broken Link T211982]

On 23rd January 2019 as we begin wrapping up the work we find JS for the same page to be 170.3kb. Of course during this time, features have been added and code has changed outside MobileFrontend so it's difficult to say how much of this is due to our work.

All: Impact on errors

[edit]

See Reading/Web/Notable incidents for errors that occurred during the lifespan of the project.

Most errors were unrelated to the webpack work but caught because of it. This metric will become more important as we switch our focus to code refactoring and improving unit test coverage.

References

[edit]
  1. ↑ 1.0 1.1 Measured via
    find src -type f | wc -l
  2. ↑ Measured via
    find tests/qunit -type f | wc -l
  3. ↑ Measured via
    find tests/node-qunit -type f | wc -l
  4. ↑ 4.0 4.1 Measured via
    find resources -type f | wc -l
  5. ↑ 
    find resources -type f -iregex '.*js' | wc -l
  6. ↑ Measured via
    find resources -type d -not -path "resources" -not -path "resources/dist" -not -path "resources/*/*" | wc -l
  7. ↑ 
    find resources -type d -not -path "resources/*images*" -not -path "resources/*styles" -not -path "resources/dist" -not -path "resources" -not -path "resources/*icons" | wc -l
  8. ↑ All 19 files are files in resources/dist representing the 19 entry points now added to MobileFrontend.
  9. ↑ Comparing the number of RL modules before and after suggests 28 modules were removed (57 - 29), however this doesn't account for the 4 modules which do not have folders in resources (and only have a single output in resources/dist - these are mobile.special.mobileoptions.scripts, mobile.special.mobilediff.scripts, mobile.special.watchlist.scripts and mobile.special.userlogin.scripts
  10. ↑ 10.0 10.1 10.2 10.3 10.4 10.5 Some code coverage existed thanks to other components making use of the component.
  11. ↑ 11.0 11.1 Initial code coverage maybe inaccurate - might improve when References code is ported to webpack
  12. ↑ Initial code coverage may be inaccurate. Might improve when Talk feature is ported to webpack.
  13. ↑ https://phabricator.wikimedia.org/T209129
  14. ↑ Relates to NoCache noop function, which doesn't make sense to test.
  15. ↑ find resources -type f -iregex '.*hogan' | wc -l
  16. ↑ find resources -type f -iregex '.*less' | wc -l
  17. ↑ 
    ag "mfExtend\(.*\, View\," src/ -l | wc -l
  18. ↑ 
    ag "mfExtend\( .*\,.*," src/ -l | wc -l
Retrieved from "https://www.mediawiki.org/w/index.php?title=Reading/Web/Projects/Invest_in_the_MobileFrontend_%26_MinervaNeue_frontend_architecture/Progress&oldid=5075145"