Reading/Web/Projects/A frontend powered by Parsoid/Q2 MVP

Here is the outcome of the spike https://phabricator.wikimedia.org/T113769 for the Q2 MVP that we're building for a mobile web front-end that uses parsoid.

Please comment on the talk page or the phabricator task for discussion of it.

Main objectives of the MVP

Ease of development and prototyping
Speed
Improved reading experience

Ease of development and prototyping

In order to ease development (of UI/UX) and the creation of different prototypes for features or user research, it seems like the following properties are desirable (which we don't have right now):

Immediate feedback when developing UIs.
Improve iteration time.
Sharing of improved or experimental work.
Make state of UI reproducible to improve bug fixing and UI testing.

Solutions will come in the form of:

Live updating UI to multiple clients on development without blowing current UI state.
Live updating "development cards" containing the isolated UI pieces with their different states. (Like visual feature/UI tests).
Fast static server serving the front-end and client cache layer.
Serializable app state enabling playing and re-playing of user sessions/state.

In order to enable better prototyping and rapid iteration, the following properties would be desirable (which we don't have right now):

Ease of development previously outlined would help with this.
Ease of deploying+sharing prototypes|in-progress|experimental work.
- For better/faster feedback from stakeholders.
- For better user research.
Checking prototypes|in-progress|experimental work with REAL data.
- For better/faster feedback from stakeholders.
- For better user research.

Solutions will come in the form of:

Previously outlined solutions on the "Ease of development and prototyping".
API driven means REAL data available when needed.
Static bundle means deployable anywhere really easily. No need for a server in experimental / prototyping stages.

Speed

We're looking to prove that we can make the mobile web (and eventually all web experiences) dramatically faster. For mobile web there's a clear objective:

Speed up Barack Obama's first render by 3x or 4x for a 2G connection on India.
- Which is more generally expressed as "Serve something useful RIGHT NOW and enhance the experience progressively as the network allows.
As a secondary objective, interacting with the page's chrome and navigation should feel instant once the page is loaded, improving perceived speed by always showing the user what is happening and minimizing network payloads by changing to API only requests.

To accomplish such dramatic speed ups is why we'll decouple most UX with the essential page payload.

To learn more check the Architecture section.

Improved reading experience

By accomplishing the previous two objectives and making use of the improved capabilities we hope we can provide a better and faster reading experience by experimenting with new interfaces, polishing the existing ones and quickly iterating through it. Not much more to explain here.

Scope

For this quarter we would like to focus on the reading experience, compared to the reading experience on the current mobile web. This is the list of features that the MVP will/could have:

Must have

Application chrome
- Menu
  - Home
Search
- Search bar on main page and on the menu bar for easy access from everywhere else
- Searching and navigation to article
Reading
- Super fast initial version and load of the article
- Expand to full article for reading
- Image gallery from images in the article
- Viewing references
- Language selector in article
- Viewing a talk page

Nice to have

Reading
- Viewing article categories
- Related articles
Main page
- Feed of suggested articles based on previous readed ones
Menu
- Settings
  - Disable images
Nearby
- List view
- Map view?
History of articles
Collections + Watchlist

Architecture

The idea is to make the site a really lean experience, to serve our users better in mobile devices (crappy connectivity), and then promote seamlessly to a web-app experience, that will allow us to provide a better web experience on bad-connectivity situations by aggressively leveraging caching on a thick client in the web browser.

Aggregated services will provide necessary data with minimal number of requests (for both server rendering and API responses).
- See red box '''API''' in diagram.
- Consumed services may be either mw.php api or restbase services.
- Aggregation should probably happen on restbase services to leverage the infrastructure.
Server will provide a minimal HTML with bare chrome and critical CSS, with a minimal version of the page's content (probably the stripped version of the lead section and table of contents).
- The same code/UI that powers and renders the web app will provide the server rendering.
  - Alternatively the server could output parsoid html (parsed or not) and/or JS data structures on the html page (if the server wouldn't able to run JS).
When the JS bundle loads the client side will enhance the chrome of the web to become a web app, avoiding further full roundtrips to the server and making the navigation happen in the client, and the access to the data via API or client side caching.
Aggressive client side caching will be implemented to minimize network roundtrips (since that's the worst case for the use case):
- Client DB for storing fetched data. (IndexedDB|WebSql|localStorage)
- Browser cache (one of these):
  - Browser Service worker for caching HMTL pages, and JS/CSS/FONTS.
  - Browser AppCache for JS/CSS/FONTS and main HTML.

Notes

The Web UI will remain standalone at all times. Serving the Web UI configured with API endpoints URLs should always work to improve the development, experimentation and prototyping experience.
Regarding server aggregation and data massaging on the server vs the client:
- Aggregating and massaging the data on the server will allow us to cache the complete aggregation and minimize both the amount of code that gets shipped to the client and the amount of network requests we perform in hostile network conditions.
- In certain cases, with secondary UI concerns or prototypes/experiments, we may send different requests with the objective of getting the article as soon as possible, which would be the most important content to get (and wouldn't want other concerns to slow it down). For example an article's related articles as an experiment or an stable feature may be decided that it's better requested on its own.