Jump to content

Architecture meetings/RFC review 2014-02-21

From mediawiki.org

17:00 UTC, February 21, at #wikimedia-office connect.

Requests for Comment to review

[edit]

Three at most, so we can go in depth.

  1. HTML templating, in particular Requests for comment/HTML templating library/KnockoutProposal
  2. Packaging (debs for now) and distribution as discussed in Requests for comment/Services and narrow interfaces
    • Long-term repo
    • Automated upload mechanism for normal devs
    • Possible use for service deploys
  3. your RFC here

Quick checkins:

  1. Architecture guidelines (quick checkin)
  2. Performance guidelines (quick checkin)

Summary and logs

[edit]

Meeting summary

[edit]

Meeting started by sumanah at 16:58:30 UTC. The full logs are available at https://tools.wmflabs.org/meetbot/wikimedia-office/2014/wikimedia-office.2014-02-21-16.58.log.html .


Meeting summary

[edit]


Meeting ended at 17:59:53 UTC.


Action items

[edit]
  • mwalker <mwalker> sumanah, my action is to port the JS IR compiler / runtime to PHP
  • mwalker Answer Nikerabbit's questions about RL integration
  • seeing this in action in a specific use case
  • gwicke or mwalker to demonstrate this in action in a specific use case ("this" being mwalker's port?)
  • sumanah to work on topic including "Channel is logged and publicly posted (DO NOT REMOVE THIS NOTE). https://meta.wikimedia.org/wiki/IRC_office_hours" - talk with Tim L
  • notes to talk page...


Action items, by person

[edit]
  • gwicke
    • gwicke or mwalker to demonstrate this in action in a specific use case ("this" being mwalker's port?)
  • mwalker
    • mwalker <mwalker> sumanah, my action is to port the JS IR compiler / runtime to PHP
    • mwalker Answer Nikerabbit's questions about RL integration
    • gwicke or mwalker to demonstrate this in action in a specific use case ("this" being mwalker's port?)
  • Nikerabbit
    • mwalker Answer Nikerabbit's questions about RL integration
  • sumanah
    • mwalker <mwalker> sumanah, my action is to port the JS IR compiler / runtime to PHP
    • sumanah to work on topic including "Channel is logged and publicly posted (DO NOT REMOVE THIS NOTE). https://meta.wikimedia.org/wiki/IRC_office_hours" - talk with Tim L
  • **UNASSIGNED**
    • seeing this in action in a specific use case
    • notes to talk page...

Full log

[edit]

See in HTML or see below.

Meeting logs
16:58:30 <sumanah> #startmeeting RFC review feb 21 on HTML templating and Services & narrow interfaces
16:58:30 <wm-labs-meetbot> Meeting started Fri Feb 21 16:58:30 2014 UTC and is due to finish in 60 minutes.  The chair is sumanah. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:58:30 <wm-labs-meetbot> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:58:30 <wm-labs-meetbot> The meeting name has been set to 'rfc_review_feb_21_on_html_templating_and_services___narrow_interfaces'
16:58:36 <sumanah> #info <gwicke> HTML templating summary is at https://www.mediawiki.org/wiki/Requests_for_comment/HTML_templating_library/KnockoutProposal
16:59:31 <sumanah> (waiting 30 seconds for more people to show)
16:59:41 * aude waves
17:00:34 <sumanah> hi there!
17:00:38 <sumanah> ok let's start
17:00:42 <marcoil> hi Sumana!
17:00:44 <sumanah> #info https://www.mediawiki.org/wiki/Architecture_meetings/RFC_review_2014-02-21
17:01:19 * sumanah invites Brion into the room
17:01:20 <MaxSem> I poked Brion
17:01:24 <sumanah> :)
17:01:34 <brion> rfc time? yeahhhhh
17:01:36 <sumanah> Hi brion!
17:01:45 <sumanah> so today we have  https://www.mediawiki.org/wiki/Architecture_meetings/RFC_review_2014-02-21
17:01:47 <milimetric> I'm fly-on-the-walling this one
17:01:51 <sumanah> HTML templating and Services & narrow interfaces
17:02:13 <sumanah> #info also today we'll have very quick checkins on performance guidelines and architecture guidelines
17:02:27 <Nikerabbit> evening
17:02:35 <sumanah> gwicke: which do you want to start with? HTML templating or interfaces/services?
17:02:39 <gwicke> welcome all
17:02:45 <gwicke> HTML templating
17:02:49 <sumanah> #topic HTML templating
17:03:07 <gwicke> clarification re SOA: this is really about packaging as mentioned in the SOA RFC
17:03:29 <gwicke> so repository setup, upload procedures, use of debs for service deployments
17:03:33 <brion> excellent
17:03:43 <gwicke> I have asked Faidon to attend if possible, hope he pops in later
17:04:02 <gwicke> so lets start with HTML templating
17:04:02 <sumanah> hi paravoid!
17:04:05 <paravoid> hello!
17:04:20 <sumanah> paravoid: http://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-office/20140221.txt for logs till now :)
17:04:30 <paravoid> thank you sumanah
17:04:39 <gwicke> the basics are discussed in https://www.mediawiki.org/wiki/Requests_for_comment/HTML_templating_library/KnockoutProposal
17:05:08 <gwicke> after the architecture summit the question was whether we can have a secure yet performant solution
17:05:30 <gwicke> we prototyped things and can now answer that question with 'yes'
17:05:33 <MaxSem> I can read Twig/Mustache/similar without knowing the language, but not this proposal's markup
17:05:54 <Nikerabbit> I've left my thoughts on the talk page of the new proposal
17:06:20 <gwicke> so, syntax; we looked at several DOM-based syntaxes out there
17:06:59 <gwicke> most are attribute-based, as that makes sense if your scopes match the DOM structure
17:07:36 <gwicke> the knockout syntax is fairly simple (JS object notation in a single data-bind attribute)
17:07:40 * brion is skimming
17:07:54 <brion> is data-bind content syntax plain json or kinda a domain-specific language?
17:08:09 <RoanKattouw> It looks a lot like YADSL to me
17:08:22 <gwicke> it is slightly extended from json, but can be parsed in less than 100 lines
17:08:34 <gwicke> a shallow parse at least
17:08:36 <brion> *nod*
17:08:38 <csteipp> I think the syntax is a concern-- probably one we can live with, but my agenda for getting something standardized is that people use it.. so if the syntax is intimidating, then I'm definitely concerned.
17:08:42 <OwynD_> good morning! catching up from the logs.
17:08:42 <gwicke> https://github.com/gwicke/TemplatePerf/blob/master/QuickTemplate/KnockoutExpression.js
17:08:48 <RoanKattouw> Which in my opinion is pretty annoying and a bad decision on the framework authors' part
17:08:55 <subbu> RoanKattouw, yadsl = yet another dsl?
17:09:14 <RoanKattouw> But I also think that the library as a whole might be the least bad option
17:09:15 <James_F> subbu: I think RoanKattouw means the tertiary meaning, Yet Another Domain Specific Language.
17:09:19 <RoanKattouw> subbu: Yeah
17:09:31 <gwicke> in quicktemplate we restrict things to basically dot notation
17:10:07 <mwalker> *quicktemplate being the current working name for the intermediate language processor
17:10:14 <gwicke> the main reasoning is that we want to keep things simple for now, and want to encourage portability and clear model / template separation
17:10:49 <brion> i like the separation of data model and template; not totally against a slight domain-specific language for the data bindings, but yeah readability and maintainability are important
17:10:50 <robla> gwicke: you mention you benchmarked the solutions.  could you publish your numbers?
17:10:52 <gwicke> we basically implement a simple subset of knockoutjs expressions
17:11:06 <gwicke> robla, you can pull the repo & run them yourself
17:11:06 <sumanah> (btw bebirchall how much is this topic related to https://www.mediawiki.org/wiki/User:5xbe/Proposal ?)
17:11:11 <marcoil> I assume QT in the examples refers to quicktemplate and not the QT C++ framework…
17:11:15 <OwynD_> So, from my perspective, the reason we built out template system was to allow us to build more complicated special pages and other "application" elements.
17:11:18 <mwalker> marcoil, correct
17:11:19 <robla> gwicke: or...you could do it   :-)
17:11:33 <gwicke> the numbers are better than compiled handlebars templates
17:11:35 <OwynD_> so it's a lot like quick templates, with some slightly different design decisions because we had the freedom to do that
17:12:04 <gwicke> which are the fastest other library in the test so far
17:12:59 <OwynD_> it sounds like there's an additional desire for you guys to have a templating system that also works for user content / wikitext.
17:13:04 <jgonera> gwicke, how much weight does it add to the page load?
17:13:25 <milimetric> 16k +  a bit as Nikerabbit mentioned in the comments
17:13:28 <subbu> one question to address would be how much peak performance weighs into decision of a templating system vs. syntax and other desirables
17:13:34 <gwicke> the syntax is fairly flexible; we picked knockoutjs as a starting point as there is a good implementation & it is close to what we want
17:13:34 <csteipp> gwicke: Preliminary numbers would be nice for everyone to see. Also, the code I looked at had a lot of "TODO: sanitize the attributes"... do we have a prototype that does sanitization that you've benchmarked?
17:13:36 <sumanah> #info Roan's concerned with the Knockout syntax being intimidating to people
17:13:51 <sumanah> #info brion likes separating the template from the data model
17:13:55 <RoanKattouw> sumanah: I did not say that
17:13:59 <gwicke> we also have a second compiler from spacebars syntax to JSON IR
17:13:59 <brion> generally i wouldn't be concerned about speed unless it's amazingly slow.
17:14:18 <jgonera> I agree with brion
17:14:24 <csteipp> #info CSteipp is concerned about the syntax being intimidating, not Road :)
17:14:31 <aude> road?
17:14:32 <gwicke> csteipp, I was shooting for an apples-to-apples comparison for now
17:14:37 <sumanah> RoanKattouw: ok, sorry for missynthesizing there - trying to #info some stuff to get it into the simplified meeting summary
17:14:39 <robla> csteipp has a cold  :-)
17:14:44 <aude> :)
17:14:45 <subbu> in my opinion, all html templating library syntax is going to be "uglier" than plain string templating. so, i think some of that is unavoidable.
17:14:48 <mwalker> jgonera, something to note is that although we're using ko syntax, you don't actually need to use it... you only need quicktemplate if the page isn't going to be dynamic
17:14:54 <csteipp> gwicke: But mustache does santization.
17:14:59 * brion agrees with subbu
17:15:01 <gwicke> csteipp, nope
17:15:06 <gwicke> only html sanitization
17:15:12 <gwicke> no attribute sanitization
17:15:17 <gwicke> see the first paragraph
17:15:21 <sumanah> wtf? I don't usually misread that badly. need to get my eyes checked
17:15:22 <RoanKattouw> My Yet Another DSL concerns are more along the lines of "why use an existing language when I can INVENT A NEW ONE and have fun!!1! ... and have bugs in my parser because I'm not reusing established well-tested code"
17:15:53 <milimetric> so let's talk a bit about the knockout syntax then, seems like at least a couple of people are concerned
17:15:58 <milimetric> I'm not sure if it's a DSL
17:16:05 <milimetric> it's json + js expressions where needed
17:16:07 <csteipp> gwicke: Right, but performance-wise, the regexes they are doing are expensive.. does your code do any simple santization?
17:16:23 <gwicke> csteipp, it does simple html escaping as in mustache
17:16:34 <csteipp> Ok cool, I missed that in the code
17:16:34 <gwicke> but no attribute sanitization (protocol filtering etc)
17:16:51 <csteipp> So can you post numbers with this?
17:16:53 <jgonera> mwalker, I might not be up to date with everything, what's quicktemplate? do we have out own JS templating system?
17:17:01 <gwicke> cscott, sure
17:17:04 <MaxSem> also, does anyone still remeber that we're supposed to have a library portable between JS and PHP?
17:17:08 <milimetric> <mwalker> *quicktemplate being the current working name for the intermediate language processor
17:17:17 <jgonera> MaxSem, good point
17:17:34 <gwicke> cscott, for now, between about even with handlebars and 30% or so faster
17:17:43 <gwicke> eh, csteipp
17:17:52 <Nikerabbit> I was under the impression that the current proposal also works in PHP, is that not the case?
17:17:53 * gwicke curses at xchat's completion
17:17:54 <mwalker> MaxSem, yep; right now it's js only, but I don't expect the port to php to be difficult or all that involved; it's some 200 lines of JS
17:18:03 <jgonera> milimetric, mwalker is there any wiki page about it? any repo?
17:18:20 <OwynD_> i can find some examples of quick template, one sec
17:18:20 <milimetric> https://www.mediawiki.org/wiki/Requests_for_comment/HTML_templating_library/KnockoutProposal
17:18:38 <csteipp> gwicke: Actual numbers in different scenarios would be really nice to see :)
17:18:39 <MaxSem> mwalker, DOM manipulations are native to JS, however they all suck in PHP
17:18:40 <jgonera> milimetric, I searched for "quicktemplate" on that page, 0 results
17:19:01 <milimetric> right, it's just what gwicke's calling it, don't think it shows up on that page
17:19:02 <brion> DOM isn't that awful in PHP. it's just verbose ;)
17:19:09 <gwicke> csteipp, will post those after re-running it- but don't take my word for it ;)
17:19:24 <milimetric> and by it I mean the intermediate language processor that takes knockout templates to the IR
17:19:28 * MaxSem refers brion to HtmlFormatter and its horrors
17:19:28 <mwalker> jgonera, it's hidden in a note: https://github.com/gwicke/TemplatePerf/tree/master/QuickTemplate is the code
17:19:29 <gwicke> jgonera, https://github.com/gwicke/TemplatePerf/tree/master/QuickTemplate
17:19:41 <jgonera> thanks
17:19:50 <OwynD_> this is an old signup form using a QuickTemplate class:
17:19:51 <OwynD_> https://github.com/Wikia/app/blob/dev/extensions/wikia/SpecialSignup/templates/Signup.php
17:19:59 <Isarra> Isn't it also slower in php?
17:20:02 <RoanKattouw> MaxSem: DOM manipulation is native in *browser* JS, it also sucks performance-wise in nodejs
17:20:03 <sumanah> should we #action something about stats?
17:20:05 <gwicke> so far this is the result of about two days
17:20:20 <OwynD_> it has an execute() function and then jumps into raw PHP.  it's pretty ugly.
17:20:22 <robla> like RoanKattouw, I'm pretty skeptical of new shiny.  if we do something new, we should make sure what we do becomes ubiquitous rather than having *yet* *another* weird MediaWiki-specific way of doing things.
17:20:27 <jgonera> ugh, so that's basically writing HTML in JSON?
17:20:30 <RoanKattouw> TrevorParscal: Log of discussion so far http://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-office/20140221.txt ; proposal we're discussing (read this first): https://www.mediawiki.org/wiki/Requests_for_comment/HTML_templating_library/KnockoutProposal
17:20:32 <jgonera> gwicke, ^
17:20:46 <gwicke> jgonera, you mean the JSON IR?
17:20:52 <gwicke> that's a compilation target
17:20:55 <jgonera> IR?
17:21:01 <mwalker> intermediate representation
17:21:01 <milimetric> intermediate representation
17:21:04 <jgonera> right
17:21:08 <OwynD_> i think the QuickTemplate that gwicke is talking about is totally different.  so, i was a bit confused. :)
17:21:15 <jgonera> but someone said that I can just use this if I don't want knockout
17:21:17 <gwicke> you can write it manually too if you feel like it
17:21:17 <milimetric> yes OwynD_
17:21:37 <jgonera> mwalker said that
17:21:45 <mwalker> OwynD_, ya... sorry --- our quicktemplate is a string based json ir to html library -- we couldn't come up with a better name last night to avoid confusions...
17:21:49 <gwicke> OwynD_, sorry for the confusing working name
17:21:56 <RoanKattouw> gwicke: This may be in the RFC and I may have missed it: is there or will there be code that translates the JSON IR to straight up non-Knockout HTML?
17:22:00 <OwynD_> no problem, all cleared up now.
17:22:00 <subbu> Knockout.JS -- (compiled by quickTemplate) --> IR ... i doubt anyone would write IR directly, that is like writing assembly.
17:22:03 <milimetric> yeah, mwalker/gwicke, are you envisioning people writing directly in IR if they want?
17:22:05 <jgonera> yes, but the thing is I don't think anyone would feel like writing the IR, gwicke, mwalker ;) so it is either knockout or nothing
17:22:06 <gwicke> spent about two seconds thinking about a name..
17:22:22 <subbu> sorry, not knockout.js but knockout.js templats
17:22:24 <gwicke> RoanKattouw, normally you'd want to do the reverse
17:22:29 <gwicke> that's what's implemented
17:22:38 <RoanKattouw> Well on the server side, yes
17:22:45 <RoanKattouw> I have a template, I precompile it to JSON IR
17:22:47 <mwalker> jgonera, yesish; it's KO syntax to JSON IR, and then use the quicktemplate stuff to render it (I was pointing out that you don't need the 16kb of KO.js if you don't need it on the page)
17:22:48 <gwicke> but converting from JSON IR to knockout should be feasible too
17:22:50 <RoanKattouw> JSON IR is delivered to the client
17:22:59 <RoanKattouw> Client does ??? , stuff appears on the user's screen
17:23:11 <RoanKattouw> I assume ??? == "render JSON IR to plain HTML" in this case?
17:23:16 <mwalker> RoanKattouw, yes
17:23:22 <RoanKattouw> OK sweet
17:23:43 <csteipp> So can one of you expand on template inclusion? From a reviewer perspective, that looks like it could be a lot of work if you have templates full of template references. Does the entire <div> get substituted in your example?
17:23:49 <gwicke> in the server-side use case loading the pre-compiled JSON from memcached and interpreting it should be pretty fast
17:23:51 <RoanKattouw> Just making sure we wouldn't be locked into any Knockout overhead on the client, even for places where we use it as a server-side templating language
17:24:03 <gwicke> much faster than DOM-based templating
17:24:15 <gwicke> while still providing the same guarantees re nesting and sanitization
17:24:52 <TrevorParscal> there was work done a few years ago to bring wikitext i18n messages to the client in a sort of JSON IR, are you guys aware of that?
17:25:04 <mwalker> csteipp, I believe the div becomes a parent element of the included template
17:25:07 <gwicke> TrevorParscal, ohh, interesting ;)
17:25:13 <gwicke> I didn't know about that
17:25:17 <TrevorParscal> this is how we do complex client "parsing" for gender and stuff
17:25:18 <gwicke> we are reinventing the wheel then
17:25:19 <RoanKattouw> TrevorParscal: By Neil K, I think?
17:25:22 <milimetric> yes mwalker, that's right about inclusion
17:25:31 <RoanKattouw> Yeah and doesn't the PLURAL/GENDER stuff in mw.jqueryMsg or whatever it is use this
17:25:32 <TrevorParscal> yes, Neil
17:25:32 <brion> yeah i recall that being used for uploadwizard
17:25:56 <TrevorParscal> i'm not saying it's as complete as this system, but just something to be aware of
17:26:10 <gwicke> the beauty of a low-level IR is that you can compile different syntaxes to it
17:26:17 <jgonera> gwicke, I'm not exactly sure what is the purpose of the IR
17:26:18 <Nikerabbit> as far as I know there is no IR that is being transmitted from server to client for i18n, just the wikitext
17:26:19 <gwicke> one syntax is wikitext, with the help of parsoid
17:26:21 <TrevorParscal> as I recall, it actually did prove very efficient despite some initial skepticism
17:26:37 <mwalker> csteipp, I'm not sure of how to avoid the template inclusion rats nest issue -- but if we're autoescaping things; what's your concern?
17:27:05 <gwicke> jgonera, the purpose of the IR is to have a very efficient and small interpreter / runtime
17:27:05 <TrevorParscal> the point of IR should be to spare the client the complex part of the process, which is parsing
17:27:09 <RoanKattouw> Hmm it might be that the message stuff is a local parser that parses wikitext into an IR on the client, rather than sending precompiled IR over the network
17:27:14 <mwalker> jgonera, the point of the IR is so that we can have a nice safe DOM based templating language that's slow to compile; but then string based on render which is really fast
17:27:21 <RoanKattouw> Which as Trevor just beat me to saying, is missing the point of an IR
17:27:24 <Nikerabbit> RoanKattouw: that
17:27:47 <jgonera> gwicke, mwalker but why if nobody will want to write IR by hand anyway?
17:27:57 <TrevorParscal> the IR should make it possible to take immediate action, not have to lookup data in the database and be in a high-performance format (JSON qualifies)
17:28:03 <gwicke> jgonera, that's what compilers are for
17:28:13 <RoanKattouw> jgonera: Execution of DOM modification is slow in environments that aren't browsers
17:28:17 <RoanKattouw> Including nodejs
17:28:34 <RoanKattouw> So being able to do manipulation on a JSON structure that's equivalent, then rendering it to HTML at the end is faster
17:28:34 <csteipp> mwalker: If it's appending the included elements into the parent, then that should be fine. I'm assuming the template has to be valid xml, so someone can't do <div id="<div template..."/>
17:28:40 <jgonera> RoanKattouw, any particular use cases we are talking about?
17:28:56 <RoanKattouw> jgonera: Someone writes a Knockout template on the server
17:28:56 <TrevorParscal> I believe in the general approach to doing the parsing on the server, and the dom building on the client
17:28:57 <mwalker> csteipp, yes; part of the final compiler will be to ensure that templates are well formed
17:28:58 <jgonera> I mean, do we even know what we want to use the client-side templating for?
17:29:00 <gwicke> csteipp, that's covered by attribute sanitization
17:29:08 <RoanKattouw> Or in a wikitext template or whatever
17:29:15 <gwicke> there is an attr binding that produces dynamic attributes
17:29:19 <RoanKattouw> Server precompiles to JSON IR, sends to client, client renders to HTML
17:29:24 <gwicke> we can do any kind of sanitization in there
17:29:34 <TrevorParscal> also, the JSON IR can be cached
17:29:38 <TrevorParscal> of course
17:29:42 <gwicke> csteipp, https://github.com/gwicke/TemplatePerf/blob/master/QuickTemplate/quicktemplate.js#L102
17:29:45 <Nikerabbit> jgonera: I want to use to separate html layout from logic
17:29:57 <jgonera> RoanKattouw, don't we lose all the benefits of Knockout (dynamic data binding) if we write in on the server?
17:29:58 <mwalker> flow also uses it to do rerendering of message boards
17:30:25 <jgonera> Nikerabbit, that's obvious, but I'm asking about specific examples where performance would be such a huge concern
17:30:26 <RoanKattouw> jgonera: If you want those benefits you need to write things on the client, is my understanding
17:30:28 <OwynD_> there's always some logic in the layout.  if you have an optional button, it's logic...
17:30:37 <RoanKattouw> This is just a way to use Knockout as a non-dynamic templating language
17:30:38 <mwalker> jgonera, yesish; server side rendering will not be dynamic -- it'll get you a static page that you can then attach KO to on the client
17:30:45 <subbu> jgonera, https://www.mediawiki.org/wiki/Requests_for_comment/HTML_templating_library/KnockoutProposal#Longer-term_architecture has a path for that if i udnerstand corectly
17:30:50 <RoanKattouw> Which sounds stupid but is apparently the fastest way they found to do server-side templating
17:31:03 <gwicke> jgonera, and https://www.mediawiki.org/wiki/Requests_for_comment/HTML_templating_library/KnockoutProposal#Server-side_pre-rendering
17:31:15 <RoanKattouw> (while using the same template format across the board)
17:31:31 <milimetric> yeah, jgonera, that last point by RoanKattouw is the basic motivation for all this
17:31:32 <gwicke> we can do server-side pre-rendering, but leave the knockout data in the DOM
17:31:42 <gwicke> client-side things can then be updated dynamically
17:32:08 <brion> i do like that
17:32:14 <milimetric> a small point about this dynamic client side behavior
17:32:20 <gwicke> the knockout attribute syntax actually helps with that
17:32:23 <brion> ability to start with filled-out data (for fast load and non-JS users) and then let JS update things live
17:32:27 <milimetric> it's not completely trivial to include a template dynamically and then have it start updating
17:32:34 <gwicke> as you can keep it in the DOM without breaking the rendering
17:32:39 <gwicke> it's also valid HTML5
17:32:43 <milimetric> you have to tell knockout about it and whether you want to control the bindings in the child template or it should do it for you
17:32:49 <milimetric> but that can be genericised
17:33:03 <TrevorParscal> i think baking data in makes caching less useful
17:33:07 <milimetric> I think this answers one of Nikerabbit's points on the RFC proposal discussion page
17:33:12 <mwalker> milimetric, yes that's true... but everything we looked at indicated it wouldn't be hard to make something to do that for us
17:33:19 <gwicke> there are some solvable issues in the pre-rendering area: https://www.mediawiki.org/wiki/Requests_for_comment/HTML_templating_library/KnockoutProposal#cite_note-3
17:33:32 <milimetric> i've built that a few times mwalker, one way to do it is a simple custom binding, about 10 lines
17:33:59 <mwalker> milimetric, can you add a reference to that somewhere on our wiki page?
17:34:03 <gwicke> TrevorParscal, it is a trade-off
17:34:07 <milimetric> sure, but it's in coco :P
17:34:16 <gwicke> pre-rendering does not make sense for everything
17:34:30 <TrevorParscal> sure, but it's a practice that can get you into trouble if you aren't drawing the line carefully enough
17:34:50 <mwalker> milimetric, heh; might still be useful
17:34:57 <gwicke> it is a possibility though, and we could make it easy to choose whether things should only run on the server, only on the client or on both server & client
17:35:03 <OwynD_> i think figuring out what you want the end result to look like comes before performance considerations.  you can make "most" things fast, one way or another. :)
17:35:17 <OwynD_> once you make something ugly, you're stuck with it
17:35:33 <gwicke> OwynD_, agreed
17:35:42 <brion> anybody up to trying a branch of their ext or core bit using this sort of system, see how it works in real life?
17:36:07 <ebernhardson> we've been looking for a templating solution in flow, but php is a must
17:36:14 <mwalker> brion, I have a couple of fundraising extensions
17:36:21 <Nikerabbit> I'm willing to try this as well
17:36:28 <brion> great
17:36:29 <OwynD_> yeah, i think one good experiment would be to convert a really simple extension with both js and PHP components to using this system.
17:36:30 <subbu> OwynD_, yes .. to repeat an earlier qn: one question to address would be how much peak performance weighs into decision of a templating system vs. syntax and other desirables
17:36:30 <gwicke> it is fairly straightforward to define a subset of the knockout syntax, and even enforce that in knockout by modifying the expression parser slightly
17:36:37 <TrevorParscal> premature optimization is evil, but considering caching and basic performance characteristics of server and client is a reasonable thing to do
17:36:39 <Nikerabbit> but for that I need more concrete ideas how this integrates with mediawiki (as I pointed out in the talk page)
17:36:42 <jgonera> OwynD_, agreed
17:36:52 <TrevorParscal> OwynD_: +1 about using this thing
17:36:58 <TrevorParscal> something non-trivial
17:37:03 <sumanah> #idea <OwynD_> yeah, i think one good experiment would be to convert a really simple extension with both js and PHP components to using this system.
17:37:04 <gwicke> OwynD_, the JSON IR decouples syntax & performance somewhat
17:37:11 <brion> gwicke: is the PHP side still all theory or any work on that yet?
17:37:15 <milimetric> mwalker: https://www.mediawiki.org/wiki/Thread:Talk:Requests_for_comment/HTML_templating_library/KnockoutProposal/General_thoughts/reply
17:37:32 <gwicke> brion, all theory right now, but porting ~600 lines of js total should not be that hard
17:37:37 <brion> spiff
17:37:37 <jgonera> I really appreciate all the effort put into this, but I'm a bit afraid we are overengineering things for our current needs
17:37:37 <gwicke> the runtime is only 260 lines
17:37:45 <brion> who wants to commit to setting that up?
17:37:51 <mwalker> that's on my plate
17:38:06 <TrevorParscal> It's important to actually use this in something non-trivial, it will evolve quickly - OOJS and OOJS-UI area both products of necessity, we need them, and that is what makes them useful - they actually solve problems
17:38:29 <brion> jgonera: it doesn't feel too big to me; it feels just flexible enough to solve known issues
17:38:33 <TrevorParscal> I'm concerned about this being used for UI, but that's another RFC I guess
17:38:35 <brion> but not *too* flexible to be crazy
17:38:36 <OwynD_> i think that will really help.  may be a simple caveman developer, but it's hard to evaluate this without seeing a bigger example.
17:38:44 <sumanah> so what specifically is the #action for mwalker? :) converting an extension to use the proposed sys?
17:38:56 <gwicke> so my instinct re syntax is to start with a really restricted subset
17:38:59 <Nikerabbit> TrevorParscal: what do you mean with that?
17:39:06 <mwalker> sumanah, my action is to port the JS IR compiler / runtime to PHP
17:39:11 <gwicke> basically only dot notation and literals
17:39:15 <sumanah> #action mwalker <mwalker> sumanah, my action is to port the JS IR compiler / runtime to PHP
17:39:34 <TrevorParscal> UI shouldn't be using widget libraries, that's what I mean - not trying to derail this convo, but I'm really serious about that
17:39:55 <OwynD_> Trevor: that was kind of my earlier point at the summit... i think it might be difficult to get 1 solution that works for both content and UI.  they're different domains, different type of user.  (content authors vs developers)
17:39:58 <TrevorParscal> sorry, SHOULD be using
17:40:09 <milimetric> :)
17:40:12 <milimetric> i was gonna say TrevorParscal
17:40:23 <sumanah> ok, any more #agreed decisions or #action items before we wrap up today's HTML templating discussion and move on to the next topic?
17:40:33 <sumanah> brion: ^
17:40:34 <TrevorParscal> OwynD_: yeah, content and UI are quite different, they need different tools
17:40:37 <sumanah> gwicke: ^
17:40:51 <brion> i think we're about good for now
17:40:58 <jgonera> I'd say that seeing this in action in a specific use case should also be an action item
17:41:03 <sumanah> go ahead jgonera
17:41:08 <sumanah> mark it
17:41:13 <mwalker> #action mwalker Answer Nikerabbit's questions about RL integration
17:41:15 <milimetric> I think what knockout could potentially give us is a standard templating approach to wrap around a widget library
17:41:28 <jgonera> #action seeing this in action in a specific use case
17:41:48 <sumanah> who'll be doing that?
17:42:07 <gwicke> I am also curious to heard about potential use cases for reactivity
17:42:11 <gwicke> *hear
17:42:20 <jgonera> I'm afraid I have too much on my plate in the nearest couple of weeks...
17:42:26 <mwalker> sumanah, presumably either gabriel or myself
17:42:36 <OwynD_> i like the idea of a generic system in core that allows people to build reusable widgets/ui components, in extensions and even in user land (like gadgets, etc)
17:42:41 <milimetric> for something not in the mediawiki universe but close, you can check out wikimetrics's use of knockout and reactivity
17:43:04 <gwicke> milimetric: *nod*
17:43:13 <sumanah> #action gwicke or mwalker to demonstrate this in action in a specific use case ("this" being mwalker's port?)
17:43:30 <sumanah> ok, we basically done here? I'm about to switch topic to Services and narrow interfaces
17:43:56 <gwicke> ideally I'd like to partner with a real project
17:44:16 <gwicke> so if anybody has a small but real project with templating needs coming up, let us know
17:44:17 <MaxSem> do we have time for SoA?
17:44:24 <milimetric> i can dedicate some of my volunteer time to help someone implement this in a real project
17:44:26 <jgonera> gwicke, what's the timeline? how soon do you want to do it?
17:44:28 * MaxSem doubts this
17:44:43 <gwicke> jgonera, with ~1 month?
17:44:46 <milimetric> but I have no mediawiki extension experience, so preferably it'd be someone with an existing extension
17:44:47 <gwicke> within
17:44:49 <OwynD_> yeah, i think that sounds like a good place to switch tracks.
17:45:06 <gwicke> alright, lets move on for now
17:45:18 <sumanah> #topic Services and narrow interfaces
17:45:22 <jgonera> gwicke, ok, I'd be interested in trying that out in mobile too then, after I'm done with some other things and possibly when Jon is back and we have more bandwidth
17:45:22 <sumanah> #link https://www.mediawiki.org/wiki/Requests_for_comment/Services_and_narrow_interfaces
17:45:26 <jgonera> which is in 1-2 weeks
17:45:29 <gwicke> we have been discussing packaging and distribution recently
17:45:53 <James_F> sumanah: We *really* need the topic to note the logging, per Legal.
17:45:57 <gwicke> jgonera: ok, cool
17:46:15 <gwicke> parsoid now has a debian package
17:46:22 <brion> \o/ package ftw
17:46:29 <gwicke> which is published only in a labs repository so far
17:46:44 <James_F> Thanks.
17:46:51 <RoanKattouw> (Missed one when converting to pipe-separation, sorry)
17:47:09 <sumanah> James_F: I'll take that out of this mtg and talk with Tim L about our meetbot install or something, thx
17:47:10 <gwicke> so one thing we are looking for is a longer-term apt repository for third party users
17:47:18 <James_F> sumanah: Sure.
17:47:21 <sumanah> #action sumanah to work on topic including "Channel is logged and publicly posted (DO NOT REMOVE THIS NOTE). https://meta.wikimedia.org/wiki/IRC_office_hours" - talk with Tim L
17:47:25 <brion> so we've got some sort of apt repository we have our custom packages in.... do we need a separate one for public-targeted packages?
17:47:33 <paravoid> yeah, that'd be best
17:47:34 <gwicke> the second point is an automated procedure to upload debs to that repository without root
17:47:50 <paravoid> so, I'll explain that, as soon as gwicke is done
17:48:09 <rdwrer> Small note, guys: There's an office hour immediately after this
17:48:24 <gwicke> the third aspect is potentially using packages for service deployments
17:48:25 <rdwrer> Which maybe would have been more evident had there been an entry in https://meta.wikimedia.org/wiki/IRC_office_hours about this meeting
17:48:28 <rdwrer> But alas
17:48:33 <sumanah> thx rdwrer - will fix that for future
17:48:39 <brion> live and learn :)
17:49:08 <brion> gwicke: so what needs to be planned for?
17:49:28 <paravoid> I don't think we can discuss all these issues in 10'
17:49:43 <gwicke> yeah, we might only get partway through
17:49:45 <paravoid> let's focus on third-party distribution for now, since that's easier?
17:49:54 <OwynD_> okay, one thing about this particular draft proposal...  it's similar to the discussions we had a few months ago about an API 2.0
17:50:02 <OwynD_> or should we just discuss packaging?
17:50:16 <brion> let's stick to packaging since we're short on time
17:50:21 <gwicke> OwynD_, this is just about packaging & distribution
17:50:40 <paravoid> right
17:50:43 <gwicke> paravoid, the decision on 3) will change the best solution for the repo
17:51:01 <gwicke> or rather, might
17:51:05 <mark> there's no way we're gonna reach a decision on 3) today :)
17:51:12 <Ryan_Lane2> especially since there's no prototype for it
17:51:20 <gwicke> it could be feasible to start with a public repo for now
17:51:34 <sumanah> #link https://www.mediawiki.org/wiki/Talk:Requests_for_comment/Services_and_narrow_interfaces - please do talk on the talkpage after this meeting :)  -- hasn't been talked on since 3 January
17:51:35 <gwicke> and if we move to deb deploys automatically dput to that repo too
17:51:35 <paravoid> I think that providing up-to-date packages for mediawiki & "mediawiki services" would do a great service to our users
17:51:38 <greg-g> (nit: is the third point of packages as deploy for services the need that the second point is helping? ie: without the 3rd, is there still a need for gwicke's second point?)
17:51:41 <gwicke> in addition to an internal repo
17:51:57 <paravoid> distributions' release schedules, even the aggressive ones, tend to have different priorities than we do
17:52:06 <paravoid> so a model similar to http://mozilla.debian.net/ would be great, I think
17:52:25 <brion> yes i def agree we should have a public repo that's easy to add to a debian or ubuntu system
17:52:33 <paravoid> great
17:52:37 <sumanah> #agreed <brion> yes i def agree we should have a public repo that's easy to add to a debian or ubuntu system
17:52:41 <paravoid> on the matter of reusing apt.wm.org
17:52:53 <paravoid> apt.wm.org is implicitly trusted across the wikimedia infrastructure
17:53:19 <bd808> Could we just used a PPA as an external deb publication point?
17:53:20 <paravoid> there are all kinds to attack vectors by opening access rights to that outside of a very strict set of users
17:53:21 <gwicke> we'd need that for deploys, but not necessarily for third party use
17:53:35 <paravoid> additionally, apt is not great at having multiple versions of the same package around
17:53:35 <bd808> *use a PPA
17:53:45 <brion> PPA works well for ubuntu; is that still easy to use on debian?
17:53:46 <gwicke> paravoid, why's that?
17:53:58 <paravoid> because you can't easily pin a "branch" or a "train"
17:54:02 <gwicke> paravoid, I thought that's mainly a limitation of reprepo
17:54:02 <paravoid> newer versions win
17:54:24 <paravoid> it's definitely a limitation of reprepro, but think of the simpler way of having to support e.g. mediawiki 1.21 and 1.20
17:54:25 <gwicke> you can hold a package
17:54:36 <paravoid> and users wanting point releases (security update) for 1.20, but sticking to 1.20
17:54:41 <gwicke> it won't be upgraded automatically then
17:54:49 <paravoid> there's no way you can do this with multiple versions in apt
17:54:52 <paravoid> so in the example I gave above
17:54:55 <paravoid> http://mozilla.debian.net/
17:55:14 <bd808> brion: "On debian 7 the add-apt-repository command is available and can be used to add any launchpad ppa repository in a single command." -- http://www.binarytides.com/add-ubuntu-ppa-repository-to-debian-7-wheezy/
17:55:16 <paravoid> you can see how multiple branches are maintained: using different suites
17:55:23 <brion> nice
17:55:50 <gwicke> paravoid, major versions / branches can also be handled with different package names
17:56:00 <paravoid> nah, that's really ugly
17:56:04 <gwicke> similar to python etc
17:56:22 <paravoid> you want people to do "apt-get upgrade" and get the latest version for the release train they've picked, I think
17:56:26 <paravoid> I haven't spent much time thinking of it, though
17:56:26 <gwicke> it is a common solution, and it does work
17:56:42 <brion> ok we're running low on time
17:56:50 <brion> what do we need to decide to talk further about?
17:56:51 <paravoid> my idea was to use a format like the mozilla.d.n, but under the releases.wikimedia.org host
17:56:54 <bd808> Apparently hashar has even registered https://launchpad.net/mediawiki
17:57:05 <brion> ooh i like using releases.wm.o
17:57:19 <greg-g> I plus 1 the mozilla.debian idea, instead of the python pakage versioning insanity (all parties have a hard time with that)
17:57:29 <gwicke> I think it would be great to start with a public repo that supports multiple versions per package
17:57:40 <paravoid> and provide different components and suites for mediawiki, parsoid etc. and different branches of them
17:57:51 <gwicke> with a script to push to it from tin or the like
17:57:55 <paravoid> and allow uploads from the respective developers of each team
17:57:59 <paravoid> instead of just ops
17:58:15 <gwicke> or a direct dput setup
17:58:31 <brion> #info PPA is a possibility; as is running a repo on releases.wikimedia.org. versioning still an issue to discuss
17:58:34 <paravoid> yup, dput and proper archive would be geat too, if developers can endure that :)
17:58:44 <paravoid> *great
17:58:57 <brion> great :D
17:59:03 <brion> can you guys add some notes on the talk page?
17:59:05 <brion> and we'll wrap up
17:59:08 <greg-g> "uploads to components based on team membership"++
17:59:08 <bd808> scap has proven that developers can endure anything
17:59:13 <brion> #action notes to talk page...
17:59:28 <paravoid> I should say, gwicke has been driving and pushing this, so credit belongs to him
17:59:45 <paravoid> I'm just giving my opinion, based on my experience/background
17:59:46 <sumanah> #info next meeting, MaxSem to talk about inline diffs
17:59:51 <brion> sumanah: ok iirc we have to wrap up for the office hours in here now?
17:59:52 <gwicke> Antoine has also been very active
17:59:53 <sumanah> #endmeeting
Retrieved from "https://www.mediawiki.org/w/index.php?title=Architecture_meetings/RFC_review_2014-02-21&oldid=915117"