Requests for comment/Magic expressions

Request for comment (RFC)
Magic expressions
Component	General
Creation date	2014-02-11‎
Author(s)	Pastakhov
Document status	implemented

Background

It is well known that the Magic words are very popular, but often one is not easy to use and have performance problems. This issue was discussed here and it was decided in favour of the Scribunto extension. I'm sure it can not be the sole true solution.

Another way

It seems that it was the first way. I just now saw the extension WikiScripts. I have not seen the code, but the description is similar to my solution. I call it magic expression.

The problem with magic occurs only when one trying to be like an expression. Magic expression is the magic words in a scripting expression. This makes the magic words user-friendly and easier for the extension developers.

Several times I did refactoring the code, and achieved excellent results.

It is the Extension:PhpTags (former title is the Extension:Foxway).

Scope and issues

Performance. somewhere between 10-500 times slower pure PHP. Weakness: the variables in the loop.
- I think it's more than enough. I am sure that it is possible not to use any loops on the wiki pages. What do you think? --Pastakhov (talk) 03:53, 12 February 2014 (UTC)[reply]
Caching. The code compilation is an expensive operation and it should be cached.
- Compilation result is an array with references. One can be serialized and it's cheap. How can it be cached? --Pastakhov (talk) 04:54, 12 February 2014 (UTC)[reply]
Syntax. I stuck with the syntax PHP, but I can also add anything from javascript.
- Will it be convenient or may confuse users? --Pastakhov (talk) 04:19, 12 February 2014 (UTC)[reply]
Sandboxing. Users must not be allowed to abuse the server resources using the scripting language. The solution must be protected from DoS attacks and security exploits.
- PhpTags sanitizes the returned data if it is not raw html from another extension. PhpTags never executes any code directly. --Pastakhov (talk) 04:19, 12 February 2014 (UTC)[reply]
- Will the limiting number of loops helps to avoid DoS attacks or need some other measure? --Pastakhov (talk) 04:19, 12 February 2014 (UTC)[reply]
Isolation No, it should not be like a black box.
- The scope of variables is limited only by own wikipage, unless they are declared as static or global. Example (Just like for functions in pure PHP) --Pastakhov (talk) 04:34, 12 February 2014 (UTC)[reply]
Extensibility. Any extension can be integrated like a constant, function or object. (there is no direct code execution, it is like magic words)
- I do Extension:PhpTags Functions that implement various PHP functions. --Pastakhov (talk) 09:40, 12 February 2014 (UTC)[reply]
Universal parser. PhpTags takes the task of parsing parameters and transmits them to other extensions as an array.
- I think it will facilitate the extension developers, especially such as in the Bug 54221.
Debugging. PhpTags can provide debugging information by which javascript can restore the sequence of operations and their intermediate values.
- I tried do it for Extension:Foxway on the server side, turned expensive and inconvenient. I think it's better to do it on the client side. --Pastakhov (talk) 05:19, 12 February 2014 (UTC)[reply]
Code editor. Something like EditArea and integration VisualEditor. Any ideas?
- ...
Portability. It uses pure PHP. Does not require anything else.
- ...
Reverse. It can be used as a safe javascript on the client side too.
- To do this, need to write a runtime class for javascript, I think it is not difficult. --Pastakhov (talk) 09:59, 12 February 2014 (UTC)[reply]

Performance

Benchmark 1. Brute force ( 100 000 loops )

Source of Benchmark 1:

$r = 0;
$i = 0;
while ( $i < 100 ) {
    $i++;
    $j = 0;
    while ( $j < 1000 ) {
        $j++;
        $r = ($r + ($i * $j) % 100) % 47;
    }
}

I did it on my notebook HP ProBook 4530s, core i5, 4G ram, openSUSE 12.3. Pure PHP is PHP 5.3.17, and PhpTags is Extension:PhpTags 1.0.7

Pure PHP took:  0.036 sec
PhpTags took:  18 sec
PhpTags/PHP:  500

Yes, it is 500 times slower than PHP, but it's just 0.00018 sec per loop (0.000015 sec for one operation)!!!

Benchmark 2. Big script

See link.

My HP ProBook 4530s:

Pure PHP: 0.007
PhpTags time usage: 0.634 sec
          Compiler: 0.517 sec
           Runtime: 0.117 sec
PhpTags / PHP = 0.634/0.007 = 90
PhpTags\Runtime / PHP = 0.117 / 0.007 = 16.7

test.foxway.org: (Xen VM on HP DL160G6)

Pure PHP: 0.005
PhpTags time usage: 0.283 sec
          Compiler: 0.236 sec
           Runtime: 0.047 sec
PhpTags / PHP = 0.283/0.005 = 57
PhpTags\Runtime / PHP = 0.047 / 0.005 = 9.4

Wow, PhpTags can run for large scripts only about 10 times slower than pure PHP. It seems incredible. Did I make a mistake in the calculations?

I do not consider compile time, because it will be done once when the page is saved. Then the compiled code will be taken from the cache. It should be cheap.