Extension talk:Cargo/Archive June to October 2015

gallery error

{{#cargo_query:tables=Pictures

|fields=Image

|where=Pictures.Ref="{{PAGENAME}}"

|format=gallery

|mode=packed-hover

}}

If i have a blank field it will error. dunno if you can catch and handle that.

Fatal error: Call to a member function getText() on a non-object in /home2/juanitos/public_html/w/w/includes/gallery/ImageGalleryBase.php on line 235

Sorry about that, and thanks for letting me know. I just checked in what I think is a fix... Yaron Koren (talk) 23:24, 8 June 2015 (UTC)

Using Queries Operator

Is it possible to use queries operators like 'minus' http://www.techonthenet.com/sql/minus.php ? Guillaume Prêcheur (talk) 12:52, 10 June 2015 (UTC)

You can't use "MINUS". Is that a standard SQL operator? It seems like it only exists for Oracle... Yaron Koren (talk) 13:37, 10 June 2015 (UTC)

map display stopped working..

i dunno why but my map display stopped working. It works fine on the semantic forms edit page but on the cargo display it failed. I'm not too sure if this is directly caused by cargo or something else.

inspect the element below the Location heading http://pepperdatabase.org/wiki/Juanitos_2015_Grow

ReferenceError: google is not defined ReferenceError: google is not defined(…) load.php?debug=false&lang=en&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=20150223T1…:13

It displays for me, actually. I think there's a problem with Cargo's Google Maps implementation where sometimes things get loaded in the wrong order. Most of the time it works, though. If you reload a few times, does the map never show up? Yaron Koren (talk) 16:29, 17 June 2015 (UTC)

Ok it doesn't display at all on google chrome canary, works ok after i refresh a couple times in chrome and works with IE. guess they did something funky on canary

Problems in Cargo 0.8/MW 1.25.1

Hi Yaron,

First of all - many thanks for such a brilliant extension! If not for you I would have to invent it by myown. I have also completely translated ru.json into Russian.

Still I got some problems. In most cases I found also workarounds.

The system: Mediawiki 1.25.1, Cargo 0.8 (latest master release), NO SMW installed.

Problem 1. Display of DATETIME.

I create a table with 2 standard fields, which will keep date/time of the creating and updating of the record:

  {{#cargo_declare:_table=Ads
    |Description=Text
    |UpdatedDT=Datetime
    |CreatedDT=Datetime
  }}

  {{#cargo_store:_table=Ads
    |Description={{{Description|}}}
    |UpdatedDT={{LOCALTIMESTAMP}}
    |CreatedDT={{{CreatedDT|}}}
  }}

Insert some data.

mysql> SELECT  UpdatedDT, CreatedDT FROM cargo__Ads ;
+---------------------+---------------------+
| UpdatedDT           | CreatedDT           |
+---------------------+---------------------+
| 2015-06-26 16:57:48 | 2015-06-26 12:39:19 |
+---------------------+---------------------+

Problem 1a. Rewriting of date/time values: the last one rewrites any previous.

  |-
  ! Updated:
  |  {{{UpdatedDT|}}} 
  |-
  ! Created:
  |  {{{CreatedDT|}}} 

Expected: two different date/time values.

Real: only CreatedDT in BOTH fields. The last value of type date/time completely rewrites any previous value.

Workaround: use additional cargo query:

  ! Updated:
  | {{#cargo_query:
    |tables=Ads
    |fields=UpdatedDT
    |where=_pageName='{{FULLPAGENAME}}'
  }}
  |-
  ! Created:
  |  {{{CreatedDT|}}} 

Problem 1b. English default formatting of date/time.

I use workaround from 1a.

Expected: 2015-06-26 16:57:48

Real: 2015-06-26 4:57:48 PM

The problem is that 1st variant is neutral and suitable for any language, while the 2nd - only for English-speaking audience.

Workaround:

use fake CONCAT in the cargo query or backticks around field name -

  ! Updated:
  | {{#cargo_query:
    |tables=Ads
    |fields=CONCAT(UpdatedDT)
    |where=_pageName='{{FULLPAGENAME}}'
  }}

Problem 1c. Unexpected date/time format.

I use MySQL function to format date/time:

  {{#cargo_query:
    |tables=Ads
    |fields=DATE_FORMAT(UpdatedDT,'%d.%M.%Y %H:%i:%s')
    |where=_pageName='{{FULLPAGENAME}}'
  }}

Expected: 26.06.2015 16:57:48

Real: 2015-06-26

Workaround: Use additional CONCAT:

 CONCAT(DATE_FORMAT(UpdatedDT,'%d.%m.%Y'), DATE_FORMAT(UpdatedDT, ' %H:%i:%s'))

Problem 2. Special:CargoExport doesn't support compound_query with calendar format.

I have different types of events and want to display them with different colors in calendar.

  {{#cargo_declare:_table=Events
    |Description=Text
    |StartDT=Datetime
    |EventType=Page
  }}

If I specify only ONE query - all is OK:

  {{#cargo_compound_query:
    |tables=Events;fields=_pageName,_pageTitle=name,StartDT;where=EventType='Exhibition';color=green
    |format=calendar
  }}

If I specify more than one query - results in empty string from the backend:

  {{#cargo_compound_query:
    |tables=Events;fields=_pageName,_pageTitle=name,StartDT;where=EventType='Exhibition';color=green
    |tables=Events;fields=_pageName,_pageTitle=name,StartDT;where=EventType='Movie';color=blue
    |format=calendar
  }}

With JavaScript debugger I discovered, that backend doesn't support notation like tables[1]...tables[n]:

 http://MYDOMAIN/index.php?title=Special:CargoExport&tables=Events&&fields=_pageName%2C_pageTitle%3Dname%2CStartDT&where=EventType%3DExhibition&order+by=_pageName&limit=100&format=fullcalendar&color[0]=green&start=2015-05-31&end=2015-07-12&_=1435356283321

- OK

  http://MYDOMAIN/index.php?title=Special:CargoExport&tables[0]=Events&tables[1]=Events&join+on[0]=&join+on[1]=&fields[0]=_pageName%2C_pageTitle%3Dname%2CStartDT&fields[1]=_pageName%2C_pageTitle%3Dname%2CStartDT&where[0]=EventType%3DExhibition&where[1]=EventType%3DMovie&group+by[0]=&group+by[1]=&order+by[0]=_pageName&order+by[1]=_pageName&limit[0]=100&limit[1]=100&format=fullcalendar&color[0]=green&color[1]=blue&start=2015-05-31&end=2015-07-12&_=1435356397824

- empty string.

Problem 3. Urlencoding of aliases.

I use following query with aliases in Russian:

  {{#cargo_query:
    |tables=Events,Places
    |join on=Events.Place=Places._pageTitle       |fields=CONCAT('[[',Events._pageName,'|',Events._pageTitle,']]')=Название,EventType=Рубрика,Region=Регион,Comuna=Коммуна,CONCAT(StartDT)=Начало
    |where=StartDT>'{{LOCALYEAR}}-{{LOCALMONTH}}-{{LOCALDAY}}'
    |order by=EventType,Начало
    |format=outline
    |outline fields=Рубрика
  }}

The query is broken, as aliases in clause ORDER BY are urlencoded. Workaround is simple - use backticks or apostrophs: `Начало` or 'Начало'.

But, first, MySQL and other databases permit field names in national languages (I use this feature rather often), second - urlencoding makes no sense in this place as the result will be broken 100%.

Once again - many thanks for this extension!!

I'm glad you like Cargo! And thank you for this detailed analysis. My responses:

1a - the real solution for this is to have Cargo itself be able to store basic page data like when a page was first created, and when it was last edited. That's something I'm planning to add in. Until then, any attempt to store that kind of data manually will be a hack, as you've discovered.

1b - Cargo needs a global variable to set whether to use 12-hour or 24-hour time; that has been a "todo" in the code for a long time. (It's not just English-speaking countries that use 12-hour time, but your larger point stands.)

1c - I have no idea why that's happening; do you?

2 - this was the most serious problem you found. I looked into it, and you're right - it was a Cargo bug. I introduced the bug about a month ago, while fixing something else; I think I just fixed this bug in Git.

3 - I'll have to look into that. Yaron Koren (talk) 00:43, 27 June 2015 (UTC)

Okay, I just checked in fixes for 1b and 3 as well. For 1b, I added in the new global variable $wgCargo24HourTime (default is false), which I have been meaning to do for a while. Yaron Koren (talk) 14:49, 29 June 2015 (UTC)

Dear Yaron, sorry for disappearance from the discussion - I have just returned to active programming and promise now to reply in time. :) I tested all changes made with Cargo 0.9.

Thanks for correcting bug 2. My calendar now shows all colors.

Bug 1b.

  ! Updated:
  | {{#cargo_query:
    |tables=Ads
    |fields=UpdatedDT
    |where=_pageName='{{FULLPAGENAME}}'
  }}

I see no effect from variable $wgCargo24HourTime. False or true, the result is always English 12-hour format. If I apply

fields=CONCAT(UpdatedDT)

- the result is always 24-hour format.

But in any case - I have workarounds for the rest. Alexey Rudenko (talk) 20:42, 23 August 2015 (UTC)

bar chart label not interpreted as wiki text

The bar-chart's labels shown as query's result are not interpreted as wiki text. I tried to encapsulate a value with "[[" and "]]" to create a link to a wiki page inside my bar chart ,

{{#cargo_query:
tables=MagDBCompetenceCollaborateur
|fields= CONCAT(CONCAT(CONCAT(CONCAT("[[",Competence), "|"),SUBSTRING_INDEX(Competence,'/',-1)),"]]")=Compétences,count(_pageName)=Nombre de collaborateurs
|group by=Competence
|where=Competence !="" AND SUBSTRING_INDEX(Competence,'/',1) LIKE "Expertise technique"
|order by=COUNT(*) DESC
|format=bar chart
}}

it doesn't work, it just show me the values inside the "[[" and "]]" as you can see :

any idea ? Guillaume Prêcheur (talk) 16:26, 30 June 2015 (UTC)

That's on purpose, actually... parsing of values could be done, but the big reason to do it, and the reason you have as well - to display links - is unfortunately not doable with the JavaScript library being used to display bar charts, NVD3. At least, it's not doable without some custom JS programming, which would (I assume) be complicated. By the way, CONCAT() can take more than two arguments, so you don't need to chain them like that - not that it matters in this case, though. Yaron Koren (talk) 19:30, 30 June 2015 (UTC)

thanks for the answer (an for the CONCAT lesson !) - Do you plan to integrate other kinds of graphic formats ?Guillaume Prêcheur (talk) 07:25, 1 July 2015 (UTC)

I don't have any specific plans - what did you have in mind? The obvious graphical format to add would be pie charts, but many of the data visualization people say that a bar chart is always better than a pie chart. Yaron Koren (talk) 13:40, 1 July 2015 (UTC)

Cargo 0.9 - outline fields

The request:

{{#cargo_query:
 |tables=Articles,Articles__ArticleTypes
 |fields=_pageName,Articles__ArticleTypes._value=Category
 |join on=Articles._ID=Articles__ArticleTypes._rowID
 |order by=Category,_pageName
 |limit=50
 |format=outline
 |outline fields=Category
}}

First page displays correctly. When I click on more results... link below, I get internal error:

[cf28a627] /index.php?title=Special:ViewData&tables=Articles%2CArticles__ArticleTypes&fields=_pageName%2CArticles__ArticleTypes._value%3DCategory&join_on=Articles._ID%3DArticles__ArticleTypes._rowID&order_by=Category%2C_pageName&format=outline&offset=50&limit=100&outline+fields=Category MWException from line 157 of /www/chileru.org/extensions/Cargo_0.9/formats/CargoOutlineFormat.php: Error: 'outline fields' parameter must be set for 'outline' format.
Backtrace:
#0 /www/chileru.org/extensions/Cargo_0.9/CargoQueryDisplayer.php(236): CargoOutlineFormat->display(array, array, array, array)
#1 /www/chileru.org/extensions/Cargo_0.9/specials/CargoViewData.php(240): CargoQueryDisplayer->displayQueryResults(CargoOutlineFormat, array)
#2 /www/chileru.org/includes/specialpage/QueryPage.php(563): ViewDataPage->outputResults(OutputPage, SkinVector, DatabaseMysqli, ResultWrapper, integer, integer)
#3 /www/chileru.org/extensions/Cargo_0.9/specials/CargoViewData.php(37): QueryPage->execute(NULL)
#4 /www/chileru.org/includes/specialpage/SpecialPage.php(384): CargoViewData->execute(NULL)
#5 /www/chileru.org/includes/specialpage/SpecialPageFactory.php(582): SpecialPage->run(NULL)
#6 /www/chileru.org/includes/MediaWiki.php(267): SpecialPageFactory::executePath(Title, RequestContext)
#7 /www/chileru.org/includes/MediaWiki.php(566): MediaWiki->performRequest()
#8 /www/chileru.org/includes/MediaWiki.php(414): MediaWiki->main()
#9 /www/chileru.org/index.php(41): MediaWiki->run()
#10 {main}

Supposing problem is in encoding of outline fields parameter as outline+fields. As a result this parameter is represented in CargoOutlineFormat->display as outline_fields with underscore.

I made dump of $displayParams in CargoOutlineFormat->display():

First page display:

Array
(
   [outline fields] => Category
)

Click on more results link:

Array
(
   [limit] => 100
   [outline_fields] => Category
   [offset] => 50
)

Alexey Rudenko (talk) 21:01, 23 August 2015 (UTC)

Sorry about that, and thanks for your work in diagnosing the error. I just checked in what I think is a fix. Yaron Koren (talk) 15:40, 24 August 2015 (UTC)

Thank you! I downloaded the latest development version 0.10-alpha (Cargo-master-0b334b2.tar.gz) - the bug is fixed. Alexey Rudenko (talk) 16:49, 29 August 2015 (UTC)

MW 1.25.2 / Cargo 0.10 alpha

I install from the scratch MW 1.25.2 and Cargo 0.10 alpha via Distributor (no other extensions).

Run update.php. Tables cargo_pages and cargo_tables are created.

After that any "normal" page (not special) throws fatal error:

Catchable fatal error: Argument 1 passed to CargoRecreateDataAction::displayTab2() must be an instance of SkinTemplate, string  given in /www/cargo.chileru.org/extensions/Cargo_0.10-alpha/CargoRecreateDataAction.php on line 97

The same happens in my working installation of MW 1.25.1, but I installed a new MW from the scratch to isolate the problem. What did I made wrong? Cargo extension is vitally important for my project, I would like to have its latest version. Thanks in advance! Alexey Rudenko (talk) 21:20, 23 August 2015 (UTC)

I just discovered this bug last week - it's actually a bug in MediaWiki 1.25-1.25.2, for extensions that use extension.json (like Cargo now does). It will be fixed in MediaWiki 1.25.3, but for now you can fix it yourself by putting in this change in your core MW code. Yaron Koren (talk) 02:37, 24 August 2015 (UTC)

I copied patched ExtensionProcessor_new-5523fe65955dbc14b87ad9dc0a08c3694fdd949d.php to includes/registration/ExtensionProcessor.php, ran update.php. Thank you, it works! Alexey Rudenko (talk) 13:20, 24 August 2015 (UTC)

MW 1.25.2 / Cargo any version

The SQL request includes MySQL function SUBSTRING_INDEX:

{{#cargo_query:
  |tables=Articles
  |fields=SUBSTRING_INDEX(Description, ',', 1), _pageName
}}

I get error:

Error: the SQL function "SUBSTRING_INDEX()" is not allowed.

If I replace SUBSTRING_INDEX() with SUBSTRING() - all works fine. No problems with SUBSTRING_INDEX() from other PHP scripts o command line.

I am 100% sure, that with MW 1.24 it worked correctly. Tested with Cargo 0.6, 0.9, 0.10-alpha on MW 1.25.2. Any suggestions? Alexey Rudenko (talk) 15:18, 9 September 2015 (UTC)

This is not related to the MediaWiki version, it's just the Cargo version. It's my fault - I added the $wgCargoAllowedSQLFunctions variable to the code over a month ago, but I haven't released a new version, so the change is not documented yet. (I doubt you saw this same error with Cargo 0.6.) I think all you need to do is add the following to LocalSettings.php, after the inclusion of Cargo:

$wgCargoAllowedSQLFunctions[] = 'SUBSTRING_INDEX';

Yaron Koren (talk) 15:37, 9 September 2015 (UTC)

Error when trying to create tables

When going to the tab "Create data table" and hitting "ok" nothing happens. My FireBug reports: "TypeError: templateData is null" (twice)

What I did to get here:

• got sources via git
• ran update.php
• database user with create/drop table rights (tested both: default mw-database settings and separate ones)
• copied quick-start templates book and author from Extension:Cargo/Quick_start_guide

with &debug=true

TypeError: templateData is null
https://HOST/_w/extensions/Cargo/libs/ext.cargo.recreatedata.js
Line 16
for ( var i = 0; i < templateData.length; i++ ) {
-----------------^

It doesn't seem to matter, if I create pages that call the templates author or book. Also, I tried to use the maintenance script to create the tables. It ran for 1 or 2 sec and then exited without any output. And of course: no tables were created... :(

BTW, I'm running a german MediaWiki with:

• MediaWiki 1.23.10
• PHP 5.6
• MySQL 5.5
• SemanticForms 3.4
• Cargo 0.10-alpha

and a few other minor extensions.

Also, setting

    error_reporting(E_ALL);
    ini_set ("display_errors", "1");
    $wgShowExceptionDetails = true;
    $wgShowDebug = true;
    $wgDevelopmentWarnings = false;

in my LocalSettings.php doesn't yield any more viable information. I'm at a loss and would appreciate any suggestion as to where to look next...

Regards,

---

Tobias Oetterer (talk) 14:06, 23 September 2015 (UTC)

kept me awake (so to speak) and I think I found the problem:

<div hidden="" id="recreateDataData" .. > was created but ext.cargo.recreatedata.js returned an empty object when trying to access it. So I supsected a failed race condition.

I then edited the section for 'ext.cargo.recreatedata' in $wgResourceModules in Cargo.php and changed position to bottom:

'position' => 'bottom',

Now it seems to work fine. No idea, if this breaks something else in the long run, though...

---- Tobias Oetterer (talk) 17:19, 23 September 2015 (UTC)

That's a great find! I don't know why that happened for you, but I'll make that change in the code. Yaron Koren (talk) 14:06, 24 September 2015 (UTC)

Cargo Template

Hi I'm trying to create a single template for a table. Looking at the example how would I add the query for the single table? I did create a template whether right or wrong and selected "create data table" came up with Create Cargo data for this template? I selected ok and nothing happened.

I can't find anything on the form layout is that the same as SMW?

If you go to Special:CargoTables, do you see the table? For the forms - the Semantic Forms documentation covers all of that, including the Cargo-specific stuff. Yaron Koren (talk) 14:17, 24 September 2015 (UTC)

nothing there. Phil Legault 26 September 2015

Enabled debugging and I don't see any errors

here is my template

<noinclude>
This is the for FEA Domain

{{#cargo_declare:_table=recommend|
recommendation=Text}}
</noinclude>
<includeonly>
{{#cargo_store:_table=recommend|
recommendation={{{recommendation|}}} }}
{{{recommendation|}}}
</includeonly>

That certainly looks like it should work... can you try calling the command-line script cargoRecreateData.php and see if that works, or if you get an error message? Yaron Koren (talk) 02:46, 27 September 2015 (UTC)

Problems with creation of Cargo tables

Hi Yaron,

I use newly installed MediaWiki 1.25.2 with extensions:

• Cargo (0.10-alpha),
• ParserFunctions (1.6.0),
• LocalisationUpdate (1.3.0).

I created Template:Book, but I can't create the table of data, because the page (index.php?title=Template:Book&action=recreatedata) has the javascript error:

SyntaxError: JSON.parse: unexpected character at line 1 column 1 of the JSON data
http://cargoen.loc/load.php?debug=false&lang=en&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=20150930T154529Z
Line 125

and when I click "OK" button, nothing happens.

From the command line I can't create the table of data, too. Nothing happens.

Please help solve the problem and create a table.

I'm guessing that the API is returning an error message. I would try going directly to "api.php?action=cargorecreatetables&template=templateNameHere" (that's the URL being retrieved behind the scenes) and seeing what it displays. Yaron Koren (talk) 16:59, 30 September 2015 (UTC)

Thanks a lot! I executed API request for creation of the table and it worked - the table was successfully created: "This is the HTML representation of the JSON format. HTML is good for debugging, but is unsuitable for application use. Specify the format parameter to change the output format. To see the non-HTML representation of the JSON format, set format=json. See the complete documentation, or the API help for more information. {"success": ""}.
What to do with the javascript error? Perhaps I should write more detailed report about an error? Sergey Mikhalev 18:27, 01 October 2015 (UTC+3)

That's unexpected. There are actually two URLs being accessed - maybe the second one is causing the problem? The second one is "api.php?action=cargorecreatedata&template=templateNameHere&table=tableNameHere". Could you try going to that one too? Yaron Koren (talk) 15:49, 1 October 2015 (UTC)

I executed API request for recreation of the table (http://cargoen.loc/api.php?action=cargorecreatedata&template=Book&table=Books) and it worked - the table was successfully recreated: "This is the HTML representation of the JSON format. HTML is good for debugging, but is unsuitable for application use. Specify the format parameter to change the output format. To see the non-HTML representation of the JSON format, set format=json. See the complete documentation, or the API help for more information. {"success": ""}. Sergey Mikhalev 09:22, 02 October 2015 (UTC+3)

Alright, so the API is not the problem. If the web interface still doesn't work for you, I would go the "recreate data" page and add "&debug=true" to the URL after "?action=recreatedata". That should lead to a more helpful JS error message. Yaron Koren (talk) 10:47, 2 October 2015 (UTC)

I executed API request for recreation of the table with the "&debug=true" parameter (http://cargoen.loc/index.php?title=Template:Book&action=recreatedata&debug=true):

SyntaxError: JSON.parse: unexpected character at line 1 column 1 of the JSON data
http://cargoen.loc/load.php?debug=true&lang=en&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=20150930T154529Z
Line 8520

In a debug mode:

		jQuery.parseJSON = function( data ) {
			// Attempt to parse using the native JSON parser first
			if ( window.JSON && window.JSON.parse ) {
				// Support: Android 2.3
				// Workaround failure to string-cast null input
8520			return window.JSON.parse( data + "" ); 
			}

			var requireNonComma,

"Data" variable is undefined before throwing an exception. Also when using "&debug=true" parameter there are 2 more errors:

TypeError: $content.find(...).makeCollapsible is not a function
http://cargoen.loc/resources/src/mediawiki.page/mediawiki.page.ready.js
Line 23

TypeError: $(...).on is not a function
http://cargoen.loc/resources/src/jquery/jquery.mw-jump.js
Line 6

Sergey Mikhalev 16:22, 05 October 2015 (UTC+3)

Okay, that's only somewhat helpful. Are you using the very latest version of the Cargo code? I just made a change last week, that moved the JavaScript for "recreate data" from the top to the bottom of the page, that may fix this issue for you, if you're not using that code yet. Yaron Koren (talk) 14:42, 5 October 2015 (UTC)

I updated the code and everything works. Thank you very much! Sergey Mikhalev 10:15, 06 October 2015 (UTC+3)

(Phew) That's good to hear! I should have thought of that earlier. Yaron Koren (talk) 14:38, 6 October 2015 (UTC)

BLOB or TEXT columns

Hey, I will soon face the task of storing quite a bit of text (>> 300 chars) in a field. Currently. I'm working with cargo and I'm really excited about the capabilities. Somehow, at the the design level, I assumed that fieldtype "Wikitext" would result in a TEXT or BLOB column. Unfortunately, it also results in varchar(300). My questions:

• Is there a way, I can increase the limit of 300?
• Is there a way to have #cargo_declare create a column of type BLOB or TEXT?
• Is any of this a permanent restriction or do you plan to add such capabilities (possibly in the near future?)?

And the bonus question:

• Do you have any idea, how cargo would react, if I manually changed the column type via phpmyadmin?

Best wishes

---

Tobias (talk) 06:02, 16 October 2015 (UTC)

You can change the size by adding "(size=..)" after the type in the #cargo_declare call. And, for recent versions of Cargo, if the size is greater than 1000, the field will be saved as type "TEXT". So I would recommend declaring the type as "Wikitext (size=2000)". It's kind of a hack, I know; and maybe the BLOB/TEXT type should really just be directly settable.

I don't think changing the type would cause problems, though your changes would be overwritten if you ever refreshed the table. Thankfully, there's probably no need for this. Yaron Koren (talk) 20:42, 16 October 2015 (UTC)

Awesome, thanks. You are the best! The size= parameter must have slipped by me, sorry. Nevertheless, works perfectly.

imho, giving the user the option to set a field type to text/blob would be better. It's straight forward and the user knows what to expect. Maybe simple introduce another field type "blob" or something like that.

Cool. That's not a bad idea - I somewhat wish I had used "String" for smaller strings, so I could have "Text" for the TEXT type, the way Semantic MediaWiki used to do it. Maybe "Longtext" would work... "Blob" might be confusing, because it wouldn't actually be the SQL BLOB type. Yaron Koren (talk) 01:30, 22 October 2015 (UTC)

I'm a stricly mysql kinda guy, so you have to forgive my inexperience with other databases. :/ Longtext however sounds perfect! :)

I favor MySQL too. :) I ended up adding a "String" type, as I should have done originally - so "Text" now always sets a DB field of type "TEXT", if you get the latest code. Yaron Koren (talk) 00:38, 9 November 2015 (UTC)

Cargo and Lua

I needed to use cargo_query in Lua module. It is clear that the ideal would be to get the result as a table Lua. But cargo_query not provide even a simple text output (csv?). We have to use a list (uncomfortable parsing) or template (very expensive). --StasR (talk) 06:32, 17 October 2015 (UTC)

cargo_query provides a link to a CSV output, which is directly available at Special:CargoExport. If you have the code create the Special:CargoExport URL directly (which shouldn't be that hard to do), it will hopefully work. Yaron Koren (talk) 23:40, 17 October 2015 (UTC)

Thank you. Intuitively it seems to me a http request even less convenient and efficient than 'list' parsing. Maybe slight improvements 'list' to turn it into a machine-processed?

The main difficulties are caused by the definition of the boundaries of records (field boundaries can be recognized by text <span class="cargoFieldName"> without loss of generality). Furthermore, it is inconvenient that the first field in the record is displayed in a special way, but it is less important and problem can be solved. --StasR (talk) 11:47, 19 October 2015 (UTC)

What are you ultimately trying to do? Maybe there's an easier way to do whatever it is. Yaron Koren (talk) 14:03, 19 October 2015 (UTC)

I need to process each record via Lua. The obvious way is to put the format of 'template' and make a template that contains only #invoke. It is technically possible, but it seems too expensive.

As long as I use a template that contains #invoke:...|fields for each record. And largely because of the excessive number of calls I want to use Cargo. --StasR (talk) 17:56, 19 October 2015 (UTC)

Alright. This might be a hack, but you could potentially use the External Data extension, and specifically its parser functions #get_web_data and #for_external_table, to get the data from Special:CargoExport to Lua in the necessary format. It wouldn't require either parsing or an HTTP request, for what it's worth. Yaron Koren (talk) 18:34, 19 October 2015 (UTC)

I, too, had to choose between External Data and the format=template "hack". As of now, I'm running the latter but I haven't done any excessive performance tests, yet. Basically, I use a template, that again calls a lua function that encases the arguments in some predefined strings so that I can easily construct a table from the return string. After some minor problems, it runs quite well now. If you're interested, I could send you my solution... ---- Tobias (talk) 18:13, 21 October 2015 (UTC)

Yaron Koren, why not make displaying format without strip markers? --StasR (talk) 08:36, 22 October 2015 (UTC)

What's a strip marker? Yaron Koren (talk) 13:00, 22 October 2015 (UTC)

Strip marker. CargoQuery.php, line ~130:

                if ( $displayHTML ) {             # i. e. not 'template' format --stasR
                        return $parser->insertStripItem( $text, $parser->mStripState );
                } else {                          # 'template' format --stasR
                        return array( $text, 'noparse' => false );
                }

--StasR (talk) 14:23, 22 October 2015 (UTC)

Oh, okay. What's the problem with it? Yaron Koren (talk) 14:41, 22 October 2015 (UTC)

The problem is still the same. Result without strip markers can be analyzed by Lua. --StasR (talk) 14:55, 22 October 2015 (UTC)

The strip thing is there so that HTML can be displayed correctly. Why not use the External Data approach? Yaron Koren (talk) 17:27, 22 October 2015 (UTC)

Best (and most performant) approach would be to create a Lua package for Cargo. Otherwise take external data or the template hack. All works fine... ---- Tobias (talk) 20:17, 22 October 2015 (UTC)

I agree. Lua package is the best solution. But first, it would be enough to analogue #for_external_table in Cargo. --StasR (talk) 11:56, 24 October 2015 (UTC)

I too agree that a Lua package would be great (and I wish I knew how to create them). But, StasR - why not just use #for_external_table directly? Or, maybe better yet, #display_external_table? Yaron Koren (talk) 03:12, 26 October 2015 (UTC)

I was hoping #for_external_table (unlike #display_external_table) alone does not cause the MW parser to interpret each row. I was wrong? --StasR (talk) 09:20, 26 October 2015 (UTC)

I don't know; I would try it. Yaron Koren (talk) 13:11, 26 October 2015 (UTC)

Yaron, I did Cargo format similar #for_external_table. It was easy, because the code is well structured. Can I offer it to you? --StasR (talk) 22:31, 27 October 2015 (UTC)

Sure; I don't really understand what that means, but I'm looking forward to seeing it. Yaron Koren (talk) 23:31, 27 October 2015 (UTC)

Please take into account that I do not do coding for many years :-). New file CargoPatternFormat.php (I designed it from CargoListFormat.php) contains:

    // ...
    function allowedParameters() {
        return array( 'pattern' );
    }
    // ...
    function display( $valuesTable, $formattedValuesTable, $fieldDescriptions, $displayParams ) {
        $text = '';
        if ( array_key_exists( 'pattern', $displayParams ) ) {
            $pattern = $displayParams['pattern'];
        } else {
            $pattern = 'Pattern not defined! ';
        }
        foreach ( $formattedValuesTable as $i => $row ) {
            $text .= preg_replace_callback ( 
                    '|\{\{\{([^}]+)\}\}\}|',
                    function ( $matches ) use ( $row ) {
                        if ( array_key_exists( $matches[1], $row ) ) {
                            return $row[ $matches[1] ];
                        } else {
                            return '(' . $matches[0] . '?)';
                        }
                    },
                    $pattern
                );
        }
        return $text;
    }

and the condition at the end of CargoQuery.php looks like this

        if ( $format == 'pattern' ) {
            return array( $text, 'noparse' => true );
        } elseif ( $displayHTML ) {
            return $parser->insertStripItem( $text, $parser->mStripState );
        } else {
            return array( $text, 'noparse' => false );
        }

Furthermore, obvious changes made in Cargo.php and CargoQueryDisplayer.php. Not made by me: (i) I have not studied the situation with the "View more"; (ii) The same should be the end CargoCompoundQuery.php, but I did not engaged. --StasR (talk) 09:12, 28 October 2015 (UTC)

What's an example of a pattern that this would be called with? Yaron Koren (talk) 23:36, 28 October 2015 (UTC)

Example:

{{#cargo_query: tables=Cities
  | fields=CONCAT(_pageName)=City,Population
  | format=pattern
  | pattern={{{City}}}'s population is {{{Population}}}.<br />
}}

But for Lua parsing I would have written something like this pattern="\127{{{City}}}\127{{{Population}}}\127" --StasR (talk) 07:09, 29 October 2015 (UTC)

Ah... now I understand. And now I know what you meant by "similar to #for_external_table". I'll look into this - maybe there's an easier way to do this, using the "template" format, but maybe not. Yaron Koren (talk) 09:03, 29 October 2015 (UTC)