Extension talk:Cargo/Archive May to June 2021

Streamlining uploading images with cargo metadata? (Single step form?)

I have a similar query to that asked by Sparr in Streamlining uploading images with cargo metadata - following the advice I created a page Project:File containing a #default_form definition, and all File pages now have an 'edit with form' tab which prompts for cargo metadata; completing ths form associates the page with the right category / template etc. and data can be queried.

But is there a way to make it a single-step process - either to do the upload on the same page as the Form, or automatically progress from one to the other? I note that Page Forms defines a special page Special:UploadWindow but I cannot find examples how to use it. Thanks, --Jaydublu (talk) 10:26, 13 June 2021 (UTC)

[BUG?] Recategorisation and _pageData table

I have a feeling that when the Replace Text extension has been used, the pages disappear from the _pageData table, or at least the pages don't show up on Cargo queries on that table. I haven't tested this out though, but can do if you think it would be useful. Jonathan3 (talk) 08:48, 1 May 2021 (UTC)

I've had a think again. I was using Replace Text to recategorise pages. I then recategorised some remaining pages manually (by editing the page) which had the same effect. I don't think it's to do with the Replace Text extension after all.

I think what's happening is this. Page X is in categories A, B, C and D. I replace [[Category:A]] with [[Category:E]]. Cargo now realises that page X is in category E, but has forgotten that it's also in categories B, C and D. _pageData._categories only contains "E". I am 99% sure of this from what I've seen of the effects of recategorisation, though I've not yet tried to reproduce it deliberately. Jonathan3 (talk) 09:15, 1 May 2021 (UTC)

A further observation. Page X again is in categories A, B, C and D. If I remove [[Category:A]] (or any category) then Cargo will come to believe that the page is in no category at all. Jonathan3 (talk) 09:39, 1 May 2021 (UTC)

It's true that categories are handled differently from all the other fields within _pageData - that's because MediaWiki does category setting via jobs, so it doesn't happen instantaneously on page save. I don't know what's happening with this particular bug, though. Does it only happen when the page is modified via Replace Text? Yaron Koren (talk) 03:55, 3 May 2021 (UTC)

I've had a look using the query below, and changing the categories to see what it says.

{{#cargo_query:
table=_pageData
|fields=_categories
|where=_pageName="{{PAGENAME}}"
|intro=Cargo says my categories are: 
}}
[[Category:A]]
[[Category:B]]
[[Category:C]]
__NOCACHE__

When the job queue is empty, it works fine. When I add/remove/edit a category, the query prints the correct sentence as soon as the page is saved.

When the job queue is not empty (e.g. when I edit a template) it does not work but, instead, does as described above.

Then, even when the job queue is empty again after running runJobs.php, it still doesn't "catch up" and show the correct categories, even when the page is purged. At that stage, I need to recreate the _pageData table, or (unrealistically) remove the category definitions, save the page, add them back to the page, and save the page again.

I don't think Replace Text is relevant except that it fills up the job queue, which in turn stops Cargo from working properly. Jonathan3 (talk) 06:54, 3 May 2021 (UTC)

I've had a think about this. The following code in CargoHooks.php, function addOrRemoveCategoryData, called by the CategoryAfterPageAdded or CategoryAfterPageRemoved hooks, is trying to get the page's existing categories from the _pageData table:

		$cdb = CargoUtils::getDB();
		$cdb->begin();
		$res = $cdb->select( $pageDataTable, '_ID', [ '_pageID' => $pageID ] );
		if ( $cdb->numRows( $res ) == 0 ) {
			$cdb->commit();
			return true;
		}
		$row = $res->fetchRow();
		$rowID = $row['_ID'];
		$categoriesForPage = [];
		$res2 = $cdb->select( $categoriesTable, '_value',  [ '_rowID' => $rowID ] );
		foreach ( $res2 as $row2 ) {
			$categoriesForPage[] = $row2->_value;
		}

When the job queue is not running, it works fine, and returns an array of the page's categories from the _pageData table. The next parts of the code either remove or add the new category to that array.

When the job queue is running, it returns an empty array even when the _pageData table contains the correct categories information. This in turn leads to the problems described above: (1) when you add a category, it ends up being the only category in the Cargo table; and (2) when you remove a category, there end up being no category in the Cargo table.

Maybe I should add that I have $wgJobRunRate = 0 and run the job queue as a cron job and/or at the command line. The problem seems to appear when runJobs.php is running (not merely when there are jobs in the queue). Jonathan3 (talk) 22:11, 7 May 2021 (UTC)

I think maybe it's getting the wrong $rowID (i.e. one which doesn't yet exist in the _pageData table). Jonathan3 (talk) 22:40, 7 May 2021 (UTC)

OK here is my last guess for now. I'm filling the job queue by making a minor edit to a Cargo template. Maybe while the job queue is running, the _ID in _pageData for the pages (or at least the page in question) changes... when the code above wrongly gets an empty array the _ID has increased by 2... this'll be why when the code above looks up the _ID there is nothing in _pageData__categories... if you look in the database, the old orphaned _rowID rows remain in _pageData__categories. Maybe the answer is to work on a replica database for both queries, or turn them into a single query using a join, then when updating use the real database having checked for any new row ID. Or don't let the the _ID in _pageData for the pages ever change. Jonathan3 (talk) 22:59, 7 May 2021 (UTC)

Having slept on it, it's clear that it's getting the correct (new) row id for the page data table but it's not matching up with the same row id in the categories full table (which still has the old row id).

There's a function that stores page data but ignores categories. I think it deletes the page data table row before recreating it. Maybe this is when the row id changes? Maybe instead of ignoring categories it should at least ensure the categories full row id matches the page data row id? Jonathan3 (talk) 07:41, 8 May 2021 (UTC)

Maybe the _rowID of _pageData___categories should be the page's ID (which stays the same even when a page is moved) instead of being _pageData's _ID field (which seems to change sometimes). Otherwise you'd need to keep them in sync somehow in the Cargo code. Jonathan3 (talk) 15:11, 8 May 2021 (UTC)

Stream of consciousness collapsed......................................................................

There's a lot to go over here. Could it be that the main _pageData storage code (called by the PageSaveComplete hook) is being called after the category info storage code (called by the two hooks you listed before)? If so, that would definitely explain what you're seeing - and if that's the case, I can't think of any way around it at the moment. Yaron Koren (talk) 18:03, 12 May 2021 (UTC)

More stream of consciousness...

Might it be the other way round? Basically addOrRemoveCategoryData checks a page's _pageData._ID row and associated _pageData__categories._rowID rows - but sometimes (caused by the job queue running) by the time it does that the page's _pageData._ID has changed and the _pageData__categories._rowID has not. So it gets the correct, new _pageData._ID but can't use it to obtain anything from _pageData__categories because _ID is no longer equal to _rowID. It then bases the add/remove on an empty array (or once, I think _ID coincided with an orphaned _rowID with category rows, so instead of an empty array it was just completely wrong).

Possible alternative solutions might be:

• Use _pageID as the link between _pageData and the _pageData__categories rows (e.g. either set _ID=_pageID and _rowID=_pageID, or just set _rowID=_pageID and change the code) as _pageID (I think) rarely/never changes.
• When onPageSaveComplete deletes the _pageData row it should either:
  • keep a record of the _ID/_rowID and send it to storeValuesForPage so that if _ID changes so does _rowID and the _pageData__categories rows therefore continue to be linked with the newly-created _pageData row.
  • delete the _pageData__categories rows too, and storeValuesForPage/storeAllData should recreate those rows from the MW database.

Jonathan3 (talk) 21:47, 12 May 2021 (UTC)

That seems implausible to me, though maybe I haven't thought about it enough. I still like my theory - do you think it could be right? Yaron Koren (talk) 02:13, 13 May 2021 (UTC)

I'm not sure. Which is the implausible part? I don't know what in the job queue changes the _ID (could be something odd about my wiki) but I did some debugging and the logs show the other stuff happening (changed _ID and unchanged _rowID wrongly getting empty array). I gave up when I couldn't fix it so wasn't 100% thorough though. Jonathan3 (talk) 06:34, 13 May 2021 (UTC)

Duplicate rows again

Did we ever get to the bottom of what causes duplicate rows?

I've just run the following query:

{{#cargo_query:
table=[table name]
|fields=_pageName, COUNT(_pageName)
|having=COUNT(_pageName) > 1
|group by=_pageName
|order by= COUNT(_pageName) DESC
}}

It showed four pages which had two rows each. Doing a null edit (edit and save with no change) gets rid of a duplicate row. Purging the page does not.

It would be good if Special:CargoTables would identify whether a table has duplicate rows.

Alternatively, can you suggest an improved Cargo query that identify duplicate rows across various tables on a wiki? Maybe using Scribunto (which I downloaded this week...)

Maybe I should do it with plain PHP and get it to email me when it finds duplicates.

Or maybe the answer is to run the various recreate table scripts nightly?

I'm using MW1.34 and Cargo code from about a week ago.

Thanks. Jonathan3 (talk) 21:52, 4 May 2021 (UTC)

I've put that query (modified to make it look nicer) into a template, and notice that in each table, the pages with duplicate rows are mainly pages I've not even looked at for ages, and also that there are never more than two identical rows. Jonathan3 (talk) 22:00, 4 May 2021 (UTC)

To contradict something above, a file I uploaded tonight appears on the list now... every time I edit the File: page the count (of rows) increases by one... Jonathan3 (talk) 22:51, 6 May 2021 (UTC)

Do you think maybe the duplicate rows thing could be linked with the recategorisation problem? E.g. a List's __fieldname rows get orphaned when their _rowID no longer is the _ID of its main table, so a proposed new main table row has an incorrect fieldname__full field which means that the "is this a duplicate row?" check wrongly thinks it's different. I've recreated all my tables recently so can't check... Jonathan3 (talk) 20:29, 8 May 2021 (UTC)

Disable storing on User namespace

Hello again!

So, as the header suggests, I'm trying to find a way to disable cargo_declare and cargo_store on the User namespace, is that possible? Currently I have some very important templates with a lot of data, and would be very awkward to query the pages and randomly there's rows with junk data.

I think it's possible to disallow certain templates to be rendered with AbuseFilter, but I'm not sure if there's another way or if that will work for most of the time. Unfortunately, I can't guarantee that all users will respect the rules by just throwing an alert.

Thanks! Lakelimbo (talk) 20:02, 5 May 2021 (UTC)

I don't think there's a way to automatically disable #cargo_store for certain namespaces... I suppose you could use #if and the {{NAMESPACE}} variable to only call #cargo_store for certain namespaces. Or you could check the namespace in the query, like "where=_pageNamespace != 2". Yaron Koren (talk) 20:15, 5 May 2021 (UTC)

I see. Could you take this as a suggestion, then? Would be awesome if we could just simply turn off any storage of data on specific namespaces. Thank you! Lakelimbo (talk) 01:54, 6 May 2021 (UTC)

Why are people putting these templates in the User namespace (presumably, in their own user page, or a subpage of it) if they're not supposed to? Is it for testing purposes, or do they just not know? My concern with shutting off certain namespaces entirely is that the usefulness of it might be limited - since there could be certain templates that you do want people to put in user pages, or in other namespaces. (Actually, there's probably more of an argument for being able to shut off storage in other namespaces, like "Talk", where there's no real reason to store data.) Yaron Koren (talk) 17:06, 6 May 2021 (UTC)

I don't know why, but it does happen, however I imagined like it being possible to disable within the #cargo_store itself, not as a "global parameter" (something like |ignore namespace=User, Talk, Help...). But I do understand why this may raise concerns. In any way, thanks again. Lakelimbo (talk) 12:47, 7 May 2021 (UTC)

That's an idea also. Actually, the more I think about it, the more I think a global variable like $wgCargoDisabledNamespaces (or $wgCargoEnabledNamespaces) could make sense, precisely for all those non-content namespaces. Or would only a parameter like "ignore namespace" work for this case? Yaron Koren (talk) 18:15, 7 May 2021 (UTC)

On my case specifically, I think just "ignore namespace" would work, but making something like $wgCargoDisabledNamespaces would be very useful I suppose, especially for wikis that heavily rely on Cargo. And sorry for not responding yesterday, I was a bit busy. Lakelimbo (talk) 20:41, 9 May 2021 (UTC)

Hey Lakelimbo if they're putting a template there and not a direct call to cargo_store, inside of the template you can do either {{#ifeq:{{NAMESPACE}}|USER||{{#cargo_store:.... or even more generaly {{#if:{{NAMESPACE}}||{{#cargo_store.... I do this on my wiki for every single template with a cargo store, with a list of explicitly allowed namespaces (in Lua not wikicode but same idea). This allows sandboxing without any problematic stores, and you can still have some other stores in the User namespace if you needed. Though if they're writing cargo_store directly it would still be a problem. But, AbuseFilter can also prohibit saving completely, instead of just a warning, if you wanted to try that. --RheingoldRiver (talk) 02:56, 31 May 2021 (UTC)

[SOLVED] Date display

I'm wrestling with date displays in infoboxes using Cargo dates. The Page Values page displays the date as I intend (January 1, 1883) but the infobox insists on displaying 1883-1-1. When I try to format the date within the infobox (using MW date formatting), I can get full dates to work, but partial dates (for instance only the year) insert TODAY'S date info in the missing sub-fields. That's no good. Is there a date formatting procedure that I'm missing? Here's an example: http://wiki.martenet.com/index.php/Pratt_Library_Branch_4 You can see the page values link displays the full month and year (I have the $wgAmericanDates set to "true"). But the infobox is listening to something else, apparently. --Parma100 (talk) 00:37, 12 May 2021 (UTC)

You could check the __precision field for the date: Extension:Cargo/Storing data#Storage of dates.

Having said that, here's what I did within a date-displaying template:

{{#ifeq:{{#len:{{{value|}}}}}|4|
  {{#time:Y|{{{value|}}}}}|
  {{#if:{{#pos:{{{value|}}}|/}}|
    {{#time:j/n/y|{{{value|}}}}}|
    {{#time:F Y|{{{value|}}}}}
  }}<!--end of if-->
}}<!--end of ifeq-->

Jonathan3 (talk) 06:56, 12 May 2021 (UTC)

Thanks for that. I think it would be more precise to depend on the __precision field in this date template. However, I can't seem to find how to retrieve that field within a query template. I see on a 2015 thread where Yoren says "I just added in (re-added, really) the ability to display date precision fields in the query. Is that by itself enough to get this working, though? Or would you also need the ability to call "CASE", or something like it, in the "fields=" parameter?" , but I don't see how to incorporate that. The internal date translations work well enough for me--I just can't find how to grab the __precision field to make the magic happen. What am I missing? --Parma100 (talk) 14:14, 13 May 2021 (UTC)

If your date field is called Startdate you'd use Startdate__precision, I think. Jonathan3 (talk) 14:20, 13 May 2021 (UTC)

That's what I figured. I seem to be getting null values, and that makes me think the fetch from the database isn't occurring. Those "hidden" fields (coord lat & long, this precision value, etc) aren't mentioned in the actual fields listing, and I'm not able to get any values, so I'm wondering if they're not being fetched. I've checked the actual MySql db, and the values are there, so storage is working. I can't figure out the retrieval. --Parma100 (talk) 14:31, 13 May 2021 (UTC)

It works now, with no processing at all, although I don't know why. It has something to do with being edited in the form. If I modify the (correct) date in the form, then save and go back to the page, the correct format appears -- month and year only, if no day is specified, and year only, if that is what is specified. I'm not going to argue with success, but I don't know why that edit vehicle works while the original form input does not. And this is without any of the processing you kindly shared above. Go figure.--Parma100 (talk) 20:16, 13 May 2021 (UTC)

Maybe an upgrade in Cargo since you last saved the pages? I've just done a quick query on a table with date fields and the dates all show properly, according to their precision... I had expected the missing parts to be filled with 1s (e.g. 2000 to show as something like 2000-01-01, May 2000 to be 2000-05-01). What happens if you just do a null edit on the page, or edit something unrelated? Maybe if you recreate the table it'll sort it out for every page. Jonathan3 (talk) 21:35, 13 May 2021 (UTC)

Actually I think it is due to the evolving nature of my coding here. Early on I used Cargo, but not Page Forms (because I didn't understand what Page Forms might be doing behind the scenes) and just used the editor for infoboxes inside the VisualEditor. That, of course will allow you to change the text, and Cargo picked up those changes. All good. But I don't think the secondary fields like __precision were getting updated as well. When I checked the MySql table, I saw lots of NULLS, but not all, in the __precision fields. I (incorrectly, now, I think) supposed that everything was getting saved properly. Later, I incorporated the Page Forms editor, abandoning the VisualEditor editing for infoboxes. Yesterday I stumbled on the fact that editing one of the "wrong" entries in Page Forms, and then saving it, automatically corrected the display. I went through and changed all of the incorrect displayed entries, and now they're all good. So I think the silver bullet is editing them in Page Forms, not using VisualEditor. This Cargo package is a well-thought-out suite of programs, and got me close to the finish line with a minimum of effort. It deserves high praise. The biggest problem may be the "nut behind the wheel!" --Parma100 (talk) 13:26, 14 May 2021 (UTC)

It's great that you got it working! Although it should work the same whether the template calls are being edited with Page Forms or VisualEditor (or by hand); so I don't know what the problem was here. Yaron Koren (talk) 18:09, 19 May 2021 (UTC)

[SOLVED] Compound, compound queries

Is it possible to get a compound, compound query without resorting to SQL? I'd like to fetch records which satisfy Criteria A AND either Criteria B OR Criteria C. I thought I could wrap the B or C code in parenthesis, as below,

{{#cargo_compound_query:
tables=buildings;where=building_type='Library' AND (architect='{{PAGENAME}}' OR architectural_firm='{{PAGENAME}}');fields=_pageName,Geo;icon=Blue-marker.png
|format=leaflet
|height= 600
|width= 1000
}}

but that doesn't fly. The parenthesis trigger an error. What is the method of combining these tests? --Parma100 (talk) 19:08, 18 May 2021 (UTC)

This probably doesn't help you but I've used the above to create materially the same query with a table of mine and it worked all right (I changed the ='{{PAGENAME parts to <> just to get a result on a test page). Maybe something else is causing the error. It's not really a compound query without the second tables= line but I guess that's next. Jonathan3 (talk) 23:05, 18 May 2021 (UTC)

Thanks for the hint. Apparently more "nut behind the wheel" stuff. I'd labeled one of my fields incorrectly. The code does work with the parenthesis. --Parma100 (talk) 16:33, 19 May 2021 (UTC)

Maintenance script to switch in Cargo table

To work round the duplicates problem, I'm going to recreate all the Cargo tables each night.

It would be good if I could use the --replacement parameter then, once the existing maintenance scripts have run, use another script to switch in the newly created replacement tables.

In the meantime I'll just choose an "Ungodly hour" to recreate the tables and hope nobody is using the website then :-) Jonathan3 (talk) 22:10, 24 May 2021 (UTC)

Annoyingly, after not using the --replacement parameter, for the first time there are duplicates pretty much straight after running the scripts :-( Usually I use that parameter and the recreation scripts clear the duplicates. Jonathan3 (talk) 22:48, 24 May 2021 (UTC)

...or how about a new parameter --switch which means the script creates a replacement table and at the end switches it in? It would mean that the Cargo tables are only incomplete for a split second and save the bother of using the web interface and all those clicks to switch the tables in :-) Jonathan3 (talk) 17:58, 25 May 2021 (UTC) ... and might avoid the duplicates problem - switching in a replacement table seems to avoid the problem - maybe because the job queue doesn't interfere with it? Jonathan3 (talk) 09:46, 30 May 2021 (UTC)

_pageData and Extension:UserMerge

When UserMerge deletes a User: page, the _pageData details aren't updated fully (unsurprisingly I suppose). I did a query out of curiosity and the deleted page appeared at the top of the list, probably because it is still in the Cargo table but doesn't have a date field any more.

{{#cargo_query:
table=_pageData
|fields=_pageName, _creationDate
|order by=_creationDate ASC
|limit=10
|format=ul
}}

Thanks. Jonathan3 (talk) 20:22, 25 May 2021 (UTC)

P.S. This resolves itself when the _pageData table is recreated. Jonathan3 (talk) 09:01, 28 May 2021 (UTC)

[SOLVED] Cargo and Extension:Popups

I'd like to use Extension:Popups, but most of my pages these days are formed from Cargo template queries, and Extension:TextExtracts, which gets the page extract for the popup, returns "..." (nothing plus a standard "..."). It would be great if somehow the popup text extract could be based a Cargo field - so when it extracts text from a page, it checks whether there's a Cargo template, and if so returns the contents of the predetermined field for that template. I think this may be a relevant link: Extension:Popups#Page_previews_API. Is this something you'd consider adding? Thanks. Jonathan3 (talk) 08:59, 28 May 2021 (UTC)

It works fine for me now after I got rid of "div" within "ExtractsRemoveClasses" in Extension:TextExtracts's extension.json file. Jonathan3 (talk) 15:27, 1 June 2021 (UTC)

[SOLVED] Cargo and Scribunto/Lua

For my first Lua script I'd like to display the contents of a Cargo table in the following format (where Name and About are fields):

Name

• About
• About

Name

• About

Name

• About
• About
• About

I imagine something similar has been done before so wonder if anyone can point me to anything. Thanks in anticipation :-) Jonathan3 (talk) 06:33, 29 May 2021 (UTC)

I've made a start by modifying the example on Extension:Cargo/Other_features#Lua_support but it seems only to be returning the first row:

local p = {}
local cargo = mw.ext.cargo

function p.Main( frame )
    local tables = 'Mytablename'
    local fields = '_pageName'
    local args = {
    }
    local results = cargo.query( tables, fields, args )
    for r = 1, #results do
        local result = results[r]
        return r .. '... '
    end
end

return p

It prints "1..." but I'd have expected "1... 2... 3... 4... " etc. Where am I going wrong? Thanks. Jonathan3 (talk) 07:10, 29 May 2021 (UTC)

I don't usually ping people but User:RheingoldRiver I heard you on Yaron's podcast and wonder whether you would be willing to share your expertise here :-) Jonathan3 (talk) 07:53, 29 May 2021 (UTC)

Got the following code to work:

local p = {}
local cargo = mw.ext.cargo

function p.Main( frame )
    local tables = 'People'
    local fields = 'Name, About'
    local args = {
    	orderBy = 'Name',
    	limit = '200'
    }
    local results = cargo.query( tables, fields, args )
    local output = ''
	local oldname = ''
    for r = 1, #results do
        local result = results[r]
        local name = result['Name']
        local about = result['About']
     
		if name ~= oldname then
			output = output .. '\n<b>' .. name .. '</b>'
		end
        oldname = name
        
        output = output .. '\n*' .. about 
    end
    return output
end

return p

I changed it to make it more generic so hope no errors crept in. I'd be interested to hear whether I could have done things better. Thanks. Jonathan3 (talk) 22:43, 29 May 2021 (UTC)

Glad you you got it working. Check out ipairs, normally you'll do something like for r, result in ipairs(results) do. ipairs iterates through an array in order, while pairs iterates through the elements of a table in arbitrary order. (ipairs is "indexed pairs"). If you're not using the r variable, a common convention is to use _ instead, so for _, result in ipairs(results).

Though I would recommend for _, row in ipairs(result) do, which is what I do on Leaguepedia and as a result is pretty standard on Gamepedia, so if you look for code examples there, you'll see this convention used. (Also a direct benefit is that row and result look more different than result and results so it's harder to misread on accident haha --RheingoldRiver (talk) 02:48, 31 May 2021 (UTC)

Thank you very much. I'll look into this. Jonathan3 (talk) 21:45, 31 May 2021 (UTC)

Merge similar cells and "striping" or rows

Each separate column is alternately grey or white in its background. This is all right for normal tables, but when "merge similar cells" is used it can create a patchwork effect. Would it be possible for the background colour to reset as soon as there is a full unmerged row? Thanks. Jonathan3 (talk) 22:08, 29 May 2021 (UTC)

Leaflet clustering

Is there some trick for getting leaflet to cluster markers? Adding |cluster=yes or |cluster=on does not seem to have any effect, although the identical code works for googlemaps. For instance

{{#cargo_compound_query:
tables=buildings;where=building_type='Library';fields=_pageName,Geo,architect,completion_date;icon=Blue-marker.png
|tables=buildings;where=building_type='Church';fields=_pageName,Geo,architect,completion_date;icon=Red-marker.png
|format=leaflet
|cluster=yes
|width=1000
|height=600
}}

does not cluster any markers. If I change the format to googlemaps it works. Do I need to add a cluster extension to leaflet? I see on the maps page that clustering is available. Just not sure how, within a cargo context, to get it to work. --Parma100 (talk) 01:26, 4 June 2021 (UTC)

Yes, clustering only works for the "googlemaps" format. Does the Maps extension support clustering for Leaflet? If so, that must involve some code that would need to be added to Cargo. Yaron Koren (talk) 17:39, 4 June 2021 (UTC)

It seems so. See https://maps.extension.wiki/wiki/Leaflet_SMW_queries. There are a number of clustering parameters specified.--Parma100 (talk) 18:26, 4 June 2021 (UTC)

[SOLVED] TypeError from Parser.php called in CargoHooks.php

I'm installing Cargo extension for the first time.

Help me, please.

Thanks in advance.

MediaWiki 1.31.8

PHP 7.0.15-0ubuntu0.16.04.4 (apache2handler)

MySQL 5.7.17-0ubuntu0.16.04.1

Cargo 2.8

There were 2 kinds of the Exception Details depending on the permissions.

(1) In case that the 3 permissions were set as follows :

$wgGroupPermissions['*']['createaccount'] = false;

$wgGroupPermissions['*']['edit'] = false;

$wgGroupPermissions['*']['read'] = false;

---

[de8ab7479ae0cc870fd568a4] /~dongshin/wiki/ TypeError from line 4841 of /home/dongshin/public_html/wiki/includes/parser/Parser.php: Argument 2 passed to Parser::setFunctionHook() must be callable, array given, called in /home/dongshin/public_html/wiki/extensions/Cargo/CargoHooks.php on line 41

Backtrace:

#0 /home/dongshin/public_html/wiki/extensions/Cargo/CargoHooks.php(41): Parser->setFunctionHook(string, array, integer)

#1 /home/dongshin/public_html/wiki/includes/Hooks.php(177): CargoHooks::registerParserFunctions(Parser)

#2 /home/dongshin/public_html/wiki/includes/Hooks.php(205): Hooks::callHook(string, array, array, NULL)

#3 /home/dongshin/public_html/wiki/includes/parser/Parser.php(336): Hooks::run(string, array)

#4 /home/dongshin/public_html/wiki/includes/StubObject.php(112): Parser->firstCallInit()

#5 /home/dongshin/public_html/wiki/includes/StubObject.php(138): StubObject->_call(string, array)

#6 /home/dongshin/public_html/wiki/includes/cache/MessageCache.php(1105): StubObject->__call(string, array)

#7 /home/dongshin/public_html/wiki/includes/cache/MessageCache.php(1081): MessageCache->getParser()

#8 /home/dongshin/public_html/wiki/includes/Message.php(1273): MessageCache->transform(string, boolean, Language, NULL)

#9 /home/dongshin/public_html/wiki/includes/Message.php(874): Message->transformText(string)

#10 /home/dongshin/public_html/wiki/includes/Message.php(934): Message->toString(string)

#11 /home/dongshin/public_html/wiki/includes/user/UserGroupMembership.php(470): Message->text()

#12 /home/dongshin/public_html/wiki/includes/user/UserGroupMembership.php(398): UserGroupMembership::getGroupPage(string)

#13 /home/dongshin/public_html/wiki/includes/user/User.php(5643): UserGroupMembership::getLink(string, RequestContext, string)

#14 /home/dongshin/public_html/wiki/includes/Title.php(2672): User::newFatalPermissionDeniedStatus(string)

#15 /home/dongshin/public_html/wiki/includes/Title.php(2651): Title->missingPermissionError(string, boolean)

#16 /home/dongshin/public_html/wiki/includes/Title.php(2737): Title->checkReadPermissions(string, User, array, string, boolean)

#17 /home/dongshin/public_html/wiki/includes/Title.php(2135): Title->getUserPermissionsErrorsInternal(string, User, string)

#18 /home/dongshin/public_html/wiki/includes/MediaWiki.php(207): Title->getUserPermissionsErrors(string, User)

#19 /home/dongshin/public_html/wiki/includes/MediaWiki.php(861): MediaWiki->performRequest()

#20 /home/dongshin/public_html/wiki/includes/MediaWiki.php(524): MediaWiki->main()

#21 /home/dongshin/public_html/wiki/index.php(42): MediaWiki->run()

#22 {main}

(2) In case that all of the 3 permissions were commented :

# $wgGroupPermissions['*']['createaccount'] = false;

# $wgGroupPermissions['*']['edit'] = false;

# $wgGroupPermissions['*']['read'] = false;

---

[dc67eb25f365c679f1763b25] /~dongshin/wiki/index.php/%EB%8C%80%EB%AC%B8 TypeError from line 4841 of /home/dongshin/public_html/wiki/includes/parser/Parser.php: Argument 2 passed to Parser::setFunctionHook() must be callable, array given, called in /home/dongshin/public_html/wiki/extensions/Cargo/CargoHooks.php on line 41

Backtrace:

#0 /home/dongshin/public_html/wiki/extensions/Cargo/CargoHooks.php(41): Parser->setFunctionHook(string, array, integer)

#1 /home/dongshin/public_html/wiki/includes/Hooks.php(177): CargoHooks::registerParserFunctions(Parser)

#2 /home/dongshin/public_html/wiki/includes/Hooks.php(205): Hooks::callHook(string, array, array, NULL)

#3 /home/dongshin/public_html/wiki/includes/parser/Parser.php(336): Hooks::run(string, array)

#4 /home/dongshin/public_html/wiki/includes/StubObject.php(112): Parser->firstCallInit()

#5 /home/dongshin/public_html/wiki/includes/StubObject.php(138): StubObject->_call(string, array)

#6 /home/dongshin/public_html/wiki/includes/cache/MessageCache.php(1105): StubObject->__call(string, array)

#7 /home/dongshin/public_html/wiki/includes/cache/MessageCache.php(1081): MessageCache->getParser()

#8 /home/dongshin/public_html/wiki/includes/Message.php(1273): MessageCache->transform(string, boolean, Language, Title)

#9 /home/dongshin/public_html/wiki/includes/Message.php(874): Message->transformText(string)

#10 /home/dongshin/public_html/wiki/includes/Message.php(934): Message->toString(string)

#11 /home/dongshin/public_html/wiki/includes/OutputPage.php(902): Message->text()

#12 /home/dongshin/public_html/wiki/includes/OutputPage.php(949): OutputPage->setHTMLTitle(Message)

#13 /home/dongshin/public_html/wiki/includes/page/Article.php(470): OutputPage->setPageTitle(string)

#14 /home/dongshin/public_html/wiki/includes/actions/ViewAction.php(68): Article->view()

#15 /home/dongshin/public_html/wiki/includes/MediaWiki.php(500): ViewAction->show()

#16 /home/dongshin/public_html/wiki/includes/MediaWiki.php(294): MediaWiki->performAction(Article, Title)

#17 /home/dongshin/public_html/wiki/includes/MediaWiki.php(861): MediaWiki->performRequest()

#18 /home/dongshin/public_html/wiki/includes/MediaWiki.php(524): MediaWiki->main()

#19 /home/dongshin/public_html/wiki/index.php(42): MediaWiki->run()

#20 {main}

--Dongshin (talk) 02:22, 6 June 2021 (UTC)

That seems like basically the same error in both cases. It's surprising, though. I have no idea what's causing this - this is straightforward PHP code that should be working. You haven't made any changes to your local Cargo code, I assume? Tied in with that, what code do you have on line 41 of CargoHooks.php? Yaron Koren (talk) 22:17, 7 June 2021 (UTC)

I didn't make any change to my local Cargo code. Anyway, I solved the symptom by downgrading to Cargo ver. 2.7.1 in the long run. Thanks anyway for your advice. I appreciate your works(Cargo, PageForms, etc.) and feel glad to see you here. --Dongshin (talk) 11:06, 13 June 2021 (UTC)

Well, I wouldn't really call that a long-term solution, but I'm glad you got it working. My new guess is that upgrading your PHP version will also fix the problem. Yaron Koren (talk) 02:00, 14 June 2021 (UTC)

Thank you for your advice! **Dongshin (talk) 09:47, 14 June 2021 (UTC)

Compound query catch-all, similar to SMW extension?

There's on feature of SMW's "compound query" that I miss, the ability to add a catch-all subquery, which would help me display red-link pages that aren't in joined tables yet.

   "#compound_query avoids displaying the same page more than once, so if a page gets returned by more than sub-query, it's displayed only for the first one. In this way, a compound query can have specific sub-queries, and then a general catch-all sub-query at the end:" 

Is it possible to replicate this in Cargo?

--Frybread (talk) 19:17, 8 June 2021 (UTC)

Calendar query with List of Dates field

With Cargo versions since this commit [May 19 18:06:14 2020 +0530, commit 188df0048010905a05e4b90ad5b82a7436f78978], cargo queries using format=calendar and a field that's a List of Dates doesn't show pages on the calendar that have more than one date in the list. If just one date in the list, it does show on the calendar.

If I revert to the commit prior to that one, it works as expected.

Was there a change in the query syntax after this commit (which concerned SQL subqueries)? I'm just using, for example, "fields=_pageName,Birthday" where Birthday is the list of dates

I'm using MediaWiki 1.35.1 --Chris Gateley (talk) 19:38, 10 June 2021 (UTC)

I'm surprised that this ever worked - the handling of "List of Date" fields has always been pretty poor, including the lack of a "_precision" field within the DB helper table for such fields. This would probably take some work to get working again. Yaron Koren (talk) 13:21, 11 June 2021 (UTC)

So is there some way to add recurring events such as birthdays to a calendar without using a "List of Date" field? --Chris Gateley (talk) 23:11, 24 June 2021 (UTC)

Compound query question

If I use a compound query with different tables and output it to a map, I'm finding that the last table in the query doesn't get all the fields transmitted to the map. In other words, with the code

{{#cargo_compound_query:
tables=buildings;where=building_type='Library' AND landscape_architect='{{PAGENAME}}';fields=_pageName,Geo,completion_date;icon=Blue-marker.png
|tables=Grounds;where=type HOLDS 'Park' AND landscape_architect='{{PAGENAME}}';fields=_pageName,Geo_center,opened;icon=Green-marker.png
|format=leaflet
|height= 600
|width= 1000
}}

the pins show up for every hit (both tables). However, the _pageName on the second table's hits doesn't get to the marker. If I swap the lines of code, the opposite happens. The second table's _pageName doesn't make it to the map pin. Any ideas? --Parma100 (talk) 15:50, 18 June 2021 (UTC)

This is just a pure guess. I have a similar query which works, but I've got an alias for _pageName (e.g. _pageName=Place) which is the same for both queries. Maybe that would fix it? Jonathan3 (talk) 22:50, 20 June 2021 (UTC)

I tried that...no joy. However, I got it to work another way: I had the coordinate fields in each table named differently: Geo and Geo_center respectively. On a hunch, I reformulated one of the tables such that the coordinate fields in both were named Geo. That worked. So whatever is happening under the hood, using different names for the coordinate field may cause problems in a map display. Interestingly, the table output and dynamic table output worked either way. And the maps all displayed the pins in the correct place. But _pageName was a casualty in whatever query was last. Alls well that ends well, but it did have me scatching my head for a while. Thanks for your input. --Parma100 (talk) 23:45, 20 June 2021 (UTC)

All's well that ends well. As it happens, in my compound query the field names are the same for both queries. I was going to suggest this as a relevant difference between us, but the example query on the extension page also has differently named fields. Jonathan3 (talk) 13:10, 21 June 2021 (UTC)

Cargo duplicate row workaround

Duplicate rows have always been an intermittent problem, but it ceases to be such a big deal if you add GROUP BY to every query. It just needs to have exactly the same rows as the underlying MySQL SELECT query, e.g.

{{#cargo_query:
tables=tablename
|fields=fieldname
|group by=fieldname
|where=_pageName="whatever"
}}

or (note the additional date row):

{{#cargo_query:
table=tablename
|fields=_pageName=Name, fieldname, datefieldname
|order by=News.RSS_pubdate DESC
|group by=_pageName, fieldname, datefieldname, datefieldname__precision
}}

The effect is that only one of the duplicate (or triplicate or whatever!) rows is returned. I hope this helps someone, and if you notice any problems with this approach then I'll benefit too. Jonathan3 (talk) 14:20, 19 June 2021 (UTC)

Date Approximation

Would it be possible to add some possibilities to the date_precision? I'm going to get a lot of users who only have approximate dates to contribute (when a building was built or when something else happened historically). I'd like to reflect the uncertainty in the infoboxes, etc, but I don't think there's a way to do that currently. The precision criteria are great for only inputting a month and year, or only a year. Could we add some other selections, for instance 4 could be a flag indicating an approximate date, 5 a flag for aproximate month and year and 6 a flag for approximate year? I would code this myself and offer it for inclusion (it would at least affect the form code and the cargo store and retrieve code, no doubt) but I don't know in the code where they might be. Any suggestions or objections? --Parma100 (talk) 00:43, 21 June 2021 (UTC)

@Parma100: I do something like this on a couple of wikis by using a second column, e.g.:

| date = Datetime
| date_precision = String (allowed values=exact,day,month,year,circa,floruit)

I wonder if trying to add too much into the built-in date handling is a good idea — it feels like it could be a slippery slope. —Sam Wilson 03:19, 21 June 2021 (UTC)

@Samwilson: That's a great idea. Thanks for pointing me in that direction. I think I can get where I need to be with a boolean field right behind each date field, and hadn't thought of that. Point taken as to adding to the complexity under the hood. --Parma100 (talk) 12:45, 21 June 2021 (UTC)

I'm not sure if adding more values to the built-in "precision" field is possible. It's built-in, meaning that users can't affect its settings, and it's based entirely on the entered date value, so "January 1950", for example, would clearly get the "year and month only" setting. What string would lead to an "approximate year" setting - "c. 1950"? Actually, maybe. And potentially a string like "1980s" could get a "decade" setting, while a string like "20th century" would get a "century" setting. But these are both English-specific, and I don't know if there's a way to parse these kinds of strings in a language-neutral way. In any case, an additional "precision" field that's actually part of the template is a reasonable approach. Yaron Koren (talk) 19:30, 21 June 2021 (UTC)

I agree. It's a simple matter to put a boolean (checkbox) right behind the date entry with the label "approximate?" on the PageForm input. Those sorts of ideas are why this kind of page is so useful. What you assume is the most efficient way to accomplish something may be way harder than someone else's approach. Good stuff. --Parma100 (talk) 21:04, 22 June 2021 (UTC)

I've often wondered if it'd be good to add a page here on which to collect common Cargo "patterns", so people could share things they've learnt. Not copy-and-pasteable things, but just general ideas of how to use Cargo (for example, I remember someone saying that they don't use page names for foreign keys, but rather set up redirects in order to have stable values). —Sam Wilson 02:00, 23 June 2021 (UTC)

A "standard patterns" section or page could be a good idea, similar to the Page Forms documentation's "Data design issues" section. It could even go in the same spot - Cargo's own "Common problems" page. Or would a separate page be better? Yaron Koren (talk) 23:07, 23 June 2021 (UTC)

A separate page, linked to in the navigation bar on the main Cargo extension page. And maybe on the Page Forms page as well. This is (another) great idea.--Parma100 (talk) 00:53, 24 June 2021 (UTC)

Yeah I think a separate page sounds good. I have a couple of ideas of things to add. I'm sure others do too! Sam Wilson 01:28, 24 June 2021 (UTC)

Alright. Any thoughts as to the name? "Design patterns"? "Standard patterns"? "Tips and tricks"? Yaron Koren (talk) 14:10, 24 June 2021 (UTC)

I'd go with "Tips and tricks." Everyone will understand what that page is for. It may have something that solves your issue, or it may not. Parma100 (talk) 17:48, 24 June 2021 (UTC)

+1 for Extension:Cargo/Tips and tricks :) Sam Wilson 00:45, 25 June 2021 (UTC)

Sorry for the delay. I was about to create that page, and then I realized that just about everything listed in the Common problems page could also be listed under "tips and tricks". Should that page just be renamed, maybe? Yaron Koren (talk) 15:57, 28 June 2021 (UTC)

Well, that would be your call, although it seems to me that "tips and tricks" aren't necessarily problems, but helpful techniques or approaches. On the other hand, the "problems" on that page have proposed solutions, and those solutions fall under the tips and tricks umbrella. Apparently I'm of two minds on this! Whichever, is fine with me (if it matters).Parma100 (talk) 21:25, 6 July 2021 (UTC)

Remove Namespace from List of Pages queries

Hello, I've been digging through the archives and this topic was the closest I could get to an answer. I have multiple tables where one of the fields contains user pages which are obviously in the User: namespace. I would like to be able to query the list without User: before each item while retaining the links. I have tried a variety of CONCAT and SUBSTRING methods but have been unable to find a solution. Please help. Thanks, Penguinaut 22 (talk) 02:32, 27 June 2021 (UTC)

I think something like CONCAT('[[', FieldName, '|', SUBSTRING(FieldName, 6), ']]') would work. Yaron Koren (talk) 15:13, 9 July 2021 (UTC)

Thank you for your response. Your suggestion works if I'm querying a single page, but not a list of pages. The query I have right now is:

{{#cargo_query: tables=Campaigns
|fields=CONCAT('[[', players, '|', SUBSTRING(players, 6), ']]')
|where=name="Sandbox Campaign"
}}

which results in the following:

Error 1064: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ') LIMIT 100' at line 1 (1.db.mywikis.net)

Function: CargoSQLQuery::run Query: SELECT CONCAT(, SUBSTRING(players, 6), ) FROM `cargo__Campaigns` WHERE name="Sandbox Campaign" ORDER BY CONCAT(, SUBSTRINGCampaigns.players__full, 6), ) LIMIT 100

Could you use #arraymap for a list? Jonathan3 (talk) 13:42, 13 July 2021 (UTC)

Okay, for a list of values it's a little trickier - you'll probably have to get into the internals of the storage, unfortunately. This should work, I think:

{{#cargo_query:
tables=Campaigns,Campaigns__players
|join on=Campaigns._ID=Campaigns__players._rowID
|fields=CONCAT('[[', _value, '|', SUBSTRING(_value, 6), ']]')
|where=name='Sandbox Campaign'
}}

Yaron Koren (talk) 00:35, 16 July 2021 (UTC)

Thank you Yaron, that worked! Penguinaut 22 (talk) 17:15, 25 July 2021 (UTC)