Extension talk:Cargo/Archive April to July 2019

Querying data in cargo with the "HOLDS" command combining with "AND" clause [RESOLVED]

Hi, How can I modify the "HOLDS" command in this example to use an "AND" clause? (Version I'm using now: MediaWiki 1.32.1 Cargo 2.1.1)

Example what I'm trying to achieve (lets pretend the Authors wrote among others one book together and Authors is decleared as a List of Strings):

{{#cargo_query: tables=Books |fields=_pageName=Book |where=Authors HOLDS 'Leo Tolstoy' AND Authors HOLDS 'Dostoevsky' }}

It works fine with 'OR' but sadly no results for 'AND'.

I asked a similar question in January - Use HOLDS with multiple values - and the suggestion then was to query the actual Cargo tables. I didn't pursue it so can't give you a full answer. Jonathan3 (talk) 13:38, 14 May 2019 (UTC)

Just tried to query the actual Cargo tables but with the same result. It works just fine with 'OR' but no sign of result with 'AND'.

{{#cargo_query:tables=Books,Books__Authors|join on=Books._ID=Books__Authors._rowID|fields=_pageName=Book,Books.Authors__full=Authors|where=Books__Authors._value = 'Leo Tolstoy' AND Books__Authors._value = 'Dostoevsky'}}

Should I assume there is still no any solution for this issue? Grombol

Update: The issue is solved! Here is the explanetion Grombol

Importing from XML Pagename delimited by ~

Got a problem. Creating a Cargo table "Sources" (and category/form/template) that has a field "Repertoire" that has multiple values (list ~ of Page) where each page is from Category=Repertoire. Each entry in the Source.Repertoire is a pageName from category Repertoire. I've uploaded a scad of new "Source" pages via XML upload that has the correct Pagenames in the Repertoire field delimited correctly. After the XML upload process, all the pages were recognized, and I can click the link in the newly created Source Page/Cargo table, and follow those links to the correct already exiting page (in category "Repertoire"). Looks like the upload worked great.

But, in the individual repertoire pages, I have a cargo query that returns a list of all sources that HOLD the pagename in Sources.Repertoire.

Here's the query in the Repertoire Template

{{#cargo_query:
tables=Source
|fields=_pageName
|where=Repertoire HOLDS '{{PAGENAME}}'}}

None of the pages uploaded by XML show correct query results. However, if I go into Form for Sources, and add individual pages in the Repertoire field. , it works. The Cargo Source table shows an entry (from the XML upload) and another entry (from the FORM). They are the same page. If I follow the XML generated link, edit it, and open the Form generated link, it's definitely the same page.

Stumped.May214 (talk) 21:03, 2 April 2019 (UTC)

Also tried

'{{FULLPAGENAME}}'

May214 (talk) 21:04, 2 April 2019 (UTC)

Just to be clear - is the problem in the data storage, or the data querying, as far as you know? Yaron Koren (talk) 23:55, 2 April 2019 (UTC)

My assumption is storage. If I add an individual entry using the Form (rather than a mass upload by XML) then the query works. THe XML loaded entries work functionally - I can click and follow links to existing pages. May214 (talk) 12:37, 3 April 2019 (UTC)

If you resave one of the imported, non-working pages (making a minor change first), does it start to show up in queries? Yaron Koren (talk) 02:24, 11 April 2019 (UTC)

No. If I go to Cargo Tables, and view the table for each entry, all the correct pages show up on the table viewer. I can click and it will open the correct page. In order for the query to work, I have to enter the page in the Form. The field in the Form is set up to autocomplete on category. When I add using the Form the query will work. But I'm adding the same page that is already showing up in the Cargo table. The exact same page. I can open the page from the cargo table, make a minor edit and save it. The query doesn't pick it up. Then if I add the same page in the Form (now I have duplicate entries of the same page in the table) and open the Form added page, the minor edit shows up (proving I don't have duplicate pages) and then the query works. Also, If I add the exact text of the page title without clicking on a page that shows up from the autocomplete options, the query doesn't work. There's something about selecting the page from the autocomplete options that makes the query work. But it shouldn't matter,right? I mean the page title is currently held in the Cargo Table and functions. May214 (talk) 19:07, 18 April 2019 (UTC)

Update. I can't copy paste text into the form either. I have to start typing the title, and then select the autocomplete option that works. May214 (talk) 19:16, 18 April 2019 (UTC)

Don't you have to include the Repertoire field in your field= line? Jeremi Plazas (talk) 19:28, 19 April 2019 (UTC)

Well, I want the query to return the pageNames in the source tables that hold a certain item in Repertoire. And it works, when I add each page by autocomplete using the Source Form. But I'll try a couple variations to see if including Repertoire in the fields work. If I just replace _pageName with Repertoire, it spits out the name of the repertoire, and not the source that holds it in repertoire. 204.124.167.5 15:51, 23 April 2019 (UTC)

So, no other suggestions? Any idea why it will work with an autocompleted entry, but not with a straight text entry? May214 (talk) 20:28, 1 May 2019 (UTC)

Sorry - I read through your explanation from April 18 about five times, but I still don't understand it. Each time you say "form" and "table", I don't know if you're talking about Source or Repertoire. This might be easier for me to understand with an example (even a fake one). Yaron Koren (talk) 00:53, 2 May 2019 (UTC)

Date and "Start date"

I'm planning to merge two tables. Table A has a Date field. Table B has Start date and End date fields. I am considering converting Table A's Date field to Start date (with no End date), and having Start date and End date for Table B's data. Would this work? Is Start date, when it's there on its own, treated as if it were Date? Thanks. Jonathan3 (talk) 23:05, 10 April 2019 (UTC)

I think having a "Start date" field without an "End date" field is simply not allowed. Yaron Koren (talk) 02:18, 11 April 2019 (UTC)

I’ll check it out but what about this? “Start date, End date - similar to Date, but are meant to hold the beginning and end of some duration. A table can hold either no Start date and no End date field, or exactly one of both.” Jonathan3 (talk) 08:08, 11 April 2019 (UTC)

That's right. Yaron Koren (talk) 15:50, 11 April 2019 (UTC)

I see what you meant now! I would end up with both a start date and end date field but usually with no value for end date. I was wondering if a start date would be as good as having a date for those entries with no end date value. Jonathan3 (talk) 16:04, 11 April 2019 (UTC)

Oh, I see. Well, either or both values can be null. Out of curiosity, why would the end date usually be null - is it because the end date is unknown, or because the concept of an end date is invalid for those pages? If the latter, it may be worth restructuring the data. Yaron Koren (talk) 17:12, 11 April 2019 (UTC)

Table A relates to documents (publication date) and table B relates to consultation documents (publication=start date, and end date). I want both together so that, for example, it’s easy to drill down on author. It seems there are two choices:

1. Use a Start Date for every document and leave End Date empty unless required.
2. Retain “publication date” as a Date, and when necessary ALSO use both Start Date (same value as the publication date Date field) and End Date.

Which seems preferable? Thank you. Jonathan3 (talk) 21:23, 11 April 2019 (UTC)

Okay, now I think I fully get it. Merging these two seems awkward... I know that it would make drill-down easier, but it still seems strange. If you're planning to do it, though, the first approach seems better. Yaron Koren (talk) 22:22, 11 April 2019 (UTC)

Thanks. I'll probably adopt a third option - keep publication date as an ordinary Date field, and add an End/Closure date also as an ordinary Date field. If I decide I need any of the special Start Date & End Date stuff (e.g. lines instead of dots on timeline format) then I can change the table definition later. Jonathan3 (talk) 13:56, 12 April 2019 (UTC)

That approach makes sense too. Yaron Koren (talk) 14:20, 12 April 2019 (UTC)

How does one exclude something in a Cargo query? [RESOLVED]

Forgive my noobiness...

How does one exclude something in Cargo? I've got a query of the type:

{{#cargo_query:
	tables=LtrhfFiles
	|fields=_pageName,Caption,AssetPage
	|where=AssetPage HOLDS '{{PAGENAME}}' AND _pageName NOT '{{{FeaturedImagePage| }}}'
	|format=template
	|template=TinyGallery
}}

Obviously i get an error, but would like to accomplish the following: find all pages in that table where the AssetPage field contains {{PAGENAME}}, but excluding the pagename {{{FeaturedImagePage| }}}.

--Jeremi Plazas (talk) 18:46, 19 April 2019 (UTC)

I can give you half an answer. For most fields, change your NOT to <> and it'll work. I can't get it to work with _pageName fields though. Jonathan3 (talk) 21:03, 19 April 2019 (UTC)

P.S. It works fine. I just forgot that the pages in my test query had a namespace (it works when adding the namespace, or instead using _pageTitle). Jonathan3 (talk) 21:11, 19 April 2019 (UTC)

Fantastic! That did it thank you. Jeremi Plazas (talk) 04:02, 20 April 2019 (UTC)

ORDER BY CASE is broken with PostgreSQL and general Pg issues

Cargo seems to quote the ORDER BY clause, making it a column name:

Apr 20 20:13:25 db postgres[62078]: [41-1] ERROR:  column "CASE WHEN "Rarity" = 'Normal' THEN '1' WHEN "Rarity" = 'Rare' T" does not exist at character 183
Apr 20 20:13:25 db postgres[62078]: [41-2] STATEMENT:  SELECT /* CargoSQLQuery::run  */  "Name" AS "Name","Rarity" AS "Rarity","5-4note" AS "5-4note"  FROM "cargo__shipDrops"    WHERE "5-4" IS NOT NULL OR "5-4note" IS NOT NULL  ORDER BY "CASE WHEN ""Rarity"" = 'Normal' THEN '1' WHEN ""Rarity"" = 'Rare' THEN '2' WHEN ""Rarity"" = 'Elite' THEN '3' WHEN ""Rarity"" = 'Priority' THEN '4' WHEN ""Rarity"" = 'Super Rare' THEN '5' WHEN ""Rarity"" = 'Ultra Rare' THEN '6'  ELSE ""Rarity"" END" DESC,"ID" LIMIT 100

Does it pass the quotation marks when used via the API?

Another note: GROUP BY clause parsing:

ERROR:  column "artist" does not exist at character 90
HINT:  Perhaps you meant to reference the column "cargo__ships.Artist".
STATEMENT:  SELECT /* CargoSQLQuery::run  */  "Artist" AS "Artist"  FROM "cargo__ships"     GROUP BY Artist ORDER BY "Artist" LIMIT 500

If "Artist" is put in double quotes for groupBy, it escapes that which results in

ERROR:  column ""Artist"" does not exist at character 90
HINT:  Perhaps you meant to reference the column "cargo__ships.Artist".
STATEMENT:  SELECT /* CargoSQLQuery::run  */  "Artist" AS "Artist"  FROM "cargo__ships"     GROUP BY """Artist""" ORDER BY "Artist" LIMIT 500

--IijimaYun (talk) 11:30, 20 April 2019 (UTC)

Storing Cargo in Scribunto pages

I'm not sure if this is a Cargo thing or a Scribunto thing but it seems like data won't store on any page with Scribunto content model - if that's a cargo thing could it be changed or added to a config setting? I want to store some cargo data about a module on its documentation, and I'd prefer to have that be an in includeonly and executed on the page itself instead of on the doc page for consistency/clarity. --RheingoldRiver (talk) 08:42, 5 May 2019 (UTC)

(This is the data I'm showing, I'd like to have it immediately update when the module page is edited rather than needing some other location blank edited as well) --RheingoldRiver (talk) 10:16, 5 May 2019 (UTC)

Well I guess I could do this without Cargo at all & instead do a DPL query to get category members and then pull data by directly transcluding everything here with mw.loadData. But it would still be nice to have Cargo able to store from Scribunto pages --RheingoldRiver (talk) 10:47, 5 May 2019 (UTC)

Are you saying that pages in the Module: namespace aren't storing Cargo data? If so, that's not too surprising, since modules can't call templates... or can they? Yaron Koren (talk) 14:42, 5 May 2019 (UTC)

Yeah, but the documentation on module pages does transclude templates (that's how you add categories to Lua modules etc), so I would expect Cargo transcluded from the documentation subpage to store properly even in that content model --RheingoldRiver (talk) 01:25, 6 May 2019 (UTC)

What's an example of such a module? Yaron Koren (talk) 03:08, 6 May 2019 (UTC)

Everything in the category I linked above, and here's a direct link to one in particular --RheingoldRiver (talk) 09:47, 6 May 2019 (UTC)

Okay, so you linked to the module, but the documentation page for it, which is a standard wikitext page, is here. But how is the category getting set for the actual module? Somehow the /doc page is doing it, but I don't see how. Is that some Scribunto feature? Yaron Koren (talk) 14:29, 6 May 2019 (UTC)

The doc page is just transcluded at the top of the module. I did weird things with titleparts on that particular one to make the display on the doc subpage completely different from the display on the module page. Also I just realized that I deleted all Cargo from this within an hour of setting it up because I did change to DPL, whoops. Maybe this is a better example of how doc pages are transcluded to module pages. --RheingoldRiver (talk) 09:37, 7 May 2019 (UTC)

Calendar display that lists different text than page name but still works as a link

I am trying to set up a calendar that displays something other than a page name in the event, but that can still link to the event page. I tried generating a link with something like

CONCAT('[[', workdays._pageName, '|', person, ' - ', Location, ']]') = name

but that just shows the []s, it doesn't use them to designate a link. I could not find an alias to use for this in the calendar documentation, and the "link field" available for the gallery etc don't seem to work. Is there a way to do this? Thanks! Tenbergen (talk) 05:52, 8 May 2019 (UTC)

I think the following will work, though I'm not sure: "fields=workdays._pageName,CONCAT(person, ' - ', Location)=name". The documentation could be better, I admit... I'm not sure I understand it myself. :( Yaron Koren (talk) 03:23, 10 May 2019 (UTC)

Thanks Yaron, that does the trick; so does that mean the first field listed will always be taken as the "link" value, and there is no named alias this, and the name as "what yous see"? I am trying to think how I would update that in Extension:Cargo/Display_formats#calendar. Tenbergen (talk) 22:45, 22 May 2019 (UTC)

Group or Order by YEARWEEK (or by anything...)

I can get Cargo to display the following: |fields=YEARWEEK(_pageData._modificationDate) - but can't get it to work with |group by= or |order by=. Is this possible somehow? Thanks. Jonathan3 (talk) 13:13, 16 May 2019 (UTC)

The SQL produced has an ORDER BY but only based on the order the fields are listed in the |fields= line, and there is no GROUP BY in the SQL at all. Just tried a really simple query and the same applies, i.e. |group by= and |order by= are being ignored. Jonathan3 (talk) 13:20, 16 May 2019 (UTC)

"Group by" works elsewhere on my website so it must be something stupid I'm doing... ignore the question for now :-) Jonathan3 (talk) 22:13, 16 May 2019 (UTC)

Correct storage of values of multiple instance templates only after Recreate Data on main cargo table

Hello everybody,

may I ask you for some time and expertise on an issue related to cargo storage of values of a multiple instance template (MIT).

Here some details:

In the form of an indicator database I use the functionality of multiple instance templates to enter current measurementvalues of the corresponding indicator. It is implemented in this way within the form of the indicator database:

{{{field|current|holds template}}}

{{{for template|assessment|multiple|add button text={{int:add-assessment}}|embed in field=indicator[current]}}}
;{{int:date}}:{{{field|assessment date|input type=datepicker|disable input field|week start=1|highlight days of week=0|date format={{int:date-format-form}}|size=8}}}
;{{int:current}}:{{{field|assessment current|size=10|default=0}}}
{{{end template}}}

All values of the MIT are stored in an additional cargo table called "assessment".

Based on the most recent currentvalue of that table, I try to calculate the current progress of the indicator and store it to the cargo-table "indicator". The formular of the progress calculation is like that:

(current - baseline) / (target - baseline) * 100

Baseline and target can be entered in the form of the indicator database, while the current value(s) are entered via MIT as stated above

The corresponding store command in the indicator template looks as follows:

{{#cargo_store: 
_table=indicator
|progress={{#expr: ({{#cargo_query:
                      tables=assessment
                     |fields=TRIM(current)
                     |where=_pageID="{{PAGEID}}"
                     |limit=1
                     |order by=date DESC
                     |more results text=
                     |no html
                    }} - {{{baseline|}}}) / ({{{target|}}} - {{{baseline|}}}) *100}}
}}

The problem is now, that every time I add a new instance with a recent current value, cargo stores "0" to the progress column. First of all I was assuming that it is maybe an data update problem and a null edit on the indicator would solve the problem. But I was mistaken. The only way to fix the problem is to recreate the indicator cargo table. After recreating the table the progress column is being calculated correctly. But as soon as I add a new instance or edit an existing instance of the MIT progress is again "0" and will only be calculated correctly after recreating the (indicator) table once again.

Anybody an idea, what could cause the problem of false storage of the current progress and how it can be resolved?

Thanks in advance. Axel

It's not surprising that this is not working - the page is querying itself, and it's storing the computed data at the same time that the original data is being stored. So everything depends on the timing - which templates get called first. I don't know what the best solution for this is. Would it be possible to not store the "progress" at all, but instead query it when needed? Or would that lead to queries that are too complex for #cargo_query? Yaron Koren (talk) 16:58, 21 May 2019 (UTC)

Text-file output display format

The T223886: Add a bibtex format page on Phabricator reminded me of something...

Would it be possible to add a text-file Export display format, with the text being formatted using a template in the same way as with the normal Template display format?

I'd like to be able to offer people a link to a RIS file for import into Endnote (reference management software). And maybe other types of file too.

Thanks. Jonathan3 (talk) 22:06, 23 May 2019 (UTC)

I wonder if it'd be cool to make a way of exporting to any other format in that fashion, e.g. a filename=foo.txt parameter would give text, and filename=foo.gpx would give XML (both coming from a template, as you say, via the |format=template |template=foo parameters). Maybe there would be weirdnesses that would arise with that though; it's not always wise to generalize things too much. Sam Wilson 12:24, 26 May 2019 (UTC)

Or maybe, in an ideal world, have all output formats default to displaying on screen (including csv etc), and have a new “destination” parameter to override that eg destination=txt, destination=csv to save as file, or destination=link to display link to display in new tab/window.

This could be in conjunction with format=template and template= (eg for RIS format) or without that (eg for CSV or other formats that Cargo itself can create).

It would also allow new formats to be added to Cargo in future (without then needing format=template any more).

I haven’t fully thought this through and maybe (even if it makes sense) it would require too much of a change to Cargo code to be worth it. Jonathan3 (talk) 14:39, 26 May 2019 (UTC)

The basic idea of being able to customize an export format is very clever - it makes sense to do this for obscure formats, rather than requiring a Cargo result format to be created every time. Though something widely-used like BibTeX seems like it should have a true result format. Maybe I'm just saying that because SMW has a "bibtex" format, though. I had never heard of RIS before - do you happen to know how popular it is compared to BibTeX?

Potentially this new query format could be called "export template", or just "export" - or maybe even just "template", with another parameter like "export" to indicate that this is an export format. The last of those seems unnecessarily confusing, though.

The "filename" parameter idea I'm not sure I understand - I mean, it's useful to allow for a "filename" option, but I don't see how the file suffix could dictate what the file type would be. It could be possible to have "file type=" parameter, to specify that manually, though, assuming that doesn't pose any security risk.

Jonathan3 - any chance that you would be willing to create this query format? I guess I could ask the same question of Sam Wilson... Yaron Koren (talk) 02:42, 27 May 2019 (UTC)

As far as I know, RIS format is used primarily in Endnote, which many academics use to store resources and create footnotes and bibliographies, though other software like Zotero can use the format too. I get the impression that there is some overlap with Bibtex but the latter is used more in publishing rather than just storing a personal database of resources/writing articles. Not sure.

If I were to create a template for the RIS format it would just take named parameters like AU=Shannon, Claude E. | TI=A Mathematical Theory of Communication - or AU={{{Author|}}} | TI={{{Title|}}} - to create text like in the example record on the Wikipedia page.

Let's say you had Bibtext as a Cargo-supported format and RIS was just a template, you could have:

• Bibtex: format=bibtex
• RIS: format=template | template=ris

I mentioned a destination parameter, and I'm not sure what the best solution would be, so here are some random ideas...

• I guess currently (using that idea) format=csv, format=excel and format=json are effectively what would be destination=file. The RIS example above, and any format=template format, would currently just display on screen, so we would need a way to know to show a download link instead. This could be destination=file, or (as suggested above), just filename=....
• Incidentally, it may be useful to be able to show the CSV/JSON (and Excel?) within the wiki page, in which case destination=screen might work.
• The "Export citation" link on this randomly-selected academic article does different things for the various formats (which include Bibtex and Endnote, but not RIS as it happens). Some display on a new web page and some prompt you to download a file. I wonder whether there could be destination=newpage (link opens new page) and destination=file (link downloads file) or, somehow, the possibility of offering both options beside each other.
• Alternatively, if the link were to view on a new web page, maybe a tooltip could explain that right-clicking will allow file download.

All the best, Jonathan3 (talk) 22:12, 27 May 2019 (UTC)

HTML Entities special export

READ: this might be just a problem of my setup, sharing it just in case someone is facing a similar issue..

Version: Cargo 2.1.2

Problem: Values containing characters which are also HTML Entities (in my case the char &) are properly stored as plain text in cargo tables, but when exporting them as csv either with or without "parse values" param, the result contains HTML encoded value, consequently causing the row to be wrongly displayed and splitted in multiple columns when opened with excel.

NOTE: I found out that parse values parameter needs to be declared with a value (e.g. |parse values=something) in order to be encapsulated and properly evaluated as WebRequest parameter. Alternatively it will not be taken in consideration..

With parse values parameter in the query, the value will be initially properly decoded as text but then parsed by the function smartParse in includes/CargoUtils.php, resulting in the csv containing an & and consequently causing the row to be splitted in multiple columnd if opened with excel.

Without parse values parameter in the query, the value will still be parsed but not be decoded, having as result the same unwanted result as above.

My dirty solution, in "specials/CargoExport.php":

// Added html decoding function for all values
function decodeHTML( $queryResults ) {
	$parsedQueryResults = array();
	foreach ( $queryResults as $rowNum => $rowValues ) {
		$parsedQueryResults[$rowNum] = array();
		foreach ( $rowValues as $colName => $value ) {
			$value = htmlspecialchars_decode( $value );
			$parsedQueryResults[$rowNum][$colName] = $value;
		}
	}
	return $parsedQueryResults;
}
// Call decodeHTML function id not set $parseValues
function displayCSVData( $sqlQueries, $delimiter, $filename, $parseValues ) {
...
	if ( $parseValues ) {
		$queryResults = $this->parseWikitextInQueryResults( $queryResults );
	}else{
		$queryResults = $this->decodeHTML( $queryResults );
	}
...

Built-in UCFirst?

Could you add a built-in ucfirst function? Since pagenames are first-letter case-insensitive it's not great to have to workaround this currently. --RheingoldRiver (talk) 00:26, 26 June 2019 (UTC)

Where would you want to call this function? I'm guessing in a #cargo_query call within Special:RunQuery, but maybe it's something else. Yaron Koren (talk) 02:52, 26 June 2019 (UTC)

Actually it's all over the place, the issue I have is players rename constantly, and often change case when they rename. So I have a table called PlayerRedirects that has all of the pages that redirect to the player's overview page, and I join that between Players table & any other table you might query with player information, with a BINARY condition to prevent dupes that would otherwise occur due to the case changing. In the past, I stored the player's display name & link as different fields in case of disambiguation, but yesterday I realized this setup is silly since I can do a pattern match to replace the disambig part of a link to get the display, so I may as well store only the link. BUT then I realized this still doesn't work, because if the display is a lowercase first letter, then the BINARY condition will fail. And I'd rather not double the size of PlayerRedirects, already a pretty big table. And since I'm publishing docs on how to access my API I don't want to do a workaround with looking at part of the name at a time to compare, I'd rather be able to tell people "just do a UCFIRST() in the BINARY condition."

One possible workaround to this would be to store only one single capitalization to the redirects cargo table, and kill the BINARY condition completely, but then either I'm further complicating by adding another where condition and column ("IsUniqueCase" which is true for exactly one capitalization per set of redirects) or removing some redirects from Cargo completely, and that might come with its own issues. But regardless the situation I have doesn't seem TOTALLY absurd for wikis that use a lot of Cargo, so I think it might be worth adding in general, as a kind of "title-ify for MediaWiki" functionality. --RheingoldRiver (talk) 12:56, 26 June 2019 (UTC)

Oh... you're talking about an SQL-style function. I thought you were talking about a parser function. You could always go into the database and create one (well, get a DBA do it)... or you could have a workaround like the one described here. Would either of those be a good enough solution? Yaron Koren (talk) 14:19, 26 June 2019 (UTC)

Oh, yeah sorry. I thought it would be similar to HOLDS, and had to be added to the extension directly. I do want something that's done server-side with a friendly syntax so I don't have to make people using our API do the workaround in the query itself, but if this can be aliased somehow without a direct edit to Cargo that would definitely be enough - I'll talk to one of our sysadmins and get back to you --RheingoldRiver (talk) 16:54, 26 June 2019 (UTC)

_pageData table stores category of pages/files only after second saving of page

I am currently switching former SMW-content to a cargo-based enviroment. In the SMW-environment we used categories massively and queried for files/pages within these categories via ask-queries.

Querying for categories directly is, as I understood the cargo documentation, not possible anymore via cargo queries. For that reason, I try to query for categories via the _pageData cargo table.

Queries like this work fine:

{{#cargo_query:
  tables=_pageData
 |fields=_pageName
 |where=_categories HOLDS "Category X" 
 |group by=_pageID
}}

The problem I face is, that the category of any page or file is only being stored to _PageData after the second saving of the page/file. Is there maybe any parameter I am not aware of, which forces the _pageData table to store the category in the first run? Or is it a bug?

I had a similar problem - queries that should have worked didn't, though I never worked out that it required a second save to make it work - and I ended up using DPL3 on this part of my website, even though I'd have preferred to use Cargo. I'd be interested to hear the solution. Jonathan3 (talk) 13:35, 1 July 2019 (UTC)

I don't think I knew about this problem before - it sounds like a bug, and, thinking about it now, maybe an intractable one. The problem is that, in order to get the category information, the code queries the "categorylinks" DB table, then stores the information in its own _pageData table. So when the page is first saved, the "categorylinks" information doesn't exist yet, so it can't be queried. The only suggestion I can think of is to... use categories less? If you store the relevant data as template parameters instead of categories, it's easier to query anyway - and easier to edit via forms. Yaron Koren (talk) 13:16, 2 July 2019 (UTC)

Maybe Cargo could use a MediaWiki hook to add/remove the relevant information in the _pageData table when a page is saved. If this would slow things down it could be optional. Jonathan3 (talk) 22:42, 6 July 2019 (UTC)

That seems like it could work, yes. Yaron Koren (talk) 22:26, 9 July 2019 (UTC)

Table X not found

Hi, I'm new to Cargo and still trying to figure out how to use it. Templates that contain tables claim the tables have not been created yet, and whenever I try to query data, I get this error. I have tried following advice from an earlier topic on this thread to create the tables manually with no luck. What might I be doing wrong? Disgustedorite (talk) 18:25, 9 July 2019 (UTC)

This section might help... Yaron Koren (talk) 19:47, 9 July 2019 (UTC)

It worked, thank you! Disgustedorite (talk) 20:03, 9 July 2019 (UTC)

Wrong Lua error when join conditions are reversed

I'm doing a relatively complicated query with a bunch of joins, and I had the following error:

Lua error: A database query error has occurred. Did you forget to run your application's database schema updater after upgrading? Query: SELECT `PR`.`_pageName` AS `_pageName`,`LG`.`LongName` AS `LeagueGroup`,`IT`.`StandardName` AS `Tournament`,`SP`.`Team` AS `Team` FROM `cargo__ScoreboardPlayer` `SP`,`cargo__LeagueGroups` `LG` LEFT OUTER JOIN `cargo__PlayerRedirects` `PR` ON ((`SP`.`Link`=`PR`.`AllName`)) LEFT OUTER JOIN `cargo__Tournaments` `IT` ON ((`SP`.`OverviewPage`=`IT`.`OverviewPage`)) LEFT OUTER JOIN `cargo__LeagueGroups__Leagues` `L` ON ((`LG`.`_ID`=`L`.`_rowID`)) WHERE (LG.Leagues__full IS NOT NULL AND PR._pageName="Bjergsen" AND IT.IsPlayoffs="0" AND IT.IsQualifier="0") ORDER BY PR._pageName,LG.LongName,IT.StandardName,SP.Team LIMIT 5000 Function: CargoSQLQuery::run Error: 1054 Unknown column 'SP.Link' in 'on clause' (esports1_cluster.aurora.local.curse.us:3306) .

However the actual error is that the last join condition is written in the wrong order: LG._ID=L._rowID instead of L._rowID=LG._ID; the column SP.Link does exist and when I set up the join properly it works fine.

I'm not sure how the errors are generated so maybe it can't know what the actual issue is, but maybe add a hint that perhaps it's the order of tables in the join conditions? --RheingoldRiver (talk) 22:51, 13 July 2019 (UTC)

I don't understand - what does the order of those two columns have to do with the error? Yaron Koren (talk) 01:39, 14 July 2019 (UTC)

Yeah exactly haha, that's the error that I got when I had the order of columns in the join listed the other way. I have no idea why it's printing this error, it seems completely unrelated to the actual problem (if there even was a problem?) If you want I can recreate the error on a page, but I'm not sure how much that will help, the only thing I did to fix it was reorder that join condition I listed before --RheingoldRiver (talk) 01:45, 14 July 2019 (UTC)

Oh, now I get it. Very strange! I have no idea... this seems like a question for an SQL expert, which I am not really. Yaron Koren (talk) 15:12, 14 July 2019 (UTC)

Duplication or missing rows in tables still an issue?

Looks like I'm about to get involved in another SMW to Cargo transition. Previously we had quite some issues with duplication of rows in cargo tables, and at other times of rows not getting added to the table when creating or recreating cargo tables. Is this still an issue, or are there new and better ways to handle it now from users on the template side? Any tricks to prevent the problem?

Secondly, is it possible to get the full image showing when using table output, instead of the thumb version, without having to use a template? It would make things much easier when you have for instance small logos/icons in games, plus various data that fits well in the table format. Pangaearocks (talk) 11:49, 14 July 2019 (UTC)

That's great! I haven't heard of duplication problems in a long time, so I'm guessing that this problem is over. As for some data not getting added - this seems to be less of a problem when tables are recreated via the command line rather than the web interface, so if it's a problem I'd try using the command line script. And to show the full image, it might work to replace the field name in the "fields=" clause with "CONCAT('[[File:', FieldName, ']]')". Yaron Koren (talk) 17:02, 14 July 2019 (UTC)

Thanks, that sounds very promising. It was quite the pain before, so I'm glad it shouldn't be an issue any more :) Cheers for the image tip as well. Once we get things set up, I'll try that on a test table and see if I get it working. Of course, we may need template output anyway, it depends on what we need, but we'll cross that bridge if we get there. Pangaearocks (talk) 02:48, 15 July 2019 (UTC)

Update: Unfortunately I still have major issues with this. It is extremely frustrating to waste days on battling with Cargo storing or updating data. I'd much prefer to spend time writing templates and start updating and correcting the actual data instead. The tables sat there for an entire day with zero progress; 0 rows. Replacement tables solution didn't work either. Both tables sat there with 0 rows after 24 hours. Today I deleted all the problem tables and recreated. Two of them got rows now. Great. Created the last two. No rows there. And suddenly Cargo decided to wipe out the rows of the two formerly working tables.

I've tried heaps of null storing and purgecache of both templates and pages, but to no avail. So am back at square one now, with zero rows in any of the problem tables (about half of all tables). Really wish I had command line access, maybe that would have helped, but I don't. I don't know what the problem is, but I start having doubts about this whole SMW-Cargo transition job when I am unsure about the stability. We simply can't have data instantly disappearing like this. And that's if all the tables actually get data in the first place. Sorry for sounding frustrated, but I really am. Pangaearocks (talk) 12:19, 6 August 2019 (UTC)

Composer installation problems

The admins on the Rimworld Wiki have tried to install Cargo, but I'm told there were some problems related to Composer. The update.php script said Composer needed to be updated. They updated it, but update.php still says Composer needs to be updated. I've not tried to install Cargo myself, nor am I an admin, so I don't know exactly how the process goes, or what the problem is here. However, can Yaron or others with installation experience try to guide us a bit? From the Version page, Cargo is installed, but with version 2.0rc1. When I click on Cargo tables, this fairly generic error message pops up:
[XS33UrP9mRLsm6nBFo-jAwAAABE] 2019-07-16 16:12:02: Fatal exception of type "Wikimedia\Rdbms\DBQueryError"

No tables are actually created yet, if that is an issue here, as I don't have the right to do that (must be admin, I think). View Data works, but not Cargo tables or Drilldown.

SMW is currently installed, and as this will be a transition process, both need to be installed for quite a while. Pangaearocks (talk) 16:15, 16 July 2019 (UTC)

I don't use Composer, and I don't know much about it. Does it have to be used for Cargo? Yaron Koren (talk) 17:23, 16 July 2019 (UTC)

Hopefully not, but that is the error message they apparently got when they tried to install Cargo, or rather when they ran the update.php script afterwards. I pointed him in this direction, so hopefully he shows up when time allows. He can be more specific about the problem. I'm just an intermediary and trying to help to get things in order so we can get started properly. Pangaearocks (talk) 19:45, 16 July 2019 (UTC)

"Max display chars" parameter of cargo queries not working?

Hello everybody,

I am using the cargo version 2.1.1 (ae3950a) and trying to use the "max display chars" parameter in a query to limit the output of characters of the query result.

{{#cargo_query:tables=wiki_news
        |fields=title, description, _pageName
        |max display chars=10
}}

The field "title" is of type String and the field "description" Wikitext.

My problem is, that the "max display chars" seems to be ignored and the query result shows more than 10 characters for all fields.

Am I using the parameter in a false way? May anybody assist me in using the parameter correctly? Or is it a bug?

Thanks in advance. --Shuitavsshente (talk) 09:55, 18 July 2019 (UTC)

The "max display chars" parameter only applies for fields of type "Text" and "Wikitext" - that wasn't explained in the documentation, for some reason, so I just added a note about it now. That still doesn't explain why it's not working for your field called "description". If you go to Special:CargoTables for that table, is "description" listed as being of type "Wikitext"? Yaron Koren (talk) 16:46, 18 July 2019 (UTC)

Thanks for the fast reply. Now I got it running, but it only seems to work with no format parameter or the default format parameter "table". I tried to use it with "format=template", but this does not work. Anyway, I also expected a different behaviour if the stored characters of a wikitext field exceed the defined number of characters via "max display chars". I was expecting that the query result cuts the output after the number of maximun chars is reached and places a "show" after this maximum of characters. But instead, the output does not show any character if the maximum is exceeded and only display "show". For my use case I had to stick to a combination of string functions like {{#len:}} and {{#sub:}}. But again, thank you very much for being always very supportive and for this great extension. --Shuitavsshente (talk) 08:36, 19 July 2019 (UTC)

Right, I think I had it just hide the entire value, instead of truncating it, because there's a chance that the first part of the text can contain wikitext, or HTML tags - and leaving them unclosed could mess the display of not just that value but the rest of the page. Maybe there's a way to always get rid of such tags, or otherwise avoid this problem, but I couldn't think of one. I'll have to look into that format problem, though. Yaron Koren (talk) 13:40, 19 July 2019 (UTC)

Multiple attaches to the same template

Currently you can declare & attach a table at the same time on/from one template, but as far as I can tell you can't attach 2 or more tables at the same time. My workaround has been to make a template that's empty except for a noincluded cargo_attach and transclude this alongside the rest of the content of the parent template, so it's not REALLY a functionality loss, but I'm going through and trying to standardize all of my declarations and this just feels really messy. Could you fix this? Thanks! --RheingoldRiver (talk) 07:57, 19 July 2019 (UTC)

Cargo behaves inconsistently when wrong entries to a table are attempted to be stored

Sorry, found another issue (recently I've been redoing a lot of tables). Basically if a template is attempting to write to a table, and one of the fields it's writing is NOT in the table, then:

• On table recreate, the data will NOT get written
• On blank edit, it WILL

I can see arguments for either version of this. Personally I'd prefer it to not only not store, but also add the page to a category "Pages attempting to write to non-existing fields" or something - or display a warning like what Extension:Loops does, but at least it should be made consistent when you're recreating vs blank editing --RheingoldRiver (talk) 23:50, 20 July 2019 (UTC)

Actually I might be wrong about this, everything is suddenly really weird, I don't know what's happening...The can't attach 2 things is definitely an issue but ignore this one for now --RheingoldRiver (talk) 01:14, 21 July 2019 (UTC)

Uncaught Exception: extension.json does not exist

To get a better understanding of more of the admin stuff I have installed Mediawiki 1.33 locally (localhost). I am on a Linux Mint 19.1 system. The wiki itself installed okay (in the end) and is working. I installed Cargo 2.2 and the update.php script ran fine from what I can gather. When I check the databases in phpmyadmin I see cargo_tables and cargo_pages, which I suppose are the two cargo tables that are inserted in the main wiki db. However, the entire wiki is down when I have Cargo enabled in LocalSettings.php. At first every page was blanked, but upon inserting

error_reporting( E_ALL );
ini_set( 'display_errors', 1 );

I at least see an error message. It complains about extension.json missing -- yet I can clearly see it under /extensions/Cargo

Fatal error: Uncaught Exception: /var/www/html/mediawiki/extensions/Cargo/extension.json does not exist! in /var/www/html/mediawiki/includes/registration/ExtensionRegistry.php:117 Stack trace: #0 /var/www/html/mediawiki/includes/GlobalFunctions.php(50): ExtensionRegistry->queue('/var/www/html/m...') #1 /var/www/html/mediawiki/LocalSettings.php(167): wfLoadExtension('Cargo') #2 /var/www/html/mediawiki/includes/Setup.php(105): require_once('/var/www/html/m...') #3 /var/www/html/mediawiki/includes/WebStart.php(77): require_once('/var/www/html/m...') #4 /var/www/html/mediawiki/index.php(39): require('/var/www/html/m...') #5 {main} thrown in /var/www/html/mediawiki/includes/registration/ExtensionRegistry.php on line 117

Currently this is at the end of my LocalSettings.php file:

wfLoadExtension( 'Cargo' );
# require_once( "$IP/extensions/Cargo/Cargo.php" );
$smwgEnabledCompatibilityMode = true;
$wgShowSQLErrors = true;
$wgShowExceptionDetails = true;

Same error message if I remove the # for require_once, but if I have require_once but not wfLoadExtension then I also get this error message:

Warning: require_once(/var/www/html/mediawiki/extensions/Cargo/Cargo.php): failed to open stream: Permission denied in /var/www/html/mediawiki/LocalSettings.php on line 168

Any idea what is wrong here and how to fix it? I've searched and searched but can't get anywhere. --Pangaearocks (talk) 19:09, 22 July 2019 (UTC)

Err, okay? This confuses me almost as much as the ShortURL manual (seriously, what the hell?), but somehow it works now. I renamed the Cargo folder and extracted anew, but this time using tar in terminal. Earlier I extracted using Archive Manager (basically WinZip / 7-Zip), using the provided .zip file. Perhaps the earlier method mucked up file permissions or something. But alas, now the wiki loaded again without errors while having Cargo enabled, and I see the Cargo special pages -- and they don't yield strange error messages either. Progress! Pangaearocks (talk) 19:25, 22 July 2019 (UTC)

Child tables or a similar solution

Hi again. Thinking about some template/data restructures as part of the SMW to Cargo transition. Let's say there is a type of item, which can be purchased or sold for X amount of currency/resource A, B or C (dollars, gold, silver, whatever). It would be nice to couple this somehow without needing a plethora of fields (resource_a, resource_a_amount, etc).

Is this possible? Or would we be better off going with a simpler but not as dynamic a solution like List of Wikitext?

Naturally there would be other, similar uses as well, so we wouldn't need infobox content like this:

|attack1dmg = 7
|attack1type = blunt
|attack1part = head
|attack1cool = 120
|attack2dmg = 9
|attack2type = blunt
|attack2part = left hoof
|attack2cool = 120
|attack3dmg = 9
|attack3type = blunt
|attack3part = right hoof
|attack3cool = 120

Pangaearocks (talk) 13:46, 23 July 2019 (UTC)

Probably not the most elegant solution, but I'm trying out having attack data in a separate table, and using it directly in the article instead of the infobox. Could get crowded there anyway. Please let me know if this is dumb and there is a much better solution I haven't thought about yet.

{{Attack|bodypart=head|type=blunt|damage=99.9|cooldown=999|chance=.99|surpriseattack=yes|surprisedamage=99.9|surprisestun=99.9}} 
{{Attack|bodypart=front left leg|type=blunt|damage=99.9|cooldown=999|chance=.99|surpriseattack=yes|surprisedamage=99.9|surprisestun=99.9}} 
{{Attack|bodypart=front right leg|type=blunt|damage=99.9|cooldown=999|chance=.99|surpriseattack=yes|surprisedamage=99.9|surprisestun=99.9}} 

{{#cargo_query:tables=Attack
|fields=_pageName=Name, bodypart, type, damage, cooldown, chance, surpriseattack, surprisedamage, surprisestun
|format=table
}}

Btw, I think it would be useful to inform about the CSS that Cargo uses, so that it's easier to change it. On a dark wiki for instance, these default cargotables are virtually unreadable. --Pangaearocks (talk) 15:03, 25 July 2019 (UTC)

No, having a separate template for this is the right approach, I think. If you use Page Forms to create and edit this content, PF supports that approach via what are called "multiple-instance templates". Yaron Koren (talk) 20:49, 25 July 2019 (UTC)

Thanks. It works well for this data. As it turns out, though, there is a lot of data of this structure, from XML files with often a nested hierarchy with multiple "rows" on each level -- if that makes sense. Looks good in XML files (see example below), but I'm finding it harder to put it into Cargo tables while trying to ensure it's reasonably easy to query on the data.

<defName>TropicalRainforest</defName>
<animalDensity>5.4</animalDensity>
<plantDensity>0.90</plantDensity>
<diseases>
	<li>
		<diseaseInc>Disease_Flu</diseaseInc>
		<commonality>100</commonality>
	</li>
	<li>
		<diseaseInc>Disease_Plague</diseaseInc>
		<commonality>80</commonality>
	</li>
	<li>
</diseases>
etc...

Sometimes even more tricky, like for body parts.

<defName>QuadrupedAnimalWithPaws</defName>
<label>quadruped animal</label>
<corePart>
	<def>Body</def>
		<height>Middle</height>
		<depth>Outside</depth>
		<parts>
			<li>
				<def>Spine</def>
				<coverage>0.03</coverage>
				<depth>Inside</depth>
			</li>
			<li>
				<def>Stomach</def>
				<coverage>0.03</coverage>
				<depth>Inside</depth>
			</li>
		</parts>
</corePart>

There are yet more sub-levels beneath this. For shallow structures I may get away with de facto child tables (like above with attack), but it seems a bit messy when what is truly one table suddenly has 1+5 tables. Especially if some of that data would fit best in an infobox. And sub-tables for sub-tables? Yuck :-(

1. For body parts, if I were to jot down a colossal hierarchy section, is Cargo able to deal with it, and is it possible to have non-unique entries in hierarchy? Different types of creatures have mostly the same body parts, but they may be configured differently, so the hierarchy would need to reflect that (birds, four-leggers, humans, etc).
2. Would it be possible to query on this and get not just the parts (like spine), but somehow the tree structure + the additional data? What I hope to achieve is to get a full overview of the body composition of each creature, with other data (each body part's health is multiplied by the creature's bodysize, which I'd like to put into a table overview, if possible).
   1. Currently this is hard-coded, like shown here, and maybe it's better to just stick with that? Sounds really hard to get all this stuff sensibly into a table.
3. Suspect I'd need to go with multiple child tables (like for attack), but isn't there a better way to deal with this, or is a feature planned for structures like this? Here is one of the files, which should give a better impression of the structure than the brief sections above. Pangaearocks (talk) 18:27, 27 July 2019 (UTC)

Looks like what I am essentially running into is this: Object-relational mapping and Object-relational_impedance_mismatch :-/ Pangaearocks (talk) 19:09, 27 July 2019 (UTC)

How to store values based on a formula

Sorry for all the questions, but I can't seem to get this sorted correctly. What I would like to do is to have a backup solution for when people don't enter data in the infobox (or make it easier to mass-update by changing the formula), but not just a default value. This example is carryingcapacity, which depends on the size of the animal. Bigger animals can carry more stuff. But let's just have a default value for now. Currently I did it like this, which at least stores the "no value set" value of 50 in the Cargo table.

{{#cargo_store:_table = Animal
| carryingcapacity   = {{#if: {{{carryingcapacity|}}} | {{{carryingcapacity}}} | 50 }}
etc.

I then hoped to be able to avoid further #if tests in the output section of the template, but currently nothing will show up, with the code below. I thought this #if also set the {{{carryingcapacity}}} variable to 50, but clearly not. How would I do that, so I can just use the variable in the output without yet further #ifs there?

Currently I have this code in the output section (it's in a classic table with auto-formatting for the div):

{{#if: {{{carryingcapacity|}}} | {{!}} <div>Carry capacity</div> {{!!}} {{{carryingcapacity}}} kg }}

Some of the fields will require fairly elaborate #ifs or a #switch, so I'd prefer to avoid repeating the same code in the output section too. Do anybody know how to best go about this? Pangaearocks (talk) 18:45, 26 July 2019 (UTC)

The only way I can think of to avoid having repeated formulas is to create a local variable for each template parameter, using the extension Variables, and then use that variable in both #cargo_store and the display. Yaron Koren (talk) 18:58, 26 July 2019 (UTC)

Thank you for the advice. Hoped to avoid that, but the below appears to be working, so it turned out a more elegant solution than I imagined :)

| carryingcapacity   = {{#vardefine: carryingcapacity | {{#if: {{{carryingcapacity|}}} | {{{carryingcapacity}}} | 30 }} }} {{#var:carryingcapacity}}

Then this in the display. Don't need the #if anymore, I think, though if feels safer to have them.

{{#if: {{#var:carryingcapacity}} | {{!}} <div>Carry capacity test</div> {{!!}} {{#var:carryingcapacity}} kg }}

Now to see if I can get this working with more elaborate formulas and switches :) Pangaearocks (talk) 20:11, 26 July 2019 (UTC)

Not sure if you're deliberately avoiding this, but {{#vardefineecho:}} exists for basically this exact use case --RheingoldRiver (talk) 19:55, 27 July 2019 (UTC)

Oh, good point. Seems like a perfect fit here, so I'll go and rewrite the code. As long as vardefineecho still enables me to use var in the display section that is (edit, which it does). Cheers. Pangaearocks (talk) 21:05, 27 July 2019 (UTC)

Slideshow formatting

When using the slideshow as display format, is it possible to hide the forward/back navigation controls? On my wiki, they currently render strangely, the back looks like ← and the forward looks like → (except the dagger looks much more distinct, the left version of the image below). They are placed well below the images leaving too much white space, I've tried to use CSS to set the max-height but it doesn't seem to effect the space between the image and controls.

Sorry about that - this bug had been in place for an embarrassingly long time (eight months). I had noticed it before too, but somehow thought it was just an issue on my system. It was actually a problem in the code - I just checked in what I think is a fix for this. Yaron Koren (talk) 17:56, 2 August 2019 (UTC)

That worked perfectly, thanks Yaron!

With regards to the white space, is there any solutions you can think of? There seems to be an awkward amount of space between the slideshow images and the forward/back navigation controls. Also, even if I embed the cargo slideshow call inside a div tag with a height limit, large (tall) images seem to "escape". Bryandamon (talk) 02:57, 7 August 2019 (UTC)

_redirectTarget in _pageData?

Would this be possible to add as an allowed value that can be added to wgCargoPageDataColumns? It would let me stop using DPL for the most part, which would be amazing. It could either be null if _isRedirect is false, or return itself in that case, I'd be doing everything in Lua anyway --RheingoldRiver (talk) 19:53, 27 July 2019 (UTC)

That's a good idea, and I'm kicking myself for having gone with _isRedirect in the first place, instead of _redirectTarget, which holds more information - if _redirectTarget is used, _isRedirect isn't needed at all, I would think. I should change that field. Yaron Koren (talk) 13:50, 30 July 2019 (UTC)

I don't think it's a totally terrible idea to keep both fields although, yeah, _pageName and _redirectTarget are sufficient to fully determine _isRedirect. FWIW I'd prefer _redirectTarget never to be null, and instead return the page itself if it's not a redirect, but like I said it doesn't really matter that much to me. Can you ping me when you add this? I'll open a ticket for us to update --RheingoldRiver (talk) 07:06, 31 July 2019 (UTC)

Actually thinking about this more, I think _redirectTarget should definitely never be null, since in this case it could be used directly in a join to check if a page or any of its redirects is being referred to --RheingoldRiver (talk) 15:18, 31 July 2019 (UTC)

Setting a value for "_redirectTarget" for non-redirect pages seems like a big hack. Couldn't you just do an "OR" in your query to get both a page and the redirects to it? Yaron Koren (talk) 01:08, 2 August 2019 (UTC)

Can't use an or because it would be a join condition, to go from a page to all of its redirects to all things that reference those redirects. I don't think it's a hack though, especially since in SMW a redirect was queryable as if it were the target page; also there's the "follow redirects" option in AWB, ParserPower extension has #follow which returns a page or its redirect target, etc - redirect target in principle means the location of the data associated to that page name, I don't think it should be null ever.

To improve semantics what about replacing _isRedirect with _redirectTarget, which is sometimes null, and add _follow or _followLocation that's never null? --RheingoldRiver (talk) 07:00, 3 August 2019 (UTC)

If going with two fields (think that would be best, personally), wouldn't it be better to have the current _isRedirect (bool) and then add _redirectTarget, but have the latter be able to store nulls? Would enable us to easily get a count of yes/no, and it seems better to me to only store a page in _redirectTarget if it's actually a redirect. Pangaearocks (talk) 09:41, 3 August 2019 (UTC)

No that makes no sense. A _redirectTarget field that can store null provides 100% of the functionality that an _isRedirect field can store. And neither is helpful for the join I described. --RheingoldRiver (talk) 10:49, 3 August 2019 (UTC)

Well, you could probably still get things working with a _redirectTarget that can be null, by calling #cargo_compound_query instead of #cargo_query, and thus combining two different queries together. But maybe that's unnecessarily complicated. You could indeed have a field that's never null, and just holds the name of the page that will be displayed - whether it's the current page or the redirect target. I don't know if a name like "_followLocation" really explains it clearly - what about a name like "_actualPage"? In theory, if you had such a field, you wouldn't need anything else - "_isRedirect" would be unnecessary, since you could just check whether the name of the page is the same as that of the "actual page". Or would another field still be helpful? Yaron Koren (talk) 03:01, 5 August 2019 (UTC)

Yeah _actualPage sounds great - I'd be hesitant to remove _isRedirect altogether just since they're all opt-in fields and maybe backwards compatibility issues? But I will almost certainly only use _actualPage --RheingoldRiver (talk) 10:37, 5 August 2019 (UTC)

Actually if it's _isRedirect, what about _isRedirect and _targetPage? --RheingoldRiver (talk) 12:50, 5 August 2019 (UTC)

I don't understand - do you want _actualPage, _isRedirect and _targetPage? Yaron Koren (talk) 13:15, 5 August 2019 (UTC)

Sorry, I was suggesting _targetPage as an alternative name to _actualPage - _actualPage sounds great to remove ambiguity here, but in isolation it's not obvious what it means. Since _redirectTarget is no longer being used as a field, "target" isn't overloaded as much so the 2 fields could be _isRedirect (for compatibility) and _targetPage (the new field). --RheingoldRiver (talk) 17:57, 5 August 2019 (UTC)

Oh, okay. I'm not sure if the name "_targetPage" is all that clear either, but I don't know. Yaron Koren (talk) 20:02, 5 August 2019 (UTC)

Maybe something with an if / or in the name like _pageNameOrRedirectTarget or _pageNameOrTargetIfRedirect or _redirectTargetOrPage or something? Tbh I'll be happy with whatever you call it, I can also write up a small docs section explaining the field and why one might want to use it --RheingoldRiver (talk) 21:05, 5 August 2019 (UTC)

Alright, I just checked in code that handles this - I thought the "or" idea was pretty good, so I compromised on the field name length and went with "_pageNameOrRedirect". Hopefully it'll work for you once you get the new code. Yaron Koren (talk) 23:49, 6 August 2019 (UTC)

Thanks! I have a concern about unicode handling, if it turns out to be a problem I'll write up in detail what's going on with that, but even if that issue comes up & isn't solvable this will guaranteed let me drop almost all my DPL which is already fantastic --RheingoldRiver (talk) 15:14, 7 August 2019 (UTC)

Bug with underscores in format=template

Hi Yaron. This is an issue that I mentioned about two years ago here that I assumed had been fixed in the meantime (you recognised it was a bug), but it appears it has not? Are there any plans to fix it?

The topic from 2017 linked above is fairly detailed, but essentially what is happening is that underscores in field names are removed/replaced with spaces, which causes nothing to show up in templates if we use the actual field name. For instance name_something would turn into name something (without underscore). Since it's recommended to use underscores in field names in the documentation, this can be pretty confusing.

Looks like a workaround is to use dashes instead of underscores, albeit it doesn't look quite as nice. Anyway, I'd really appreciate this getting fixed so we can use the proper field names with underscores when using template output format. Pangaearocks (talk) 01:34, 30 July 2019 (UTC)

Don't use dashes instead of underscores, you run into a lot of annoying syntax issues. I recommend that as a blacklisted character. If you use my Module:CargoQuery from the Gamepedia help wiki this should workaround the issue - or you can use some other special character, but don't use dash --RheingoldRiver (talk) 03:12, 30 July 2019 (UTC)