Extension talk:Cargo/Archive January to February 2020

incomplete csv download

I should have 260+ rows in my csv file downloaded but I am only getting 101 rows. It is a simple one template creating a table.

What do I need to look at to fix? Thanks, Margaret

Add a "limit" value to the query/URL, like "limit=500". Yaron Koren (talk) 18:51, 3 January 2020 (UTC)

Multiple-Condition joins?

Is this intentionally disallowed in Cargo? Could it be allowed? Or am I doing something wrong? join on=(SP.OverviewPage=TP._pageName AND SP.Link=TP.Link) (or any multiple-condition join I try) returns an error message "Table and field name must both be specified in 'TP._pageName AND SP.Link'." (Also, if you think this would be easy to do, maybe I can try implementing myself) --RheingoldRiver (talk) 19:47, 4 January 2020 (UTC)

You just need to separate the two parts with "," instead of " AND ". (And take out the parentheses.) This is shown in a few of the examples for #cargo_query in the documentation, but I realized it's never actually spelled out. I just added a little bit about it to the documentation. Yaron Koren (talk) 03:24, 6 January 2020 (UTC)

Oh, that makes sense, I don't know why I didn't try that tbh, I just assumed each item in the comma-separated list had to belong to a different set of tables. Thanks! --RheingoldRiver (talk) 07:05, 6 January 2020 (UTC)

Oh, I didn't realize that these two were for the same pair of tables. It's good to know that that works. Yaron Koren (talk) 15:55, 6 January 2020 (UTC)

Oh LOL - I'm not actually 100% sure it's working properly when there's 3+ tables total involved, I was getting results other than what I expected when I did this. I ended up getting something to work but it wasn't what I initially expected to do.

{{#cargo_query:tables=ScoreboardPlayer=SP,TournamentPlayers=TP,PlayerRedirects=PR1,PlayerRedirects=PR2,TournamentPlayers=TP2
|join on=
 SP.OverviewPage=TP._pageName,
 SP.Link=TP.Link,
 SP.Link=PR1.AllName,
 PR1._pageName=PR2._pageName,
 PR2.AllName=TP2.Link
|fields=SP.Link, CONCAT('[[',SP.OverviewPage,']]')=Page
|where=TP._pageName IS NULL
|having=MAX(TP2.Link) IS NULL
|group by=SP.Link,SP.OverviewPage
}}

This was my final query, I wanted to check when a player was missing from a manual list of everyone who participated in a tournament, but the two names used could be different and I didn't want false positives. When I get a chance I'll try and recreate the version that didn't work, though this query was complicated enough that I'm unconvinced it's Cargo's fault in this case and not mine. --RheingoldRiver (talk) 06:37, 7 January 2020 (UTC)

Actually looking at this again I'm unconvinced this is what I want lol, maybe I need to play with this more still.... --RheingoldRiver (talk) 06:42, 7 January 2020 (UTC)

Ah ok I just needed one additional join condition of TP._pageName=TP2._pageName. It looks like the first copy of TournamentPlayers isn't doing anything, but actually that restricts the possible fields enough that the HAVING on a join this size doesn't time out. So, it seems actually this is fine, I probably do need this much complexity in the query. I have some cases where I created fields that were concatenations of other fields just for the sake of a join in the past, so I'll try changing those to use this method instead so I can delete the concatenation fields, I'll update if anything doesn't work. --RheingoldRiver (talk) 06:49, 7 January 2020 (UTC)

Alright. For performance reasons, it may make sense to replace some of these joins with "where" additions. For instance, that last join you mentioned may be better done by adding " AND TP2._pageName IS NULL" (if I understand the query correctly). And maybe that "having" clause can also be replaced by "TP2.Link IS NULL" in "where"? I may be simply misunderstanding the query, though. Yaron Koren (talk) 14:27, 7 January 2020 (UTC)

So I need to find an entry of SP.Link where the corresponding TP._pageName is null. That's simple enough, just put those 2 conditions in the join and set TP_pageName is null in the where. The problem is, this falsely includes players where the names don't match. So I need SP.Link -> PR1 -> PR2 -> TP.Link to match up, and also for SP._pageName = TP._pageName in this case (otherwise it could be finding a place where a player is in the participants list of one event but not another). I need the HAVING because PR._pageName -> PR.AllName is one-to-many, so if I just say WHERE TP2.Link IS NULL, it'll say "oh it's satisfied for one of the PR.AllName so let's include this" and I get a bunch of false positives. So the where TP1 is null narrows down my possible solution space, and then the having all TP2.Link be null removes all false positives from that solution space. I can't change TP._pageName=TP2._pageName to be a WHERE condition because somewhat counterintuitively, I DON'T want that to be true - I want TP2._pageName to be NULL. (TP._pageName = TP2._pageName OR TP2._pageName IS NULL in the where condition also doesn't work, for the same reason that the having is necessary - I get a bunch of false positives in which this is true for one but not all of the names available from PR. So the final query was:

{{#cargo_query:tables=ScoreboardPlayer=SP,TournamentPlayers=TP,PlayerRedirects=PR1,PlayerRedirects=PR2,TournamentPlayers=TP2
|join on=
 SP.OverviewPage=TP._pageName,
 SP.Link=TP.Link,
 SP.Link=PR1.AllName,
 PR1._pageName=PR2._pageName,
 PR2.AllName=TP2.Link,
 TP._pageName=TP2._pageName
|fields=SP.Link, CONCAT('[[',SP.OverviewPage,']]')=Page
|where=TP._pageName IS NULL
|having=MAX(TP2.Link) IS NULL
|group by=SP.Link,SP.OverviewPage
}}

and I'm pretty sure there's no way to simplify without adding in a bunch of false positives due to the one-to-many relationship. It occurs to me though that I might be better of making a new version of PlayerRedirects that has all ordered pairs of names in it, so that I can simplify queries somewhat by having only one single copy of that table in the join...maybe I'll set it up that way sometime. --RheingoldRiver (talk) 01:17, 8 January 2020 (UTC)

Multiple-Condition joins (another case)

{{#cargo_query: tables = a, b
| join on = a.p=b.p, a.q=b.q
| fields = a.p=ap, b.p=bp, a.q=aq, b.q=bq, a.x
| where = a.x="xxx" OR ( b.x="xxx" and b.y=1)
}}

I get a different number of rows depending on the order of conditions in JOIN. Only the last condition is used. How can this be explained? What am I doing wrong? --StasR (talk) 09:22, 9 January 2020 (UTC)

I'm not surprised that the order matters - small changes can make a big difference, as also seen in the section above. Are you sure that only the last "join on" condition is used? Yaron Koren (talk) 17:22, 9 January 2020 (UTC)

The order here can affect the performance, but not the result. It seems to me that the above request shall be equivalent to the SQL-query:

SELECT a.p AS ap, b.p AS bp, a.q AS aq, b.q AS bq, a.x 
  FROM a JOIN b 
  ON a.p=b.p AND a.q=b.q 
  WHERE a.x="xxx" OR ( b.x="xxx" AND b.y=1)

SQL result does not depend on the order. And it returns far fewer strings, because both ON-conditions apply. --StasR (talk) 18:48, 9 January 2020 (UTC)

Are you sure that only the last "join on" condition is used? — Yes, I sure. I made a simple illustrative example. --StasR (talk) 14:34, 10 January 2020 (UTC)

Okay, thanks for putting that together. I think the issue is that Cargo uses "LEFT OUTER JOIN" for joins, which is why you're seeing that changing behavior. Actually, just switching the left and right sides of each join will probably lead to different results too. I don't remember now why I chose "LEFT OUTER JOIN". It's certainly worth discussing - maybe a simple "JOIN" would be better. Or maybe the syntax should allow people to set the join type. (If you want to try a different kind of join, by the way, the relevant code is in /includes/CargoSQLQuery.php - there are four instances of "LEFT OUTER JOIN" that would all need to be changed.) Yaron Koren (talk) 15:17, 10 January 2020 (UTC)

"LEFT OUTER JOIN" and even exchanging the main and joined table does not change the SQL result (I checked). I think Cargo overwrites previous element instead of adding a new element on the parsing JOIN conditions. --StasR (talk) 16:35, 10 January 2020 (UTC)

Yes, indeed! You're right. I think the previous section made me a little overly-optimistic about multiple joins on the same tables. I just checked in what I think is a fix for this. Maybe no change to "LEFT OUTER JOIN" is necessary. Yaron Koren (talk) 20:02, 10 January 2020 (UTC)

After this fix is the syntax the same or will you put multiple-condition joins in one line together with AND ? --RheingoldRiver (talk) 18:26, 12 January 2020 (UTC)

No, the syntax is the same; what happened before is that earlier join conditions were just getting overwritten. Yaron Koren (talk) 00:49, 13 January 2020 (UTC)

Storing correct redirects in _pageData (again)

Would a solution for this be to store _pageData once at the normal time, so that it can be used in processing the rest of the page, and then go back and re-store it again at the very end of page save? This would require an extra blank edit for a page's categories to apply to itself of course, but it would still be a huge improvement over current. If this seems reasonable to you I'll try and implement myself. --RheingoldRiver (talk) 03:14, 10 January 2020 (UTC)

What's the problem that requires a solution? Yaron Koren (talk) 04:13, 10 January 2020 (UTC)

Oh I wrote redirects, I meant categories, sorry. When wrong categories are stored the first time they get added on pages. --RheingoldRiver (talk) 05:27, 10 January 2020 (UTC)

Oh, okay. Well, you might be in luck! The latest version of Cargo, 2.4 (just released yesterday) contains an attempted fix for the category storage problem, added in December. If/when you get the latest version, hopefully the category setting will work better for you... Yaron Koren (talk) 14:57, 10 January 2020 (UTC)

Oh yay, thanks! I'll open a ticket for us to update. --RheingoldRiver (talk) 15:10, 10 January 2020 (UTC)

Incorrect Drilldown

Cargo v. 2.3.1. The table contains 4125 rows. As can be seen on this screenshot, for all the fields, where is the complete list of values, the amount is much smaller. (I beg your pardon, porno is just an abbreviation for sequence number in Russian :-) --StasR (talk) 18:00, 12 January 2020 (UTC)

Yaron, do I understand correctly that the sum of value counters for each field must be equal to the number of rows of the table? --StasR (talk) 10:06, 14 January 2020 (UTC)

I'm guessing that the issue is the overlap. So, for example, the first "bucket" is 0-60, while the second is 60-200, so a page that had a value of 60 for that field would get listed in both. I'm not sure what's the best handling for that kind of thing. The problem with making the second bucket 61-200 instead is that there might be non-integer values - so a page that had a value of 60.5 would not show up anywhere. You could have something like ">60-200", but that might look like awkward. Or maybe the current approach is fine? This is all based on the assumption that that is indeed the issue - maybe it's something else. (Another option is that there are pages that have more than one value for this field, in different buckets.) Yaron Koren (talk) 14:34, 14 January 2020 (UTC)

The field 'locno' is integer from 1 to 4 (no NULL). And sum (23+10+20+1=54) considerably less than the number of rows (4125). Sums in all fields with a complete enumeration are very small. --StasR (talk) 16:50, 14 January 2020 (UTC)

Oh, the number within the buckets is smaller than it should be. For that "locno" field - which of these values should have a bigger amount? Or is it all of them? Yaron Koren (talk) 17:11, 14 January 2020 (UTC)

Statistics for "locno" and "event" (collapsed):

 locno 	COUNT( locno ) 	
1 	3472
2 	619
3 	33
4 	1

 event 	COUNT( event ) 	
1 	36
2 	104
3 	1057
6 	207
7 	268
8 	367
13 	466
14 	4
15 	42
16 	418
17 	2
18 	396
19 	149
20 	1
21 	377
22 	174
23 	22
24 	2
25 	30
26 	2
27 	1

--StasR (talk) 19:20, 14 January 2020 (UTC)

That's very strange... are there a lot of rows per page, or just one? Yaron Koren (talk) 19:59, 14 January 2020 (UTC)

That's very-very strange. Pages generate a large number of rows (but I do so in many projects). Writing data takes place fully and rewriting does not change these amounts. In what module generated the Drilldown results? I'll try to trace and debug it. --StasR (talk) 09:22, 15 January 2020 (UTC)

Okay, now it's less surprising. Special:Drilldown, for better or worse, breaks down the data by pages and not rows. So if there are 10 rows in a page all matching a filter value, that will show up as one result, not 10, in both the filter listings and the actual results. (At least, that's how it's supposed to work.) Does the data make more sense now? Or are the numbers still off? Yaron Koren (talk) 14:15, 15 January 2020 (UTC)

Then it explains everything, thank you! --StasR (talk) 18:15, 15 January 2020 (UTC)

Great. By the way, if you can think of some way to make this clearer in the interface - or, for that matter, if you think the results should change to refer to rows and not pages - let me know. Yaron Koren (talk) 14:00, 16 January 2020 (UTC)

Drilldown

I have noticed that in Drilldown the filter options end up being what you put in the parameters of a template. But that's a problem. If I put yes in a parameter called Presidential, it will appear as "yes" instead of "Presidential". Is there a way to correct that? --Hispano76 (talk) 00:48, 20 January 2020 (UTC)

Do you have an example demonstrating this? Gryllida 01:11, 20 January 2020 (UTC)

Something like this would be the use of the template

On code: {{#if: {{{presidential|}}} | {{{presidenctial|}}} }}

On infobox in page: | presidential = Sí

--Hispano76 (talk) 01:19, 20 January 2020 (UTC)

Just to clarify: in Special:Drilldown, you want the filter called "Presidential" to also have the value "Presidential"? If so, then within the #cargo_store call you can have something like |Presidential={{#ifeq:{{{presidential|}}}|Sí|Presidential|Not presidential}}. Maybe I'm misunderstanding what you're asking for, though. Yaron Koren (talk) 15:42, 20 January 2020 (UTC)

Thanks @Yaron Koren: It's partially working. Even if it says presidential= yes it will still appear in the office as Not Presidential But in the title of the filter and the options they look correct but badly listed. I would like a solution to use this in other parameters/filters/options. --Hispano76 (talk) 19:19, 20 January 2020 (UTC)

That was my mistake, it's working now. Thank you very much. --Hispano76 (talk) 00:12, 21 January 2020 (UTC)

Suggestion to stop aliasing "_pageName" as (nstab-main) at Special:CargoTables/in queries

In this view where it says Page, I think it should say _pageName instead, overall it would be less confusing, I think "how to get page column" is the most frequently asked Cargo-specific question I get from people trying to use my API. I'm gonna update my docs to emphasize this, but figured I'd mention it here too --RheingoldRiver (talk) 20:40, 23 January 2020 (UTC)

I'm glad we're discussing this. In addition to _pageName, there are also the pre-defined tables, _pageData and _fileData, which of course are composed entirely of pre-defined columns. In the case of those columns, I actually want to add descriptive aliases - since a string like "_creationDate" does not mean anything to someone who doesn't speak English. But of course it's also useful to see the actual column name - something I hadn't though enough about before. Maybe the best solution is to show both, and have each such column header look something like "Page (_pageName)". What do you think?

By the way, as far as I know the "Page" alias only shows up in Special:CargoTables, not in regular queries. Is that not true? Yaron Koren (talk) 01:56, 24 January 2020 (UTC)

Ah, yeah you're right, only that one view. I did want to talk to you eventually about adding native support for the docstrings that atm I'm showing via Lua/ajax workaround on table pages (example) - maybe setting default docstrings for these fields along with calling the field name by the real name in the actual table would work? I'm not sure how Cargo would store these docstrings though, maybe one internal table that has table name, field name, and docstring could be used. --RheingoldRiver (talk) 21:13, 24 January 2020 (UTC)

We did actually already talk about the descriptions thing, here. I think it's an unrelated issue, though... unless you think descriptions should be added to the column headers as well (as a hover thing, maybe?). In any case, I still like my "Page (_pageName)" idea. Yaron Koren (talk) 00:59, 27 January 2020 (UTC)

Tbh I don't like that, I think it's obvious what it means only if you already know what it means, at which point you don't need the extra descriptor. Having a full docstring above the table where it says "table stucture" currently is 99% whitespace would allow for enough space to actually explain what the field names mean, I feel pretty strongly that's the correct way to do this and also should be extended to allow users to add their own docstrings to non-default fields. --RheingoldRiver (talk) 00:25, 30 January 2020 (UTC)

Really, you think table headers with columns like "_creationDate", "_modificationDate", etc. are good enough, even for people who can't speak English, as long as those terms are defined above the table in their language? What if those terms were in Chinese - would that setup still be good enough? Yaron Koren (talk) 01:11, 30 January 2020 (UTC)

Yeah, cos the parser function / what you query the fields isn't translated. Unless you want to build in support for queries to use the translated name, I don't think there's any need to show anything than the actual names of the fields as they are used when you write the queries, cos that information is there for developers. If the field names were in Chinese it could be assumed anyone looking at the table page knows enough Chinese to code in it (or at least is used to doing things like counting the number of rows down in the docs and columns across in the table) --RheingoldRiver (talk) 02:27, 30 January 2020 (UTC)

Hm, you might be right... there is a certain symmetry in having the "field description" part get used for both user-supplied descriptions (for regular fields) and built-in translations/descriptions (for built-in fields). Yaron Koren (talk) 04:12, 30 January 2020 (UTC)

Error in "where" parameter: the string "#" cannot be used within #cargo_query.

How come this is the case? I wanted to search for tournaments whose names have a # in them because they're probably causing issues with some links. --RheingoldRiver (talk) 18:36, 2 February 2020 (UTC)

That restriction is there to prevent people from doing something malicious, because "#" is used to specify a comment in MySQL. I don't know of a way to allow these that wouldn't potentially enable malicious behavior. Maybe there's a way, but until one is discovered, I'm afraid you'll have to... not search on that. Yaron Koren (talk) 03:33, 3 February 2020 (UTC)

Ah that makes sense, thanks. --RheingoldRiver (talk) 04:07, 3 February 2020 (UTC)

Cargo 2.4 failing to store categories in _pageData

On gbf.wiki we recently upgraded to Cargo 2.4 (2e93e5c) and a new issue has cropped up.

Cargo doesn't always save all categories on a page to _pageData.

• Pages with missing categories will only add any new or renamed categories to _pageData when updated. Old missing categories remain missing.
• Creating a new page with exactly the same categories is usually stored correctly in _pageData.
• Null editing a page does not fix _pageData, and on pages without issues sometimes causes categories to disappear.
• We do not know what causes the categories to disappear from _pageData in the first place. These pages worked fine before 2.4.

I created a new category "Cargo Issue" for the known affected pages, but it seems to be spreading. Any help on tracking down the issue and what extra info we could provide to help solve it would be greatly appreciated.

Sorry about that - the saving of categories did indeed change, in an effort to improve it - clearly it hasn't always led to improvements. For now, I think there are two ways to get a page's category data to be correct: remove all of the page's categories, save the page, then add them back in and save again; or just call the setCargoPageData.php script, which should result in correct category data for all the pages. Yaron Koren (talk) 14:14, 4 February 2020 (UTC)

If it's not working as-is I think saving _pageData twice in the course of a page save, once at the start, and then again at the very end would work without being too expensive an addition. --RheingoldRiver (talk) 19:03, 4 February 2020 (UTC)

The problem is, even saving it at the end of a page save (or waiting 10 seconds to save it, or some such) wouldn't necessarily work - the category data can take a long time to get saved. I don't know what the best approach is for saving category data. Yaron Koren (talk) 19:42, 4 February 2020 (UTC)

While running setCargoPageData.php we possibly found the cause of another error that has been confounding us. The error we get is:

>Wikimedia\Rdbms\DBQueryError from line 1587 of /srv/gbf.wiki/mediawiki/includes/libs/rdbms/database/Database.php: A database query error has occurred. Did you forget to run your application's database schema updater after upgrading?
Query: SELECT  COUNT(*)  FROM cargo___pageData    WHERE _pageID = '3622' AND _creationDate = '2016-09-25 8:58:38' AND _modificationDate = '2019-07-27 10:37:06' AND _creator = 'AdlaiT' AND SUBSTR(_categories__full, 1, 300) = '5★_Characters|Attack_Characters|Characters|Characters_with_Blush_Value|Characters_with_Story_Art|Extended_Mastery_Characters|Human_Characters|Light_Characters|Male_Characters|Melee_Characters|Pages_using_DynamicPageList_parser_function|Premium_Draw_Characters|SR_Characters|Voice_Actor:_Sōma_Sait AND _numRevisions = '65' AND _isRedirect = '0' AND _pageNameOrRedirect = 'Feather (SR)'
Function: Wikimedia\Rdbms\Database::select
Error: 1300 Invalid utf8mb4 character string: 'C5' (10.0.2)

It appears to me like perhaps the 299 byte long right side of the SUBSTR comparison is cropped using php substr which is not multi-byte safe. The full VA category is "Voice Actor: Sōma Saitō", note the missing ō at the end and how it seems to have swallowed the ' in the query.

This also affects the _fullText column if byte 1000 is part of a multi-byte character. Ypnocsd (talk) 21:30, 4 February 2020 (UTC)

Sorry about that! I just checked in what is hopefully a fix, by replacing the substr() call with mb_substr(). Yaron Koren (talk) 21:02, 5 February 2020 (UTC)

We've updated and this does indeed fix the invalid utf8mb4 characters.

However, it still seems like any edit will clear most or all categories in _pageData for the page and even after waiting 12+ hours remain gone. Extra troublesome is that we can't use null edits to fix the problem but afaik have to run setCargoPageData.php again to fix the categories.

Is there anything else we can do? Ypnocsd (talk) 14:07, 7 February 2020 (UTC)

You can make sure the job queue is not backed up, by running the runJobs.php script. Yaron Koren (talk) 15:13, 10 February 2020 (UTC)

The job queue is getting executed in a timely manner but perhaps Cargo is making assumptions about which order jobs are executed? Are there job types that must be run before cargoPopulateTable?

We've tried running the job queue serially and it does not appear ta make any difference. _categories are wiped from _pageData instantly never to return, making me think the job queue isn't even involved. Ypnocsd (talk) 12:47, 11 February 2020 (UTC)

Storage of categories definitely uses jobs - I'm pretty sure that, if you disable the running of jobs, the set of categories won't change. No, the order in which jobs are run shouldn't matter. I don't know why categories are getting deleted. Is there any pattern to which categories get deleted and which don't? Yaron Koren (talk) 14:13, 11 February 2020 (UTC)

Existing categories seem to always disappear when doing a null edit. (Both via Page Edit and Api Edit.)

New pages seem to be stored correctly. Sometimes large edits also get stored correctly.

If you only add one or two categories to an existing page they sometimes get added. But will usually disappear if another category is added. Or some mix of new categories will be stored. New categories have a higher survival rate than old ones.

Whether user is registered or not does not seem to matter.

We're currently using 2.4 (ce4ef66) but will ask admin to update to latest. Ypnocsd (talk) 16:03, 11 February 2020 (UTC)

Through additional investigation, I've found the following:

Sample request:

1. _pageData._pageID=16145 has _pageData._ID=87239 and 4 existing categories from cargoPopulateTable
2. I add a new category to _pageData._pageID=16145 and observe a call to deletePageFromSystem for _pageData._pageID=16145.
3. Inspecting _pageData._pageID=16145 it now has _pageData._ID=109638.
4. _pageData___categories now contains the 4 old categories under _rowID=87239 and 1 new category under _rowID=109638.

• The addOrRemoveCategoryData hook is correctly called. It correctly adds or removes categories that have changed.
• However, addOrRemoveCategoryData does not take height for the case where _ID has changed.
• When making a null edit _categories__full is never updated and will contain NULL.
• It seems to me like a possible solution would be to use _pageID rather than _ID in _pageData___categories. However, this does not solve _categories__full being NULL after edit with no changed categories.

Ypnocsd (talk) 13:22, 12 February 2020 (UTC)

Thanks for doing that thorough investigation - it's very helpful. Using _pageID rather than _ID is an interesting idea, but I think it would cause complications of its own - plus, as you note, it wouldn't solve the _categories__full problem. I think I understand the issue now, though: on your system, the add/remove categories hooks are getting called before the page gets saved. I had assumed that they will always get called afterwards. Maybe it's due to some setting on your system (perhaps in your case, the category stuff is not being done via jobs at all, but just as a standard part of the save process). Or maybe the jobs are just running very fast. In any case, I think that would explain the outcomes you're seeing. I see a potential fix - if you can, please try changing line 292 of Cargo.hooks.php (or CargoHooks.php, depending on your exact version of the code) from:

CargoPageData::storeValuesForPage( $wikiPage->getTitle(), $useReplacementTable, false );

...to:

CargoPageData::storeValuesForPage( $wikiPage->getTitle(), $useReplacementTable, true );

...and let me know if that improves things at all. Yaron Koren (talk) 15:14, 12 February 2020 (UTC)

Yes this seems to fix the immediate issue for us. The only new "feature" I noticed was one time all categories got stored twice in _categories/_categories__full when viewed via Special:CargoQuery. Null editing the page fixed it, and I don't think the duplicates would cause any issues for us.

Some more info about our job queue since it may be of interest. Normally we run it with php runJobs.php --maxjobs 500 --procs 6. However, when doing this the cargoPopulateTable kept throwing duplicate _ID exceptions so we now run that job type separately in one process (assuming it is a concurrency issue).

While trying to solve the categories issue our admin removed the --procs 6 and the separate cargoPopulatetable call (so php runJobs.php --maxjobs 75) but it did not fix the issue. Ypnocsd (talk) 18:26, 12 February 2020 (UTC)

Internal error on Recent Changes after running cargoRecreateData.php

On one of my wikis I get error "Internal Error" on the "Recent Changes" page after I run cargoRecreateData.php. I initially thought this was a MediaWiki core problem, see https://phabricator.wikimedia.org/T241302 for my bug report and additional info there. It turns out Recent Changes breaks because cargoRecreateData.php creates entries in table recentchanges where rc_title is blank (ie SELECT * FROM `recentchanges` where not (rc_title > "") order by rc_timestamp DESC is not empty). The records are:

recentchanges

rc_id	rc_timestamp	rc_actor	rc_namespace	rc_title	rc_comment_id	rc_minor	rc_bot	rc_new	rc_cur_id	rc_this_oldid	rc_last_oldid	rc_type	rc_source	rc_patrolled	rc_ip	rc_old_len	rc_new_len	rc_deleted	rc_logid	rc_log_type	rc_log_action	rc_params
40968	20200204172054	17	0		1	0	0	0	0	0	0	3	mw.log	2	127.0.0.1			0	513	cargo	recreatetable	<a href="https://< deleted >/index.php/Special:CargoTables/allDates">allDates</a>
40967	20200204172038	17	0		1	0	0	0	0	0	0	3	mw.log	2	127.0.0.1			0	512	cargo	recreatetable	<a href="https://< deleted >/index.php/Special:CargoTables/SE">SE</a>
40966	20200204172030	17	0		1	0	0	0	0	0	0	3	mw.log	2	127.0.0.1			0	511	cargo	recreatetable	<a href="https://< deleted >/index.php/Special:CargoTables/Person">Person</a>
40965	20200204172019	17	0		1	0	0	0	0	0	0	3	mw.log	2	127.0.0.1			0	510	cargo	recreatetable	<a href="https://< deleted >/index.php/Special:CargoTables/ClinicalShift">ClinicalShift</a>

If I delete the changes, Recent Changes works again, but running the .php command again breaks recent changes again. I run that .php command on a crontab in several other wikis, and none of them have this issue. Tenbergen (talk) 17:45, 4 February 2020 (UTC)

It looks like there was some change in MW 1.34 so that now Cargo's logging gets added to the "recentchanges" table (it shouldn't). If you don't mind, could you try changing line 1160 of /includes/CargoUtils.php from:

			$log = new LogPage( 'cargo' );

...to:

			$log = new LogPage( 'cargo', false );

...and see if that fixes the problem, if you delete those entries and run the script again? Yaron Koren (talk) 20:57, 4 February 2020 (UTC)

Yes, that fixed the problem: I deleted the lines with empty title in the table, re-ran cargoRecreateData.php and the lines did not re-appear. It sounds like something that should have affected all the other wikis in the same way, wonder why it only became a problem on this one. Will you apply the edit to the master of future updates of Cargo, or do I need to add the edit to may checklist for updates? Thanks, Yaron! Tenbergen (talk) 20:51, 12 February 2020 (UTC)

Okay, great. I don't know either why it was only a problem some of the time! But I just checked this fix in to the Cargo code. Yaron Koren (talk) 21:38, 12 February 2020 (UTC)

Turns out this didn't fix the issue after all, when I logged on today the error was back. When I checked the recentchanges table it again had entries with rc_title="". I just looked at another wiki and it also contains entries with rc_title="", but no error in recent changes.

Custom warning on a table or field not existing for cargoquery api

This is maybe something to talk about at emwcon but I have a lot of people using my API at this point, and not a great way to communicate with everyone. When I make changes to my tables I'd really like to be able to set my own warnings for certain Cargo failures...for example if a table doesn't exist then have it try to print (api-cargoquery-tablenotexists-TableName) or if a field doesn't exist then (api-cargoquery-fieldnotexists-TableName-FieldName) so that I can write custom messages there (and if there's no such message then just print the default). How doable would it be to detect some specific error types and try and print a system message for those instead of the actual SQL warning? Is that a reasonable thing to do? What other warnings should maybe get custom messages? --RheingoldRiver (talk) 19:14, 4 February 2020 (UTC)

That's an interesting question. How unique would these error/warning messages be? If they would all essentially take the form of "Field X was renamed to Y", there may be an easier solution than creating custom messages for each one. Yaron Koren (talk) 19:45, 4 February 2020 (UTC)

I'd probably want them to be pretty unique - for example, one of the changes I'm planning is to delete a table called Disambiguations and replace it with a table called Entities where you tell if a page is a disambig by checking, is EntityName equal to _pageName - I'd want to explain this in my warning of any query that uses the Disambiguations table currently. I also have another table that currently has two fields, RosterNames and RosterLinks, and I want to delete RosterNames with the understanding that in RosterLinks if you pattern match out (%) you get the name (it took me over a year to realize this for disambiguation links lol). So that notice would say "please use RosterLinks and..." instead of just a simple field replacement.

Another option though, is to have an overrideable i18n - e.g. try to print (api-cargoquery-tablenotexists) unless api-cargoquery-tablenotexists-TableName is defined, etc. Then the requirements on the user are lessened, while still having better error messages than the default. Though how to communicate what new field name corresponds to what old field name, or table name, etc, seems pretty complicated to me, I'm not sure realistically how much could be done other than letting the user just write free text messages. --RheingoldRiver (talk) 22:08, 4 February 2020 (UTC)

What about turning the "api-cargoquery-tablenotexists" into a long piece of text, explaining everything? Like "If you are looking for the table A, you'll need to ... If you are looking for field B in table C, you'll need to ...". The advantage of that is that it's very easy to make changes to - plus, it doesn't require any coding changes. :) Is that feasible? Yaron Koren (talk) 04:21, 5 February 2020 (UTC)

Is there a character limit? If not then that would totally work, I could just include a changelog from the past 90 days or something. --RheingoldRiver (talk) 04:14, 6 February 2020 (UTC)

I'm not aware of any real character limit. Yaron Koren (talk) 17:15, 6 February 2020 (UTC)

Ok sounds good then. Is there a system message already I can edit, or by no coding change did you just mean no major change? If so what's the message? (not sure how to send an api query in qqx to check). --RheingoldRiver (talk) 18:46, 6 February 2020 (UTC)

Oh... for some reason, I thought "api-cargoquery-tablenotexists" was an actual error message. Turns out that no such thing exists, unfortunately. So maybe some coding will be required. By "API", do you mean the actual API at api.php, or Special:CargoQuery/Special:CargoExport, or all three? Yaron Koren (talk) 20:00, 6 February 2020 (UTC)

Oh haha no that was just a hypothetical system message name. I only care api.php, but I guess the special pages would be good to have the same thing at. For my purposes, having any field in a "failure" api response contain this system message would be sufficient, so if it's easy to just add a field called "error_help" or something and include that alongside the actual error message, for any error at all, would totally work. --RheingoldRiver (talk) 21:15, 6 February 2020 (UTC)

2.4 patch caused every table with a 0 in an integer field to store duplicates

Maybe there were extra criteria for making it store a dupe, but we had to roll the patch back (gg my page-move fix, atm I have a cron task blank editing all new redirects from page moves lol). Gamepedia forked a while ago to fix some issues on our platform specifically so it's possible it's only a problem for us. But do you know any changes you might have made that could cause this? Is it a global issue or just us? --RheingoldRiver (talk) 18:49, 6 February 2020 (UTC)

Oops topic should say row, not table --RheingoldRiver (talk) 18:50, 6 February 2020 (UTC)

Sorry, yes, this is a real bug - introduced right before version 2.4 came out. I just checked in a fix for this. Maybe the admins there can add it manually... it's just a one-character change. Yaron Koren (talk) 16:30, 10 February 2020 (UTC)

Ah thanks! I opened a new ticket on our end to retry the upgrade (though this time I'll test it on staging first) --RheingoldRiver (talk) 04:25, 12 February 2020 (UTC)

Using Extension:Cite with Cargo

I would like to be able to add citations to template data stored in cargo fields, but without the citation itself getting stored in the field. For instance, a page has a table that displays fields gathered from Extension:Page Forms, and next to some fields are reference numbers that link to the footer where references are displayed. In this way a user could edit the source to add references rather than me needing to include a unique reference field to accompany every field.

Example page data that calls a template:

{{Movie
|studio=Universal
|title=King Kong
|releasedate=1933 <ref>https://www.imdb.com/title/tt0024216</ref>
|language=English
}}

Example Template:Movie:

<noinclude>
{{#cargo_declare:
_table=Movies
|Studio=Page
|Title=Page
|Release_Date=Date
|Language=String
}}
</noinclude>

<includeonly>
{{#cargo_store:
_table=Movies
|Studio={{{studio|}}}
|Title={{{title|}}}
|Release_Date={{{releasedate|}}}
|Language={{{language|}}}
}}

{| class="wikitable"
|Studio
|{{{studio|}}}
|-
|Title
|{{{title|}}}
|-
|Release Date
|{{{releasedate|}}}
|-
|Language
|{{{language|}}}
|}

==References==
<references />
</includeonly>

--AgentIrons (talk) 00:04, 13 February 2020 (UTC)

I think the best option for this, though it might sound like overkill, is to have a separate field for the reference(s) for each field - like "Studio_References", "Release_Date_References", etc. Then the template can display them together, while still storing them separately in Cargo (if the references even need to get stored). It might make the data entry easier too. Yaron Koren (talk) 03:37, 13 February 2020 (UTC)

Duplicate _ID when storing data

I've been trying to figure out why we keep getting duplicate _ID values on gbf.wiki and I found this piece of code in CargoStore.php line 317:

>		// We put the retrieval of the row ID, and the saving of the new row, into a
		// single DB transaction, to avoid "collisions".
		$cdb->begin();

		$res = $cdb->select( $tableName, 'MAX(' .
			$cdb->addIdentifierQuotes( '_ID' ) . ') AS "ID"' );
		$row = $cdb->fetchRow( $res );
		$curRowID = $row['ID'] + 1;
		$tableFieldValues['_ID'] = $curRowID;

... calculate what to insert ... which takes time making it more likely to get a collision ...

		// Insert the current data into the main table.
		CargoUtils::escapedInsert( $cdb, $tableName, $tableFieldValues );

		// End transaction and apply DB changes.
		$cdb->commit();

As far as I know this will not guarantee a unique _ID in MySQL/MariaDB with the default InnoDB settings (REPEATABLE READ) as the SELECT statement produced by MediaWiki doesn't seem to specify FOR UPDATE which is needed to lock the row from being read by other concurrent transactions.

Please see https://dev.mysql.com/doc/refman/8.0/en/innodb-transaction-isolation-levels.html#isolevel_repeatable-read and https://mariadb.com/kb/en/mariadb-transactions-and-isolation-levels-for-sql-server-users/

To explain, as far as I can see this happens:

PHP1: BEGIN;
PHP1: SELECT MAX(_ID) FROM cargo___pageData;
PHP1: 103575
PHP2: BEGIN;
PHP2: SELECT MAX(_ID) FROM cargo___pageData;
PHP2: 103575
PHP1: INSERT INTO cargo___pageData (_ID, ...) VALUES (103576, ...);
PHP1: COMMIT;
PHP2: INSERT INTO cargo___pageData (_ID, ...) VALUES (103576, ...);
PHP2: Exception

And for MySQL/MariaDB I believe this is how to do it correctly:

PHP1: BEGIN;
PHP1: SELECT MAX(_ID) FROM cargo___pageData FOR UPDATE;
PHP1: 103575
PHP2: BEGIN;
PHP2: SELECT MAX(_ID) FROM cargo___pageData FOR UPDATE;
PHP1: INSERT INTO cargo___pageData (_ID, ...) VALUES (103576, ...);
PHP1: COMMIT;
PHP2: 103576 <--- SELECT query was blocked until previous transaction finished
PHP2: INSERT INTO cargo___pageData (_ID, ...) VALUES (103577 ...);
PHP2: COMMIT;

You can test this by launching up two concurrent mysql command line sessions and running the commands manually.

Looking at the mediawiki source code there's a 'FOR UPDATE' option for select which may help https://doc.wikimedia.org/mediawiki-core/master/php/classWikimedia_1_1Rdbms_1_1Database.html#a3b03dd27f434aabfc8d2d639d1e5fa9a It does mention something about not working with MAX but in my manual testing it did so not sure about the details... Ypnocsd (talk) 18:45, 13 February 2020 (UTC)

Thanks for this diagnosis! It makes a lot of sense. And sorry for the delay. I just checked in a patch, here, based on your suggestion. I hope this fixes the problem... Yaron Koren (talk) 18:47, 24 February 2020 (UTC)

Line return appears before each Cargo query result [RESOLVED]

I had Cargo queries within wikitext * bullet points, in the middle of the bullet point's content, and it all displayed fine in Cargo 2.2.

On moving to Cargo 2.4 each Cargo query result began with a line return, which broke the bullet point format.

I am working round this problem by going through all the queries and adding |format=template|template=Cargo trim to them, where Template:Cargo trim contains just {{{1}}}. I'm not entirely sure why this works.

Maybe the default "list" output format has been changed. If this is a necessary change for another reason, then could you maybe add a new "plain text" output format?

I have a feeling the problem existed before but something fixed it before the query result was displayed. I think this because (under Cargo 2.2) sometimes when a bullet point was sent to Twitter an extra line return appeared, which did not appear on the wiki page.

Apologies in advance if I am barking up the wrong tree. Jonathan3 (talk) 20:41, 13 February 2020 (UTC)

No, I think that's a real issue - I'm guessing that's due to this very recent change. (I'm assuming you're using the very latest code, not the official version 2.4.) For some reason, I didn't think putting a newline before each value would have any bad side effects. I didn't think about queries being placed inside a bulleted item. I'll have to think about the best way to fix this... maybe a new parameter is needed. Yaron Koren (talk) 20:58, 13 February 2020 (UTC)

Yes I'm using the latest code. When I removed the '*' the query result still appeared on a new line (rather than as part of the paragraph). I guess this was because you are adding \n\n to the value. I'm happy with things for now and will reply to any further comments here. Jonathan3 (talk) 21:18, 13 February 2020 (UTC)

P.S. There were more consequences than I have time to work round so I've moved "back" to official version 2.4 :-) Jonathan3 (talk) 21:35, 13 February 2020 (UTC)

Oh, I didn't realize that both added newlines would get reflected in the output. (Though it seems obvious now.) I just checked in a fix so that only one newline gets prepended to the value, instead of two - hopefully that solves this problem. Yaron Koren (talk) 04:10, 14 February 2020 (UTC)

I've removed the workaround and can confirm that it's all back to normal. Thank you for the quick response. Jonathan3 (talk) 14:28, 23 February 2020 (UTC)

Can we revisit this please? I may have mistakenly said it worked again. Here it is: git checkout 2.4 (HEAD is now at b181454... Version 2.4) - works fine. git checkout master - error resurfaces. Let me know if you need any more details. Jonathan3 (talk) 20:33, 25 February 2020 (UTC)

@Yaron Koren: I've just checked and (a) a single line return breaks a * bullet point and (b) __NOTOC__ doesn't need a line return (HE__NOTOC__LLO displays HELLO). Would this mean you can alter your code again to help me out with my bullet points? Thanks in anticipation! Jonathan3 (talk) 22:17, 4 March 2020 (UTC)

That's true. I actually checked in another change/fix to that "NOTOC", etc. change yesterday - could you get the latest code and see if it works any better for you? Yaron Koren (talk) 03:32, 5 March 2020 (UTC)

Yes, it all works now - thanks! :-) Jonathan3 (talk) 09:22, 6 March 2020 (UTC)

Using two different templates to store data in the same row of a table

I have some legacy content I want to store in Cargo. Because it's legacy content, some of the data is dealt with by template A and one by template B. If I understand right, I should be able to declare the Cargo table in e.g. template A, and then attach to the table from template B and store data in a field. This isn't working right for me, instead of adding data to the same row, template B adds a new row of data. Can I even do this, just have it update one field?

template A

 
noinclude 
{{#cargo_declare:
_table = t
|fieldA = String
|fieldB = String
|fieldC = String
}}
includeonly
{{#cargo_store:
_table = t
|fieldA = x
|fieldC = z
}}

template B

 
noinclude 
{{#cargo_attach:
_table = t
}}
includeonly
{{#cargo_store:
_table = t
|fieldB = y
}}

Unfortunately, that can't be done - every call to #cargo_store adds a new row to a table. It sounds like the best option for you is to have these two templates populate two different tables, and then join them in queries, based on _pageID or some such. Yaron Koren (talk) 04:13, 14 February 2020 (UTC)

Possible to map list to radio button on PageForm?

I'm trying to map Status=List (,) of String (allowed values=Implemented,Planned ,Not Applicable) to a radiobutton field type in a form. Unfortunately I only seem to be able to get a list of checkboxes. Any thoughts on the best way to force a single value for a list?

I have no idea why you'd want to do that, but I think you can do it with just "input type=radiobutton" in the form definition. Or does that not work? Yaron Koren (talk) 15:05, 19 February 2020 (UTC)

Maybe it's for this: Extension:Page_Forms/Input_types#"show_on_select"? Jonathan3 (talk) 14:29, 23 February 2020 (UTC)

Query fields with template wikitext

I'm creating a page form that processes a cargo query to display a large table. PageForms apparently doesn't really like processing row results with wikitext templates with any format. So I turned instead to editing the fields parameter of my original cargo_query, which according to the documentation is able to process wikitext. Unfortunately, that seems to be only partially true, since templates aren't being handled. Even a simple call doesn't work properly, i.e., it always evaluates to the same result

 Concat('{{#ifeq:',Mytable.field1,'|val|1|0}}')

Nested templates simply return an error. Is there a solution to this?

My version is MW 1.30.1, PHP 5.5.9, MySQL 5.7.25

Is the idea that the actual form syntax (field tags and such) is generated via a Cargo query? If so, your best bet is probably to use the Scribunto extension, and generate the form text via a Lua module that in turn calls the relevant query. Yaron Koren (talk) 14:14, 21 February 2020 (UTC)

No the idea is to apply formatting on the queried data using templates, i.e. link to different pages dependent on returned content. Similarly, in a separate effort, I'm trying to use templates to create different WHERE clauses in a cargo query, so that I can call the query from a template and dynamically set different WHERE clauses according to the template parameters. Thisi also doesn't work, for the same reason I suspect.

I don't fully understand your first question yet but in relation to your separate effort, I've got templates whose parameters go to Cargo queries, so this sounds like it should work. I'd be interested in seeing more details of what you've tried. Jonathan3 (talk) 14:33, 23 February 2020 (UTC)

Openlayers map appears in front of Foreground skin drop-down menu

If z-index is changed from 749 to 1 (using the browser console) this problem is resolved. What would be the best way to change this properly? Thanks. Jonathan3 (talk) 14:40, 22 February 2020 (UTC)

Adding #OpenLayers_Map_2_OpenLayers_Container { z-index:1 !important } to MediaWiki:Common.css seems to work... though I'd be interested to hear whether there might be other complications! Best wishes, Jonathan3 (talk) 14:47, 22 February 2020 (UTC)

Satellite as default view in Google maps format

Is this possible? If not, would it be possible to add it as an option? Jonathan3 (talk) 15:51, 22 February 2020 (UTC)

It could be added as an option, I just didn't think it would be that useful. Is it? Yaron Koren (talk) 04:40, 24 February 2020 (UTC)

It would be nice (as it's so common to have the standard view and rare to have the satellite view) but overall I admit it probably isn't useful enough :-) Jonathan3 (talk) 15:24, 24 February 2020 (UTC)

Don't let me put you off though :-) Jonathan3 (talk) 21:22, 25 February 2020 (UTC)

Rearrange Drilldown page

I'd really like to be able to change the Drilldown page so that the filters are on the left and the results are on the right (or on a small screen, top and bottom as it is currently).

If you could possibly put these two elements into separate identified divs I am sure that this would be simple enough. Would that be all right? I would be very grateful. Jonathan3 (talk) 00:03, 23 February 2020 (UTC)

Maps functionality on Drilldown page

When I use Openlayers Maps or Google Maps on the Drilldown page, the coordinates are plotted correctly with red markers, but it's not possible to click on them. Is this the way it is meant to work, or a problem on my site? Ideally I'd like them to work as on normal query results, i.e. click on them and you can get to the relevant Cargo page. Thanks. Jonathan3 (talk) 00:12, 23 February 2020 (UTC)

That's definitely not intentional! I don't know how that bug came in, or when. I just checked in what I think is a fix for this. Yaron Koren (talk) 18:50, 24 February 2020 (UTC)

I've downloaded the latest code and it still doesn't work (the clustering on the Google maps does work though maybe it worked before). I use Chrome but checked with Internet Explorer (incidentally the Openlayers map appears all-pink on IE). Thanks. Jonathan3 (talk) 21:57, 24 February 2020 (UTC)

The content of the CargoMapData span has these:

{"name":"a,b","title":null,"lat":b,"lon":a,"otherValues":[]}

I wonder whether otherValues should contain something. Jonathan3 (talk) 22:06, 24 February 2020 (UTC)

Oops, sorry about that! I accidentally fixed the bug only for fields named "Coordinates". I just checked in what is hopefully a real fix. Yaron Koren (talk) 02:15, 25 February 2020 (UTC)

It's worse now, I'm afraid. git checkout 2.4 - maps appear but can't click markers (as above). git checkout master - MySQL error message on Drilldown pages with Googlemaps or Openlayers maps tab selected.

The error is

/Special:Drilldown/Contact?tab=Google%20map Wikimedia\Rdbms\DBQueryError from line 1457 of xxx/Database.php: A database query error has occurred. Did you forget to run your application's database schema updater after upgrading?
Query: SELECT `Contact_alias`.`_pageName` AS `Page name` xxx
Error: 1054 Unknown column 'Contact_alias.Type' in 'field list' (xxx)

If you need any further information then please let me know. Jonathan3 (talk) 20:44, 25 February 2020 (UTC)

OpenLayers map appears all pink on iPhone

The +- zoom buttons, red marker, gridlines, and copyright message all appear as normal - but the entire map is pink!

It works on the PC and iPad but strangely not on the iPhone (I got someone else to check his too).

The Googlemaps output format works fine.

Does this ring any bells? Is there anything I'd need to do to fix this? It always used to work, and I don't think I'm doing anything different.

Thanks. Jonathan3 (talk) 14:16, 23 February 2020 (UTC)

I don't know, but if you do a web search on "OpenLayers pink" there are a lot of results. Yaron Koren (talk) 04:48, 24 February 2020 (UTC)

It also doesn't work on IE11 and Edge 44 (it works on Chrome 80) all on Windows 10. The error messages on Edge are like this:

SEC7120: [CORS] The origin 'http://www.mywiki.com' did not find 'http://www.mywiki.com' in the Access-Control-Allow-Origin response header for cross-origin image resource at 'http://c.tile.openstreetmap.org/5/15/10.png'.

Does this help? Jonathan3 (talk) 22:52, 24 February 2020 (UTC)

It doesn't help me, but then I know very little about OpenLayers. Yaron Koren (talk) 02:15, 25 February 2020 (UTC)

From what I can gather, it's probably a problem with the browsers rather than the OpenLayers server Access-Control-Allow-Origin setting, my server, MediaWiki or your extension, but I'm still looking into it so can't be sure... :-) Jonathan3 (talk) 22:52, 25 February 2020 (UTC)

Problem with 'tokens' field

Hi,

Thank you for the amazing extension.

I tried having 'tokens' in a field similar to how it's described at 36:50 of this video https://www.youtube.com/watch?v=n3t4nr6GG7A

However, after I set new tokens in a new form, when I try to reuse them, it takes the whole block of those tokens as a single token, and it does not allow me to select individual tokens.

Here is some pictures to show you the problem. https://imgur.com/a/TKDQrF6

What can I do?

From memory (I’m typing on the phone) you need “List (,) of” String (or whatever) in the cargo_declare and “list” in the form field. Jonathan3 (talk) 07:49, 24 February 2020 (UTC)

Hi Jonathan3, thank you for the reply. So I did change the fields from "String" to “List (,) of String" in the TEMPLATE (as it said the Cargo table was declared by the template), so it looks like this (the fields I want the tokens are the 'Topic' and 'Field'

<pre>
{{Project
|Type=
|Topic=
|Field=
}}
</pre>
Edit the page to see the template text.
{{#cargo_declare:_table=Projects|Type=List (,) of String (allowed values=experimental,meta-analysis)|Topic=List (,) of String|Field=List (,) of String}}
</noinclude><includeonly>{{#cargo_store:_table=Projects|Type={{{Type|}}}|Topic={{{Topic|}}}|Field={{{Field|}}} }}{| class="wikitable"
! Type
| {{#arraymap:{{{Type|}}}|,|x|[[x]]}}

|-
! Topic
| {{{Topic|}}}
|-
! Field
| {{{Field|}}}
|}

{{DISPLAYTITLE: {{{title|}}} }}

[[Category:Projects]]
</includeonly>

However in the FORM this is how the field currently looks like

! Topic: 
| {{{field|Topic|input type=tokens}}}

where do I put the "list" element here?

Thank you MavropaliasG (talk) 14:42, 24 February 2020 (UTC)

Like this: {{{field|Topic|input type=tokens|list}}}

Also, after changing #cargo_declare you need to recreate the whole table for your changes to take effect. It may be that then you don't even need to add "list" to the form (it works out some things for itself) but I don't know for sure. Jonathan3 (talk) 15:29, 24 February 2020 (UTC)

Thank you very much, these were great instructions, it now works! MavropaliasG (talk) 01:14, 25 February 2020 (UTC)