Extension talk:RDFIO

Previous page history was archived for backup purposes at Extension talk:RDFIO/LQT Archive 1 on 2015-07-10.

Hello,

While testing the sample import I get the following exception

/mediawiki/index.php/Special:RDFImport Error from line 729 of /var/lib/mediawiki/vendor/semsol/arc2/store/ARC2_Store.php: Call to a member function escape() on null

Here is my installation detail

MediaWiki 1.34.2

PHP 7.4.3 (apache2handler)

MySQL 8.0.21-0ubuntu0.20.04.4

Semantic MediaWiki3.1.6

RDFIO v3.0.2

Another additional detail that I find weird is that I have to set up store every time I visit Special:RDFIOAdmin page.

I am able to successfully setup store on each attempt.

I'm getting this exact same error message (same line #729). Am hoping @SHL can help!

Here's my installation details:

MediaWiki 1.34.2

PHP 7.3.21 (litespeed)

MariaDB 10.3.23-MariaDB-cll-lve

Lua 5.1.5

Semantic MediaWiki 3.1.6

RDFIO v3.0.2

semsol/arc2 2.5.1

The code from line 727 to 729 shows the following - I've bolded line 729:

/* exact match */

        else {

            $sql = 'SELECT id FROM '.$this->getTablePrefix().$tbl." WHERE val = BINARY '".$this->db->escape($val)."' LIMIT 1";

            $row = $this->db->fetchRow($sql);

            if (null !== $row && isset($row['id'])) {

                $r = $row['id'];

            }

        }

        if ($r && (strlen($val) < 100)) {

            $this->term_id_cache[$term][$val] = $r;

        }

        return $r;

    }

-Suzy (suzyemail (at) gmail (dot)com)

Here's the backtrace for the error:

[X1QH3BjFAzimaGlZi2m3AQABBAg] /wiki/index.php/Special:RDFImport Error from line 729 of /home/bookbin1/public_html/wiki/vendor/semsol/arc2/store/ARC2_Store.php: Call to a member function escape() on null

Backtrace:

#0 /home/bookbin1/public_html/wiki/vendor/semsol/arc2/store/ARC2_StoreQueryHandler.php(31): ARC2_Store->getTermID(string, string)

#1 /home/bookbin1/public_html/wiki/vendor/semsol/arc2/store/ARC2_StoreSelectQueryHandler.php(1023): ARC2_StoreQueryHandler->getTermID(string, string)

#2 /home/bookbin1/public_html/wiki/vendor/semsol/arc2/store/ARC2_StoreSelectQueryHandler.php(996): ARC2_StoreSelectQueryHandler->getTriplePatternSQL(array, string)

#3 /home/bookbin1/public_html/wiki/vendor/semsol/arc2/store/ARC2_StoreSelectQueryHandler.php(1005): ARC2_StoreSelectQueryHandler->getPatternSQL(array, string)

#4 /home/bookbin1/public_html/wiki/vendor/semsol/arc2/store/ARC2_StoreSelectQueryHandler.php(996): ARC2_StoreSelectQueryHandler->getDefaultPatternSQL(array, string)

#5 /home/bookbin1/public_html/wiki/vendor/semsol/arc2/store/ARC2_StoreSelectQueryHandler.php(1005): ARC2_StoreSelectQueryHandler->getPatternSQL(array, string)

#6 /home/bookbin1/public_html/wiki/vendor/semsol/arc2/store/ARC2_StoreSelectQueryHandler.php(996): ARC2_StoreSelectQueryHandler->getDefaultPatternSQL(array, string)

#7 /home/bookbin1/public_html/wiki/vendor/semsol/arc2/store/ARC2_StoreSelectQueryHandler.php(924): ARC2_StoreSelectQueryHandler->getPatternSQL(array, string)

#8 /home/bookbin1/public_html/wiki/vendor/semsol/arc2/store/ARC2_StoreSelectQueryHandler.php(475): ARC2_StoreSelectQueryHandler->getWHERESQL()

#9 /home/bookbin1/public_html/wiki/vendor/semsol/arc2/store/ARC2_StoreSelectQueryHandler.php(68): ARC2_StoreSelectQueryHandler->getQuerySQL()

#10 /home/bookbin1/public_html/wiki/vendor/semsol/arc2/store/ARC2_StoreSelectQueryHandler.php(36): ARC2_StoreSelectQueryHandler->getSQL()

#11 /home/bookbin1/public_html/wiki/vendor/semsol/arc2/store/ARC2_Store.php(648): ARC2_StoreSelectQueryHandler->runQuery(array, integer)

#12 /home/bookbin1/public_html/wiki/vendor/semsol/arc2/store/ARC2_Store.php(598): ARC2_Store->runQuery(array, string, integer, string)

#13 /home/bookbin1/public_html/wiki/extensions/Rdfio/classes/RDFIO_ARC2StoreWrapper.php(138): ARC2_Store->query(string)

#14 /home/bookbin1/public_html/wiki/extensions/Rdfio/classes/RDFIO_ARC2StoreWrapper.php(162): RDFIOARC2StoreWrapper->getURIForEquivURI(string, boolean)

#15 /home/bookbin1/public_html/wiki/extensions/Rdfio/classes/parsers/RDFIO_URIToWikiTitleConverter.php(68): RDFIOARC2StoreWrapper->getWikiTitleByEquivalentURI(string)

#16 /home/bookbin1/public_html/wiki/extensions/Rdfio/classes/parsers/RDFIO_URIToWikiTitleConverter.php(51): RDFIOURIToTitleConverter->getExistingTitleForURI(string)

#17 /home/bookbin1/public_html/wiki/extensions/Rdfio/classes/parsers/RDFIO_ARC2ToWikiConverter.php(31): RDFIOURIToTitleConverter->convert(string)

#18 /home/bookbin1/public_html/wiki/extensions/Rdfio/classes/RDFIO_RDFImporter.php(77): RDFIOARC2ToWikiConverter->convert(array, array, array)

#19 /home/bookbin1/public_html/wiki/extensions/Rdfio/classes/RDFIO_RDFImporter.php(48): RDFIORDFImporter->importFromArc2Data(array, array, array)

#20 /home/bookbin1/public_html/wiki/extensions/Rdfio/specials/SpecialRDFImport.php(87): RDFIORDFImporter->importTurtle(string)

#21 /home/bookbin1/public_html/wiki/extensions/Rdfio/specials/SpecialRDFImport.php(35): RDFImport->importData(RDFIORequestData)

#22 /home/bookbin1/public_html/wiki/includes/specialpage/SpecialPage.php(575): RDFImport->execute(NULL)

#23 /home/bookbin1/public_html/wiki/includes/specialpage/SpecialPageFactory.php(611): SpecialPage->run(NULL)

#24 /home/bookbin1/public_html/wiki/includes/MediaWiki.php(296): MediaWiki\Special\SpecialPageFactory->executePath(Title, RequestContext)

#25 /home/bookbin1/public_html/wiki/includes/MediaWiki.php(900): MediaWiki->performRequest()

#26 /home/bookbin1/public_html/wiki/includes/MediaWiki.php(527): MediaWiki->main()

#27 /home/bookbin1/public_html/wiki/index.php(44): MediaWiki->run()

#28 {main}

-Suzy

#0 /var/lib/mediawiki/vendor/semsol/arc2/store/ARC2_StoreQueryHandler.php(31): ARC2_Store->getTermID()

#1 /var/lib/mediawiki/vendor/semsol/arc2/store/ARC2_StoreSelectQueryHandler.php(1023): ARC2_StoreQueryHandler->getTermID()

#2 /var/lib/mediawiki/vendor/semsol/arc2/store/ARC2_StoreSelectQueryHandler.php(996): ARC2_StoreSelectQueryHandler->getTriplePatternSQL()

#3 /var/lib/mediawiki/vendor/semsol/arc2/store/ARC2_StoreSelectQueryHandler.php(1005): ARC2_StoreSelectQueryHandler->getPatternSQL()

#4 /var/lib/mediawiki/vendor/semsol/arc2/store/ARC2_StoreSelectQueryHandler.php(996): ARC2_StoreSelectQueryHandler->getDefaultPatternSQL()

#5 /var/lib/mediawiki/vendor/semsol/arc2/store/ARC2_StoreSelectQueryHandler.php(1005): ARC2_StoreSelectQueryHandler->getPatternSQL()

#6 /var/lib/mediawiki/vendor/semsol/arc2/store/ARC2_StoreSelectQueryHandler.php(996): ARC2_StoreSelectQueryHandler->getDefaultPatternSQL()

#7 /var/lib/mediawiki/vendor/semsol/arc2/store/ARC2_StoreSelectQueryHandler.php(924): ARC2_StoreSelectQueryHandler->getPatternSQL()

#8 /var/lib/mediawiki/vendor/semsol/arc2/store/ARC2_StoreSelectQueryHandler.php(475): ARC2_StoreSelectQueryHandler->getWHERESQL()

#9 /var/lib/mediawiki/vendor/semsol/arc2/store/ARC2_StoreSelectQueryHandler.php(68): ARC2_StoreSelectQueryHandler->getQuerySQL()

#10 /var/lib/mediawiki/vendor/semsol/arc2/store/ARC2_StoreSelectQueryHandler.php(36): ARC2_StoreSelectQueryHandler->getSQL()

#11 /var/lib/mediawiki/vendor/semsol/arc2/store/ARC2_Store.php(648): ARC2_StoreSelectQueryHandler->runQuery()

#12 /var/lib/mediawiki/vendor/semsol/arc2/store/ARC2_Store.php(598): ARC2_Store->runQuery()

#13 /var/lib/mediawiki/extensions/Rdfio/classes/RDFIO_ARC2StoreWrapper.php(138): ARC2_Store->query()

#14 /var/lib/mediawiki/extensions/Rdfio/classes/RDFIO_ARC2StoreWrapper.php(162): RDFIOARC2StoreWrapper->getURIForEquivURI()

#15 /var/lib/mediawiki/extensions/Rdfio/classes/parsers/RDFIO_URIToWikiTitleConverter.php(68): RDFIOARC2StoreWrapper->getWikiTitleByEquivalentURI()

#16 /var/lib/mediawiki/extensions/Rdfio/classes/parsers/RDFIO_URIToWikiTitleConverter.php(51): RDFIOURIToTitleConverter->getExistingTitleForURI()

#17 /var/lib/mediawiki/extensions/Rdfio/classes/parsers/RDFIO_ARC2ToWikiConverter.php(31): RDFIOURIToTitleConverter->convert()

#18 /var/lib/mediawiki/extensions/Rdfio/classes/RDFIO_RDFImporter.php(77): RDFIOARC2ToWikiConverter->convert()

#19 /var/lib/mediawiki/extensions/Rdfio/classes/RDFIO_RDFImporter.php(25): RDFIORDFImporter->importFromArc2Data()

#20 /var/lib/mediawiki/extensions/Rdfio/specials/SpecialRDFImport.php(83): RDFIORDFImporter->importRdfXml()

#21 /var/lib/mediawiki/extensions/Rdfio/specials/SpecialRDFImport.php(35): RDFImport->importData()

#22 /var/lib/mediawiki/includes/specialpage/SpecialPage.php(575): RDFImport->execute()

#23 /var/lib/mediawiki/includes/specialpage/SpecialPageFactory.php(611): SpecialPage->run()

#24 /var/lib/mediawiki/includes/MediaWiki.php(296): MediaWiki\Special\SpecialPageFactory->executePath()

#25 /var/lib/mediawiki/includes/MediaWiki.php(900): MediaWiki->performRequest()

#26 /var/lib/mediawiki/includes/MediaWiki.php(527): MediaWiki->main()

#27 /var/lib/mediawiki/index.php(44): MediaWiki->run()

#28 {main}

Posted my stack trace above!

Hello,

RDF (here: Turtle) triples such as

:A :hasID "123", "456" ;

or

:A :hasID "123" ;

:A :hasID "456" ;

(which amounts to the same) can be imported by RDFIO, and the import log states that all triples were correctly imported. However, it looks like the last property value on import, here: "456", is the sole that gets retained (overwriting all previous ones).

On the other hand, there is no problem in introducing multiple values "by hand" in the SMW page, so the problem seems to be with RDFIO.

Now:

- did I possibly do something wrong? or

- is this a bug in RDFIO, and if so, how to fix it? or

- is this a necessary feature?

TIA

Airy Magnien

Hi Airy,

Ah, I think this is actually because of how RDFIO is designed. It assumes there is only one fact per unique page/property pair. I think we might not have realized that there could be a need for supporting multiple such. I'm wondering a bit what the RDF standard says about this though ... if this is supposed to be allowed in RDF, I think we should label this as a bug in RDFIO.

Best

Samuel

Hi Samuel,

Actually, I found a related issue in the SMW Git, that has been solved around April 12th. I lost track of it, however. I'll search again...

Hi Samuel

If you have some time, maybe you could have a glimpse at https://github.com/rdfio/RDFIO/compare/master...dataspects:multival regarding importing multivalue RDF like e.g. :InFunctionalCategory "Block system", "CBI". Essentially I am using array keys that are concatenated from predicate and object values.

Please note that I have not touched any code that seems to be dealing with annotations that are somehow contained in templates. So, for now, I do not know how those cases are handled in view of my code changes.

Many thanks

Lex

Hello,

On my SMW website (not yet public), I have created a "class" with all bells & whistles : category, page form, template, properties...

I can import a page (from Turtle format) where the page has the property rdf:type "TestClassCategoryName"; the result is ok, the page belongs to the category "TestClass" as expected, but thee template does not get applied. All property values appear as free text.

I can click on the tab "Edit with form", and I see it appearing with the mention "Warning: This page already exists, but it does not use this form.". The form is indeed the intended one, that I linked with the category, and as mentioned before, all semantic properties appear as properly encoded with double brackets etc. in the free text box, while all property boxes are empty. See screenshot.

Is there a way to "force" the filling of the form with the property values? I could not find any indication by browsing through the web or through this site.

TIA

Airy Magnien

screenshot

Hi again Airy,

This again seems to be a bug in RDFIO. I think it is related to Issue 58 on GitHub. Templates are not automatically applied, if the page is not already using a template.

I'd love to have a look at it, but is very constrained on time unfortunately, so don't know when I can do it. Any contributions are welcome. E.g, if somebody could make a rough patch / pull request, I could look over it and finish it.

Best Samuel

No mater what I paste into Special:RDFImport, I get returned to the page $1/Special:RDFImport, and (quite naturally) get an error message. Does anyone here happen to know what the problem might be?

Oops, sounds strange.

Could you please report your version of MediaWiki, Semantic MediaWiki, RDFIO, PHP and operating system, as well as any more details on how you installed RDFIO? I.e, did you install the ARC2 library properly?

Versions:

MediaWiki  1.21.5 (98e2d46)
PHP        5.3.3-7+squeeze18 (cgi-fcgi)
MySQL      5.1.73-1

Semantic MediaWiki         (Version 1.8.0.5)
RDFIO                      (Version 1.9.0 alpha)
Wiki ObjectModel Extension (Version 1.0.2 alpha)

on a Debian server. I cloned the extensions from https://github.com/samuell/RDFIO.git. As for ARC2, I got no error messages, and Special:ARC2Admin says Store is already set up.. Not sure what else to check for.

Ok, thanks for the info! I'll try to see if I can reproduce this! // Cheers

Upgrading to MediaWiki 1.23alpha, SMW 1.9.2 alpha, RDFIO 1.10.0 alpha didn't help. I guess there must be something in my config, but I can't figure out how to troubleshoot.. :-/

Can you confirm that the error you're getting is the same?

Yes, exactly the same.

RDFIO is now updated with many fixes, dropped and updated dependencies and simplified installation. Please see the installation section here for how to set up the latest version: https://github.com/rdfio/RDFIO#installation

Please from now on use the GitHub issue tracker to post issues, feedback and questions! Activity on this talk page is too easily missed by the RDFIO developers, and we find it much easier to get an overview of current issues and discussions, on GitHub.

Hello there,

i would like this not showing up each time i import something new. How would we do this?

Hi, could you please explain your problem with some more information, like a screenshot? Also, which version of RDFIO, SMW and MW are you using? If you want, you can post an issue to the github issue tracker: https://github.com/rdfio/RDFIO/issues ... where it will be easier to include pictures and discuss the issue in more detail! :)

I am trying to make an RDF import, but whenever i put too many statements, it seems RDFIO is unavailable to handle it. After like 3 minutes, page turns white and i see that only like 15 statemets were correctly imported.

If i do not import much it works (with about 20).

Atm i am trying with like 250, and seems it is processing still(about 5 minutes done) so seems things are going smoothly.

Problem is i want to import about....2000. Any parameter that i can config. to do so, or is it a bug which happens when there is a lot of info going?

Hi! Thanks for testing it out! This is unfortunately a known problem, and has to do with the fact that a page write in MediaWiki is quite a heavy task, since text has to be indexed, many tables updated etc.

We have been thinking about adding a batch import feature, that would import a number of triples per "run", and then either refresh the page to continue, or that it is completely done from the commandline, but we haven't got there yet.

Ok thanks! That would certainly work. I had to do the import in about 6 times, of about 250 statements each. Hope i can export data directly from mySQL to reause it on other databases and do not have to do this each time!

Have a look at the new commandline batch import script:

https://github.com/samuell/RDFIO/blob/master/maintenance/importRdf.php

If you have commandline access to the wiki, that might help, especially if you can change the timeout settings for commandline php to not time out too fast. The import probably is still equally slow, but hopefully can be done with a longer timeout this way.

Even better, if you have access to batch XML import in MediaWiki, is to use the new rdf2smw tool, which converts RDF n-triples to a MediaWiki XML dump:

https://github.com/samuell/rdf2smw

(If you need to convert from RDF/XML to N-Triples, you could use the rdf2rdf tool, for example:https://github.com/knakk/rdf2rdf ).

Plan to add support for more formats in rdf2smw shortly too.

Best // Samuel

Just a ping that the importRdf.php script is now included in the latest RDFIO release!

Hello there, I tried to install this extension on my Wiki (Version 1.24), which is already running with Semantic MediaWiki (Version 2.2) for a while now. Because of that, I need to update my database after installing the RDFIO extension. Using the runJobs.php, I get this Error-Message:

"PHP Fatal error: Call to a member function mapDataItemToElement() on null in [...]/extensions/SemanticMediaWiki/includes/export/SMW_Exporter.php on line 593"

This Error-Message only appears while the RDFIO-Extension is activated... Any suggestions or clues?

Thanks :)

Mmm..i am having the same error.

This is related with SMW version. Since version 2.2, RDFIO needs to be actualized by creator (since it is not longer working with actual implementation). He has been warned by SMW dev already as stated by MWJames :

https://github.com/samuell/RDFIO/issues/11

So? What to do?

Do we have to wait for SMW 2.3?

According to new RDFIO maintainer Ali King, RDFIO should now be updated for SMW 2.3. Please report any remaining issues in the issue tracker: https://github.com/rdfio/RDFIO/issues

Same here...I get this error-message every time I try to edit an article.

RDFIO has now been updated to work with SMW 2.4, and installs via Composer. Please see the updated and much simplified installation instructions: https://github.com/rdfio/RDFIO#installation

I installed the Virtual machine appliance image using Virtualbox which worked fine. What I couldn't find were the passwords or logins for Linux, Linux root, mediawiki, semantic media wiki, MySQL. Does anyone know these? Thanks

Very sorry about this! The README is now updated with this information. See further down under the "How to" heading:

https://github.com/samuell/rdfio-vagrantbox#how-to

Kind Regards // Samuel

Hey there!

This might see a bit more complex than usual questions, but i would really apreciate the help.

At the moment i have to import a lof of classes and subclasses, and i am seeing that after for example creating a category in smw (called "Categoria15"), i get this at the rdfio endpoint(?s ?p ?o):

http://ExerWiki/mediawiki-1.24.1/index.php/Especial:URIResolver/Categor-C3-ADa-3ACategoria15 http://www.w3.org/2000/01/rdf-schema#label Categoria15

http://ExerWiki/mediawiki-1.24.1/index.php/Especial:URIResolver/Categor-C3-ADa-3ACategoria15 http://www.w3.org/2000/01/rdf-schema#isDefinedBy http://localhost:85/mediawiki-1.24.1/index.php/Especial:ExportRDF/Categor%C3%ADa:Categoria15

http://ExerWiki/mediawiki-1.24.1/index.php/Especial:URIResolver/Categor-C3-ADa-3ACategoria15 http://www.w3.org/1999/02/22-rdf-syntax-ns#type http://www.w3.org/2002/07/owl#Class

(Apart from the triples coming from Swivt ontology)

So i undertand, in order to create a class an import it to SMW in it´s natural format i would need at least this 3 rdfs statemets, declaring the label of the URI, which thing is defined by and saying is a class.

Aftewards i would have to do something similar to declare that the class is subclass of another.

Which would be the statements to use, to import a class with that format which was called for example..."category10"?

I do not know if it is clear what i am asking for.

Thanks.

Hi 00maiser00!

I think these are things that are created by Semantic MediaWiki automatically when a Category page is created, and should not need to be created, to create a class.

Basically a class is detected as soon as a "http://www.w3.org/1999/02/22-rdf-syntax-ns#type", or a "http://www.w3.org/2000/01/rdf-schema#subClassOf property" are seen.

If you are interested in the details, this can be seen in detail in the code, in the highlighted part here: https://github.com/samuell/RDFIO/blob/master/classes/parsers/RDFIO_ARC2ToWikiConverter.php#L40-L62

For what i detect, after we make an import of an statement (for example):

Car [(rdf)issubclassof] Vehicle

The process that they follow is:

1º RDFIO gives a URI for car and vehicle

2º Semantic mediawiki gives a URI to car and vehicle

3º Semantic mediawiki says the uris of RDFIO and SMW are the same.

4º Semantic mediawiki makes the declaration(indicating if it is subclass, class....) using the SMW URI.

Would be lovely to merge 4 steps in 3, as clearly, one should not be relevant (regarding user, perhaps it is needed for implementation) as feels pages with "equivalentURI" type statements.

Would this be possible from RDFIO side? Or would it involve changing SMW?