Extension talk:External Data/Archive 2012

JSON issue

Hi - I'm having an issue with json data. I'm using External Data v1.3.5. Mediawiki version is 1.16.2 [1]

• Json: [2]
• My SMW page: [3]
• The "tags" array in the json doesn't seem to be parsed...

--Skenaja 21:40, 30 January 2012 (UTC)

Hi - my guess is that the issue is the double brackets around the "tags" value, which I think makes it an array of an array. Why are there two brackets there - do you know? Yaron Koren 13:14, 31 January 2012 (UTC)

I'm not too sure, I've raised it with that site's developers. The json does parse OK @ JSLint.--Skenaja 16:54, 31 January 2012 (UTC)

Actually, the brackets might not be the issue - the problem might just be External Data's parsing of arrays in JSON. Someone just sent me a patch, I think coincidentally, that might fix this very issue. Hopefully I can add it in sometime in the next few days. Yaron Koren 17:05, 31 January 2012 (UTC)

I've had it confirmed to me that the double brackets are indeed arrays of arrays - here's another sample json file which shows it better: [4] --Skenaja 22:56, 31 January 2012 (UTC)

Hi Yaron - Did you get a chance to add the patch?--Skenaja (talk) 15:53, 11 November 2012 (UTC)

Ah... thanks for the reminder. I completely forgot about that patch until you brought it up. I just finally checked it in. It does work better on double brackets, i.e. arrays of arrays, than the previous code - if you try to get the value for "tags", you get a single string ("london_borough"), instead of a null value. I don't know if that's the fully correct result - I'm not sure what the ideal handling would be in this case. Yaron Koren (talk) 03:01, 21 November 2012 (UTC)

Get data from Sqlite Db

Hi,

I would like to know if it's possible to retrieve data from sqlite db ? I've tested a few conbinations in Localsettings but not luck... I use Mediawiki 1.16.5.

Thanks in adavnce. Vincent.

There was no support for SQLite, for some reason - I don't know why I never added it in. I just added it now, I think, along with support for DB2, which was also missing. If you use SVN, you should re-get the code - if not, you can just duplicate this change in your code. Then, you should be able to set "sqlite" as the DB server type. Please let me know if that worked... Yaron Koren 14:36, 31 January 2012 (UTC)

Thanks for the fast reply, I'll give you feedback shortly.

Vincent.

Could you tell if my Localsettings.php is good ?

$edgDBServer['SQLITE'] = "/mnt/sqlite/infos_db"; ==> folder of sqlite db
$edgDBServerType['SQLITE'] = "sqlite";
$edgDBName['SQLITE'] = "infos_db";
$edgDBUser['SQLITE'] = "";
$edgDBPass['SQLITE'] = "";

Error: ( SQLite database not accessible)

Thanks in advance

I assume you're setting the username and password in the original - beyond that, I don't know. I'm used to the DB server being a name or a URL of some sort, and not a directory, but I don't know how SQLite works. Yaron Koren 15:45, 31 January 2012 (UTC)

HI, After a few search around compatibility between php and sqlite. It seems that sqlite version i use (sqlite3) uses PDO function (PHP) to connect to that db. Connection and query syntax seem to be different from ED_Utils.php code. I'm not skilled enough to review ED_Utils.php code.

Thanks for the help. Great job, i use it for Mysql access without any problem. Vincent

Hi, I'm gone through sqlite connection. it works great. For information you have to :

Add $wgSQLiteDataDir = "/root/forlder/of/sqlite"; in your Localsettings.php

Vérify you have RW rights on sqlite db.

$edgDBServer['SQLITE'] = ""; ==> Not used
$edgDBServerType['SQLITE'] = "sqlite";
$edgDBName['SQLITE'] = "infos_db"; ==> Name of sqlite db without extension (Be sure to have .sqlite extension
 because DatabaseSsqlite.php script will add it automatically) 
$edgDBUser['SQLITE'] = ""; ==> Not used
$edgDBPass['SQLITE'] = ""; ==> Not used

Thanks Yaron, it also works great for Oracle Database (10G R2). Vincent G

Thanks for letting me know about that, and for diagnosing the issue. I just checked in some code that I think handles this issue in a nicer way - you can see the new documentation for how to run it. If you get the new code and try it out, please let me know if it works for you. Yaron Koren 20:26, 3 February 2012 (UTC)

GetData from same wiki (CSV)

Issue

I am trying to get CSV data from a page in the same wiki. I have duplicated the example from:

http://discoursedb.org/wiki/Fruits_table

I have tried both the explicit URL and the fullurl and various combinations of raw data and Special:GetData in the URL. No matter what, the tables are blank. When I follow the links, the following test appears in the browser window:

 Name,Color,Shape Apple,Red,Round Banana,Yellow,Oblong Orange,Orange,Round Pear,Yellow,"Pear-shaped"

The tables are completed when I pull directly from the example external URL:

 http://discoursedb.org/wiki/Special:GetData/Fruits_data

I am must be doing something bone-headed, but can't see my error. Did I miss some setting? Here is the abbreviated content of my Fruits table page:

Thanks in advance, Rodger

Example

From my local wiki using explicit URL (Fails)

Using: https://10.34.161.20/index.php?title=Special:GetData/Fruits_data&action=raw

{{#get_web_data:url=https://10.34.161.20/index.php?title=Special:GetData/Fruits_data&action=raw
|format=CSV with header
|data=name=Name,color=Color,shape=Shape}}

{| class="wikitable"
! Name
! Color
! Shape {{#for_external_table:<nowiki/>
{{!}}-
{{!}} [[{{{name}}}]]
{{!}} {{{color}}}
{{!}} {{{shape}}} }}
|}

From my local Wiki using fullrul (Fails)

Using fullurl: {{fullurl:Special:GetData/Fruits_data|action=raw}}

{{#get_web_data:url={{fullurl:Special:GetData/Fruits_data|action=raw}}
|format=CSV with header
|data=name=Name,color=Color,shape=Shape
}}
{| class="wikitable"
! Name
! Color
! Shape {{#for_external_table:<nowiki/>
{{!}}-
{{!}} [[{{{name}}}]]
{{!}} {{{color}}}
{{!}} {{{shape}}} }}
|}

From external Wiki (Works)

Using http://discoursedb.org/wiki/Special:GetData/Fruits_data

{{#get_web_data:url=http://discoursedb.org/wiki/Special:GetData/Fruits_data|format=CSV with header|data=name=Name,color=Color,shape=Shape}}
{| class="wikitable"
! Name
! Color
! Shape {{#for_external_table:<nowiki/>
{{!}}-
{{!}} [[{{{name}}}]]
{{!}} {{{color}}}
{{!}} {{{shape}}} }}
|}

I have tried on a couple of my wiki VMs with the same result., including updated version, but same behavior regardless. Seems like it wouldn't work for the external URL if there was a version conflict.

MediaWiki 1.18.1, Semantic Bundle, ..External Data (Version 1.3.5), ...

Discussion

Hi - that's quite a bit more information than was necessary. :) But I admire your thoroughness. Anyway, I assume the issue is that the server, strange as it may sound, can't access itself. It's trying to reach its data via the IP address, and that one, I assume, is closed off. You'll need to modify something for the server to be able to access itself via HTTP. Yaron Koren 04:36, 15 February 2012 (UTC)

That makes sense. Thanks for the quick response! Rodger

I don't think I can overcome the server access. I don't have the expertise to suggest a change to the admins, and don't know if would be feasible. Is there a way that something like the following could be made to work?

  #get_web_data:url={{localurl:Fruits_data}} or
  #get_web_data:url={{localurl:Special:GetData/Fruits_data}}

Or a parallel #get_local_data? I am familiar with the internal objects, but would really like to use CVS or XML when I have it, rather than translate to internal object syntax.

I can't think of a way to do that, unfortunately... though, what about using "localhost" instead of the IP address? That might work. Yaron Koren (talk) 02:50, 17 February 2012 (UTC)

Anybody found a solution with this issue? I have the same issue using a Wiki in an secure environment (https://wiki.xxx) (Christoph, 9. May 2012)

get_db_data Parsing

This is related to the discussion from about a year ago:

http://www.mediawiki.org/wiki/Extension_talk:External_Data/Archive_2011#get_db_data_-_Data_parsing

This data retrieval DOES NOT work when the field with a convert is in the 2^nd position

{{#get_db_data:
  server=MSTC
  |from=Tool
  |where= Acronym_Txt ='ITEM'
  |data=mstcName=Title_Txt,mstcRevDate=CONVERT(VARCHAR(12), Documentation_Rev_Dt, 107) AS Rev_Dt
}}

This data retrieval DOES work when the field with a convert is in the 1^st position

{{#get_db_data:
  server=MSTC
  |from=Tool
  |where= Acronym_Txt ='ITEM'
  |data=mstcRevDate=CONVERT(VARCHAR(12), Documentation_Rev_Dt, 107) AS Rev_Dt, mstcName=Title_Txt
}}

--Wolcott 20:10, 16 February 2012 (UTC)

Have table display results horizontally rather than vertically

I have this query against the MySQL db that goes something like this:

{{#get_db_data:
db=WikiUser
|from=ProfilePagePic
|limit=10
|order by=page_id desc
|data=user=user_name,pic=il_to
}}

From that I am able to pull in the results into a table like so:

 {| class="wikitable" id="test"
!Recent Profiles
|-
{{#for_external_table:
|-
| [[File:{{{pic}}}|100px|{{{user}}}|link=User:{{{user}}}]]}} 
|}

And it displays the results just fine. However, I was wondering if there is a simple way that I am missing where you could have the extension display the results horizontally instead of vertically.

So instead of seeing a list of names like this:

Profiles
Name 1
Name 2
Name 3

I want it to look like this:

Profiles
Name 1	Name 2	Name 3

--Stanimalz (talk) 16:04, 22 February 2012 (UTC)

Hi - I would try removing the "|-" line under the #for_external_table line. Yaron Koren (talk) 16:08, 22 February 2012 (UTC)

That worked perfectly. Thanks for the quick response.--Stanimalz (talk) 17:47, 22 February 2012 (UTC)

Oracle usage

Hi,

can somebody tell me how to get the extension working with oracle?
I can't figure out which settings i have to set.

And how does the Localsettings.php look like if i use the extension to connect an Oracle DB.

My Attempt:

$edgDBServer['bla'] = "blabla.bla.de";<br>
$edgDBServerType['bla'] = "oracle";<br>
$edgDBName['bla'] = "bladb";<br>
$edgDBUser['bla'] = "blauser";<br>
$edgDBPass['bla'] = "blapw";<br>

And my Query looks like:

{{#get_db_data:<br>
db=bladb<br>
|from=SYS.TABLE_PRIVILEGE_MAP<br>
|data=priv=PRIVILEGE,nam=NAME,row=ROWID<br>
}}

This should probably be more obvious in the documentation, but you should have "db=bla" instead. Yaron Koren (talk) 13:30, 13 March 2012 (UTC)

If i type in "db=bla" i receive the following error in my wiki:

Database error

A database query syntax error has occurred. This may indicate a bug in the software. The last attempted database query was:

ALTER SESSION SET CURRENT_SCHEMA=BLA

from within function "DatabaseOracle::selectDB". Database returned error ": ".

Any idea?

These is a Systable which is readable by my user.

-jan

This could be an error in MediaWiki's handling of Oracle, or maybe just a permission error. I would try getting some more basic data from that database, like the contents of a regular table, to see where the problem is coming from. Yaron Koren (talk) 17:29, 13 March 2012 (UTC)

I'm also having trouble setting up an Oracle connection in Mediawiki 1.18.1 and ExternalData 1.3.5, with a configuration that works fine in Mediawiki 1.16.0 with the same ExternalData version. By trouble I mean I am getting the Database error with ALTER SESSION SET CURRENT_SCHEMA=BLA as the last attempted query when attempting a very basic query from a normal table (same query works fine if I write a simple php script with oci calls). I noticed that between these versions of MW, the DatabaseOracle.php include file has grown twofold. Any thoughts on what change may be breaking External Data? Thanks for your help in advance. --Omer

My problem was solved by making the following change in LocalSettings.php:

old: $edgDBServer['somedb'] = "the.db.server.com";
new: $edgDBServer['somedb'] = "the.db.server.com/somedb";

where "somedb" is the instance name. I hope this helps others with similar issues when upgrading MediaWiki past 1.16. --Omer

Microsoft Sql-Server Error [solved]

Hi,

can somebody tell me how to get the extension working with Microsoft Sql Server and MediaWiki 1.18?
I have a working MediaWiki 1.15 installation with External Data connecting to Microsoft Sql Server 2008R2

Now i want to Upgrade to MediaWiki 1.18 and all works fine, but the ExternalData Extension fires some Warnings and Ends up with an Fatal error. I don't think that it is a configuration error, because of the running MediaWiki 1.15 installation. The Messages i get:

Warning: Invalid argument supplied for foreach() in ...\includes\db\DatabaseMssql.php on line 978<br />
Warning: Invalid argument supplied for foreach() in ...\includes\db\DatabaseMssql.php on line 978<br />
Warning: Invalid argument supplied for foreach() in ...\includes\db\DatabaseMssql.php on line 978<br />
Warning: Invalid argument supplied for foreach() in ...\includes\db\DatabaseMssql.php on line 978<br />
Warning: Invalid argument supplied for foreach() in ...\includes\db\DatabaseMssql.php on line 978<br />
Warning: Invalid argument supplied for foreach() in ...\includes\db\DatabaseMssql.php on line 1010<br />
Fatal error: Unsupported operand types in ...\includes\db\DatabaseMssql.php on line 1027<br /> 

Details of this Installation are on my UserPage, the Servertype in LocalSettings is set to mssql: $edgDBServerType['NAME_OF_DATABASE'] = "mssql";

--TomyLee (talk) 06:08, 4 April 2012 (UTC)

Hi - if possible, could you include here the #get_db_data call you're making? Feel free to change around any table or field names, if it's private. Yaron Koren (talk) 15:15, 4 April 2012 (UTC)

the get_db_data i have testet are:

{{#get_db_data:<BR />
|server=org_srv<BR />
|from=Wiki.SQL_Auswertungen<BR />
|data=Auswertung_Id=Auswertung_Id,Bearbeiter1_Name=Bearbeiter1_Name,Bearbeiter2_Name=Bearbeiter2_Name,Land_Id=Land_Id,Land_Name=Land_Name,Land_IK=Land_IK,Hersteller_Id=Hersteller_Id,Hersteller_Name=Hersteller_Name,Auswertung_Name=Auswertung_Name,Lieferung=Lieferung
}}

and

{{#get_db_data:<BR />
|server=org_srv<BR />
|from=Wiki.SQL_Auswertungen<BR />
|where=1=1<BR />
|limit=10000<BR />
|order by=Auswertung_Id ASC<BR />
|data=Auswertung_Id=Auswertung_Id,Bearbeiter1_Name=Bearbeiter1_Name,Bearbeiter2_Name=Bearbeiter2_Name,Land_Id=Land_Id,Land_Name=Land_Name,Land_IK=Land_IK,Hersteller_Id=Hersteller_Id,Hersteller_Name=Hersteller_Name,Auswertung_Name=Auswertung_Name,Lieferung=Lieferung
}}

--TomyLee (talk) 05:44, 5 April 2012 (UTC)

Okay - if the same error happens for both of those calls, then the issue isn't with the #get_db_data call. I should have asked this before, but - did you see the note in the documentation about MSSQL? It could be that you just need to use the Extension:MSSQLBackCompat extension, if you're not using it already. Yaron Koren (talk) 16:14, 5 April 2012 (UTC)

I have installed Microsoft drivers for PHP for SQL Server and the External Extension works with MW 1.15, but not with MW 1.18. (Within the MW 1.15 installation i need to replace mssql with sqlsrv in DatabaseMssql.php to get it working). The DatabaseMssql.php within the MW 1.18 was unchanged. Now I have changed the Code a little bit to print out the variable around the error and can see some of the Data from the Sql-Server. The connecting to SQL Server works and MW gets the data but can't processing it. The MSSQLBackCompatExtension is only to query MSSQL from Linux, we are running MW on Windows. So I think the Microsoft drivers for PHP are the right choice (also we don't have a /etc/freetds.conf configuration file).

Maybe this helps; I have insert

echo "<BR />" . $row;

at line 978 and can see that the result is the first Datarow of my Datatable:

-1 Warning: ...
Testname Warning: ...
Testname Warning: ...
-1 Warning: ...
nicht zugeordnet Warning: ...
- Warning: ...
0 Warning: ...
allgemein Warning: ...
allgemein nicht zugeordnet Warning: ...
0 Warning: ... Warning: ... on line 1011 Fatal error: Unsupported operand types in ..\includes\db\DatabaseMssql.php on line 1028

Then I have print out the two parts of Line 1028 and gets: $this->mRows[$this->mCursor] : -1, $arrNum : empty Array (with print_r($arrNum) --TomyLee (talk) 05:50, 10 April 2012 (UTC)

Unfortunately, I know very little about SQLServer, or about MediaWiki's support for it. And I don't know much about running MediaWiki on Windows, for that matter. If you can, I would write about this to one of the general MediaWiki venues, like the #mediawiki IRC channel or the mediawiki-l mailing list. Yaron Koren (talk) 13:41, 10 April 2012 (UTC)

I had a very similar problem. All I did was comment out lines 974 and 976 and added a while loop into DatabaseMssql.php it seems that sqlsrv_fetch_array only gets one row? And mine works fine now

$rows = sqlsrv_fetch_array ( $ queryresult, SQLSRV_FETCH_ASSOC );

foreach ( $rows as $row ) {

to

//$rows = sqlsrv_fetch_array ( $ queryresult, SQLSRV_FETCH_ASSOC );
while ($row = sqlsrv_fetch_array ( $ queryresult, SQLSRV_FETCH_ASSOC )) {
//foreach ( $rows as $row ) {

Craig Purchase (talk) 16:11, 21 June 2012 (UTC)

This works for me too! Thanks very much for your help. --TomyLee (talk) 06:36, 22 June 2012 (UTC)

That's great news! I just added this tip to the documentation. Yaron Koren (talk) 15:51, 22 June 2012 (UTC)

POST instad of GET

In order to connect to a local webservice I needed to post some data.

{{#get_web_data:
url=http://localhost:8080
|format=JSON
|data=URL=uri
|postData=<mws:query
  xmlns:mws="http://www.mathweb.org/mws/ns"
  xmlns:m="http://www.w3.org/1998/Math/MathML"
  limitmin="0"
  answsize="30"
  totalreq="yes"
  output="json"
  >
    <mws:expr>
        <m:apply>
            <mws:qvar>op</mws:qvar>
            <mws:qvar>x</mws:qvar>
            <mws:qvar>x</mws:qvar>
            <mws:qvar>y</mws:qvar>
        </m:apply>
    </mws:expr>
</mws:query>
}}

Only very few code modifications were sufficient to achieve that.

Should I share that changes. If yes where and how?

--Schubi87 (talk) 21:43, 11 April 2012 (UTC)

Yes, that would be great! You can either create a patch in Bugzilla, or send me an email at yaron57 at gmail.com. Yaron Koren (talk) 19:23, 12 April 2012 (UTC)

Are you going to release the post option I send you soon, or should I create a new branch?

--Schubi87 (talk) 10:12, 28 June 2012 (UTC)

Extension:External Data fails without notice

Hello,

We've been trying to implement External Data extension to the company's wiki, but it seems to behave a little unstable. We have got it working already, but from times to times it fails again. I could not say the causes, but I suspect it has to do with some sort of updates.

Some months ago, we had External Data 1.3.5 working on Mediawiki 1.7.1 and Semantic Mediawiki 1.5.5 for testing. We created two or three Sandbox pages todo test it. Recently, we updated Mediawiki to version 1.7.3. I could not say that update is the cause bacause we have not accessed those Sandbox pages to check it. No other extension/database/PHP was updated during this time. Last week I started working to bring the extension to production environment, and tried to use the documentation examples before using my own files. Nothing worked. Pages simply doesn't retrieve any information from SCV files either inside or outside the wiki. Then I've checked the older pages that were working in the earlier tests. They were failing too.

I tried to update External Data to version 1.3.6, but the problem remains.

Well, I know the information I am giving above is not enough to make a diagnosis. Never mind about that. But I'm not sure where to get the information to do that.

Is there any checklist we have to observe in order to avoid External Data stops working after Mediawiki or third party extensions updates. Any other idea on where to search in order to find the cause of the problem?

Thanks a lot in advance.

I would try copying the text from this page to a page on your wiki, and seeing if that works. By the way, when you wrote "MediaWiki 1.7.3", I assume you meant 1.17.3? Yaron Koren (talk) 13:39, 19 April 2012 (UTC)

Hi.

I did. But, it didn't work, also. Tks Pslima (talk) 16:57, 19 April 2012 (UTC)

Okay. What operating system is the wiki running on, and what version of PHP are you using? Yaron Koren (talk) 17:18, 19 April 2012 (UTC)

Hi.

• Debian 6.0 - kernel 2.6.32-5-openvz-amd64
• PHP 5.3.3-7+squeeze8 (apache2handler)
• MySQL 5.1.61-0+squeeze1

Tks Pslima (talk) 15:57, 26 April 2012 (UTC)

Query Returns no values?

Hi

I am attempting to use this Extension on my Wiki to retrieve data from a CSV file stored on a dropbox (which is available to the Public), the problem i get is that is runs, but returns blank values. I have looked for help but cant seem to find the answer anywhere.

The code i am using is is a sample which is avaible on the Extensions page:

{{#get_web_data:url=http://semanticweb.org/wiki/Special:Ask/-5B-5BGermany-5D-5D/-3FBorders/-3FPopulation/-3FArea/format%3Dcsv/sep%3D, |format=csv with header|data=bordered countries=borders,population=population,area=area}} * Germany borders the following countries and bodies of water: {{#arraymap:{{#external_value:bordered countries}}|,|x|[[Borders country::x]]}}. * Germany has population {{#external_value:population}}.

I get no error messages when save the page either.

Its important you know that this is being run from an internal server from work, so i am wondering, could that have anything to do with the fact that i cant retrieve any values? as in, would the Firewalls from the server be preventing this extension from extracting the data? I say this because i ran the same piece of code on "Discourse DB" Wiki (as they have the extension installed) and it worked... Thanks

Claudio

Yes, my guess is that the firewall is preventing your server from hitting outside URLs. If you have command-line access on your server, you can try calling something like "curl http://google.com" to see if that works. Yaron Koren (talk) 15:33, 29 June 2012 (UTC)

External data still flawed after SMW refreshData

Referring to External_data_flawed_after_SMW_refreshData, we are getting the same error with SMW 1.6.1. In other words, we observe that after rebuilding the SMW database a number of user pages that fetch name/mail/phone data from LDAP are showing flawed data. There are groups of user pages that show the same data, even though they have different LDAP usernames. The only way to correct it is by clicking "edit" and then save. The database tables show that the wrong values for name/mail/.. are stored in e.g. smw_atts2 together with the s_id of the flawed page. This is not fun when there are hundreds of pages.... What to do? Bernadette (talk) 19:08, 28 June 2012 (UTC)

The extension SemanticUpdateOnPurge might be able to help in this situation. Yaron Koren (talk) 15:35, 29 June 2012 (UTC)

I think I found the bug: in doGetLDAPData (and all other doGet... functions) the global $wgTitle->getTitle() is used to gather the name of the current page. However when running the jobqueue, this call always returns "runJobs.php". The use of $wgTitle is deprecated anyway, and using the preferred $parser->getTitle() cures the problem in my case. - LosWochos (talk) 16:01, 17 July 2012 (UTC)

Thanks for that suggestion! I just checked your suggested changes into the Git repository - hopefully that fixes things. Yaron Koren (talk) 21:10, 18 July 2012 (UTC)

Issues getting xml to work using xpath

I'm trying to get external data to work using xml and xpath and was hoping someone could help me out. So far I can access an xml to populate a table or substitute a word but once I try and use xpath nothing is being returned. I have looked at the example and followed them to get csv and xml(with out xpath) working.

{{#get_web_data:url=http://localhost/wiki/tables.xml
|format=xml
|use xpath
|data=name=name}}

'''test:'''{{#external_value:name}}

{| class="wikitable"
! NAME {{#for_external_table:<nowiki/>
{{!}}-
{{!}} [[{{{name}}}]] }}

Did you try actually using XPath in the 'data' parameter, like "data=name=/abc/name" or something? Yaron Koren (talk) 12:35, 6 July 2012 (UTC)

Thanks for getting back to me. Yes, I tried using "data=name=/abc/name" I also copied the examples from http://www.discoursedb.org/wiki/External_Data_XPath_demo to see if I could get it to work. are there any other extensions that I should also install to get this to work?

example from http://www.discoursedb.org/wiki/External_Data_XPath_demo after being copied locallly, sample.xml which is the same as the one from the example is access from the address show below

{{#get_web_data:url=http://localhost/wiki/sample.xml
|format=xml
|use xpath
|data=salutation=/letter/salutation,maxlength=/letter/title[2]/@maxlength,address=/letter/address[@translate='no'],url=/letter/weblink}}

'''Salutation:''' {{#external_value:salutation}}

'''Max length:''' {{#external_value:maxlength}}

'''Address:''' {{#external_value:address}}

'''URL:''' {{#external_value:url}}

Thank you for your help Mw49 (talk) 04:39, 9 July 2012 (UTC)

Just to make sure: are you using the latest version of External Data, 1.4? This feature was only added in very recently. Yaron Koren (talk) 12:54, 9 July 2012 (UTC)

Yep that looks to be the problem, sorry for wasting your time. --Mw49 (talk) 05:14, 10 July 2012 (UTC)

That's great. No problem - it didn't take much time on my side. Yaron Koren (talk) 12:48, 10 July 2012 (UTC)

#for_external_table support for #get_ldap_data

The repository looked ad-hoc so I had no idea where to contribute this...

diff --git a/ED_ParserFunctions.php b/ED_ParserFunctions.php
index a5e8e8d..19c9bb9 100644
--- a/ED_ParserFunctions.php
+++ b/ED_ParserFunctions.php
@@ -201,10 +201,13 @@ class EDParserFunctions {
                $mappings = EDUtils::paramToArray( $args['data'] ); // parse the data arg into mappings

                $external_values = EDUtils::getLDAPData( $args['filter'], $args['domain'], array_values( $mappings ) );
-
-               // Build $edgValues
-               foreach ( $mappings as $local_var => $external_var ) {
-                       $edgValues[$local_var][] = $external_values[0][$external_var][0];
+
+               foreach( $external_values as $i => $row ) {
+                       foreach ( $mappings as $local_var => $external_var ) {
+                               if ( array_key_exists( $external_var, $row ) ) {
+                                       $edgValues[$local_var][] = $row[$external_var][0];
+                               }
+                       }
                }
                return;
        }

As a side-note I also had to add "ldaps://" in my $edgLDAPServer, too.

Hi - this looks good, but what bug does it fix? (I know very little about the LDAP handling.) Yaron Koren (talk)

On the extension page it says "Note that #get_ldap_data will only retrieve one result." - even though a full result was being fetched by LDAP the array format being constructed was not kosher. User:Memeyou

Oh yes, that makes sense - thanks. Yaron Koren (talk) 12:47, 2 August 2012 (UTC)

Error: Incomplete information for this server ID

Hi,

I am trying to retrieve data from a mysql database. I am currently getting the following: Error: Incomplete information for this server ID. I have been unable to figure out what I am doing wrong. Below are my current settings. Would you be able to tell me what I am doing incorrectly?

Thanks for any help,

Christy

LocalSettings.php:

## Database Settings for request_form
$edgDBServer['request'] = "localhost";
$edgDBServerType['request'] = "mysql"; 
$edgDBName['request'] = "request_form";
$edgDBUser['request'] = "user";
$edgDBPass['request'] = "password";

This is the get_db_data call located on my page:

{{#get_db_data:
db=request_form
|from=request_table
|limit=10
|data=name=name,lastname=lastname,email=email
}}

Server Config:

Apache:2.2.22
PHP Version: 5.3.13
MySQL Version: 5.524
MediaWiki 1.19.1

It should be "db=request", not "db=request_form", in the #get_db_data call. Yaron Koren (talk) 18:01, 2 August 2012 (UTC)

Thank you for your help. I made the change, but I am still getting the same error. I forgot to add that I am using MediaWiki 1.19.1. - Christy

What version of External Data are you using? Yaron Koren (talk) 19:03, 2 August 2012 (UTC)

Version 1.4 Cvmasson (talk) 19:13, 2 August 2012 (UTC)

Did you put those database settings in LocalSettings.php after the inclusion of External Data? Yaron Koren (talk) 19:37, 2 August 2012 (UTC)

Yes Cvmasson (talk) 19:39, 2 August 2012 (UTC)

Sorry - I have no idea, then. Yaron Koren (talk) 19:43, 2 August 2012 (UTC)

No problem I will keep working on it. Thank you so much for your help and time Cvmasson (talk) 19:46, 2 August 2012 (UTC)

Probs with special characters

Hi, I have a problem with special character like "é" or "è". When my data has a special character like "é" or "è" (UTF8), I can't display him. I don't know if the problem is when I display the data or if I request the database. Do you have any idea  ?

Thank you for your help.

Is this after a call to #get_db_data? And if so, what kind of database are you using (MySQL, etc.)? Yaron Koren (talk) 15:48, 27 August 2012 (UTC)

Yes, I call #get_db_data. It's a Mysql data. This problem appear only when the data contains the special character "é" et "è" and the page display nothing.

Hi - I can't duplicate this problem. What versions of MediaWiki, PHP and External Data are you using, and what operating system? Also, are you literally seeing a blank page? If so, then see here. Yaron Koren (talk) 18:03, 28 August 2012 (UTC)

Hi, Mediawiki 1.18.2, php 5.3.3, the last release of external data.Centos 6.2 is my operating system. I don't have à a blank page. I have my mediawiki page with the title but i don't have the content of my mediawiki page.

Hi i solve my problem, my database is in latin1, i use CONVERT( "MYDATA" using UTF8).

Ah, that's very good to know - thanks for the diagnosis. I'll try to add a setting for that in the code. Yaron Koren (talk) 16:28, 30 August 2012 (UTC)

variables in parameter url

Hi folks,

how to use a variable, suchs {{PAGENAME}} in the url-parameter of #get_web_data?

Thx for any hints.

Cheers, --Rolze (talk) 09:50, 29 August 2012 (UTC)

Ok, it works, you have to use the urlencodeversion {{PAGENAMEE}}. ;) --Rolze (talk) 10:56, 29 August 2012 (UTC)

Database connection

Hello, I'm doing a query to a external database and I don't get any results. Is there a way to know if I'm connecting with the data base? Thank you!!

Hello , you should sign your posts with "--~~~~", then it is more clear to the other users "who" is asking the questions and when. As for your problem, to what kind of database are you trying to connect? Br --Jongfeli (talk) 14:16, 17 September 2012 (UTC)

A Mssql database. Any idea??Thank you.--Miriam (talk)

Did you see the instructions on the main page for dealing with MSSQL databases? Yaron Koren (talk) 22:04, 17 September 2012 (UTC)

Yes, I followed all the steps on the main page but when I do a query I don´t get any results so I would like to know if I'm connecting with the database.In the debug_log I have this message:

DatabaseBase::query: Writes done: INSERT INTO `wikitext` (old_id,old_text,old_flags) VALUES (NULL,'{{#get_db_data:\ndb=PRUEBA\n|from=BPCUSTOMER\n|data=articulo=BPC.ACCCOD_0\n}}\n\n{{#external_value:articulo}}','utf-8') --Miriam (talk)

Which of the three options for MSSQL, if any, did you go with? Yaron Koren (talk) 12:42, 18 September 2012 (UTC)

Did you mean one of these??:

If the wiki is running on Windows, you can install Microsoft drivers for PHP for SQL Server.

If the wiki is running on Linux, the MSSQLBackCompat extension may help. If you use this extension, you will need to specify "mssqlold" (not "mssql") as the database type.

If you get messages at the top that look like, "Warning: Invalid argument supplied for foreach() in ...\includes\db\DatabaseMssql.php", this code change might help: in the MediaWiki file includes/db/DatabaseMssql.php, look for lines that look like:

I chose the first one. --Miriam (talk)

Yes, that's what I meant. I don't know, then - my strong guess is that PHP is somehow unable to connect to the database, but I don't know anything more than that. Yaron Koren (talk) 17:13, 18 September 2012 (UTC)

One question, is there any data in the result set of the Query you are using when you run it directly in MySQL WorkBench? As for PHP, when you are using MediaWiki with Apache 2.2 on in Windows OS not all version of PHP "work". Please read this. I don't have any experience with IIS. When installing PHP on a windows OS the Multi-Byte String option in the "Extensions" is not enabled by default but you need it for External Data. Please make sure the MySQL extension is also enabled but you already have a running wiki so that does not make sense. Where you able to setup your wiki from scratch? Where there any faults during the setup? If you are not sure just do a test setup again using only the root account of the MySQL instance. Don't enable any extension and make, delete, rename and move some pages to see if "all" is working fine, then enable External Data and see what happens. --Jongfeli (talk) 06:24, 19 September 2012 (UTC)

One more thing. I assume you know this but when your run only the Query in Mediawiki the data will not show, after the {{#get_db_data: call you need something like this (in table form):

{{#for_external_table:expression}}
{| class = "wikitable"
! Column 1
! Column 2
! Column 3
! Column 4{{#for_external_table:<nowiki/>
{{!}}-
{{!}} {{#external_value:Value 1}}
{{!}} {{#external_value:Value 2}}
{{!}} {{#external_value:Value 3}}
{{!}} {{#external_value:Value 4}}
}}

--Jongfeli (talk) 06:35, 19 September 2012 (UTC)

Using ODBC drivers on Linux

I have been trying to connect to a SQL Server database from MW 1.19 running on RHEL 5 and have been reading all the notes. I see that I cannot use the "mssql" database type because it only works on Windows. I installed the MSSQLBackCompat extension and changed to "mssqlold", but this has problems because php-mssql support is missing.

Given that this is RHEL 5 and I therefore cannot easily recompile PHP with the --with-mssql option (an assumption, mind you), and there is no php-mssql yum package for RHEL 5, it looks like I am stuck (i.e. the function mssql_connect, for example, is missing).

However, I note that Microsoft officially supports "SQL Server ODBC Driver for Linux", which in turn requires the use of unixODBC 2.3.x. Great! I installed both of those and can connect to my database using the command line.

I then used the RHEL 5 php-odbc package to install the ODBC library in PHP. I used this to connect from PHP to the database. Great! PHP ODBC supports unixODBC already.

The question is, can External Data be extended to support an "odbc" database type, so that we can use the features of External Data with the ODBC conenction to any supported database through the PHP ODBC library? Presumably, this would mean creating some sort of conenctor that calls the odbc_connect PHP functions of PHP ODBC, etc.

It would be cool if there were such a capabiity, and I assume it's possible - if someone is willing to create code to support such a thing, I would be glad to add it in. Yaron Koren (talk) 18:43, 4 October 2012 (UTC)

xpath & namespaces

Hi - is there any chance that you could add support for namespaces in the xpath, as I'm finding it impossible to parse xml otherwise...

• Sample page: [5]
• Sample XML: [6]

Errors returned when defining "|data=Text1=//ukm:AffectingTitle":

Warning: SimpleXMLElement::xpath() [simplexmlelement.xpath]: Undefined namespace prefix in /home/alexsken/public_html/foiwiki/extensions/ExternalData/ED_Utils.php on line 345 Warning: SimpleXMLElement::xpath() [simplexmlelement.xpath]: xmlXPathEval: evaluation failed in /home/alexsken/public_html/foiwiki/extensions/ExternalData/ED_Utils.php on line 345 --Skenaja (talk) 16:11, 11 November 2012 (UTC)

I just checked in a fix to External Data that gets rid of those namespace errors. I still can't get it to extract any data from that XML page, but hopefully that's a separate issue. Yaron Koren (talk) 17:38, 16 November 2012 (UTC)

JSON Results - Only one instance of a value displayed, not all received

Hello,

I have prepared a test page showing what happens, here: http://www.music-china.org/w/TESTPAGE. Basically I can call upon the Google v3 API and receive a number of results for a video search. However instead of all instances of a variable (videoId), I only get one result being displayed using

{{#for_external_table:

.

Not sure what I did wrong, any suggestions? --Azchael (talk) 15:09, 5 December 2012 (UTC)

Ooh, that's bad. JSON parsing got changed in the latest version, 1.4.2, which is what you're using. I just tried it out with the old code, from 1.4.1, and it works correctly with that one. So I'd recommend reverting to 1.4.1 (let me know if you need help with that one). The old code unfortunately produces a lot of PHP "notices" - I can fix that part, but for now you should just turn off display of notices, if you see them. Meanwhile, I'll look into fixing the code. Yaron Koren (talk) 20:26, 5 December 2012 (UTC)

Hi Yaron, thanks for the quick response. I changed back to 1.4.1 version and it works! (You can see on the TESTPAGE, I increased maxResults to 50 and voila). Can you let me know when you had the time to fix the code? Would love to use the latest version :). --Azchael (talk) 02:02, 6 December 2012 (UTC)

Hi Yaron, one more item I noticed. When I used version 1.4.1 and I create the caching wiki table "ed_url_cache" it doesnt work. He needs the table "wiki_ed_url_cache" in order to save the JSONs / URLs. When I check my API Stats on Google it seems we are still launching a cache request for every pageview even though the actual result should have been cached. Is the name "ed_url_cache" maybe used also for cross-checking request URL against cached URL? Because in that case we would have to change that to "wiki_ed_url_cache" as well, or? --122.100.154.118 03:48, 6 December 2012 (UTC)

The JSON stuff was fixed in version 1.4.3, which was finally released today. As for the DB table - there's only one; and if there's any problem with the name, you can (hopefully) fix it in LocalSettings.php. Yaron Koren (talk) 05:10, 3 January 2013 (UTC)