Final Report — Google Summer of Code 2020 with Wikimedia Foundation

Abstract

As Google Summer of Code 2020 comes to an end, I cannot help but think how accelerated these months felt. From the excitement of getting accepted into the program, interacting with the project mentors for the first time, pressure of preparing for the weekly code reviews to those enjoyable meetups of Wikimedia, the last three months have truly been a roller coaster ride for me. Having no formal experience in writing quality software, this program taught me the essence of structured programming workflow starting from code planning to writing tests and completing documentation. Technical writing was something I was just not familiar with, but after this program I am much better with the process. All in all, under the able guidance of extremely helpful mentors Tim Moody and George J Hunt, I can definitely say that Google Summer of Code 2020 was a wonderful experience for me and helped me expand my technical skills by leaps and bounds. This report summarises the highlights of my project - 'Design and implement a tool to create overlays on pre-rendered offline maps for Internet In A Box' .

Internet In A Box

My project is a basically an enhancement feature to the Internet In A Box. Internet-in-a-Box is software solution that can be implemented in a small, low-cost device to provide essential Internet resources without any Internet connection. It provides a local copy of a terabyte of the world’s Free information. It includes Wikipedia in 37 languages, a library of 40,000 e-books, most of the world’s open source software and source code, hundreds of hours of instructional videos, and world-wide mapping down to street level.

Project Summary

As mentioned, one of the core features of IIAB is the ability to render offline maps of a selected region based on a user’s preference. This project aims to develop functionality to display an overlay of local information like medical facilities, disease outbreaks, important properties, etc. This can enhance the readability of the map for the user and provide useful information in times of need for them.

Timeline of Feature Addition

Phase I :

Query from Wikidata:

The most important deliverable for Phase 1 was to develop a functionality to get GeoJSON output from SPARQL queries. The goal was to write a number of usable queries for Wikidata regarding information that would be useful for any number of users in a particular location. This included queries regarding nearby hospitals, airports, railway stations etc.

GeoJSON on Maps:

To visualise those queries, there was a drag-and-drop component from Open Layers in place, where we could drop any GeoJSON file, and that would generate the markers on the map for the specified points. For conversion to GeoJSON, we used an online tool called geojson.io, that would help in the process.

Styling of Markers:

The markers generated by the Openlayers library, were quite basic in nature, and would provide difficulty in putting distinguishable queries on the map. In other words, we needed hospitals to be signified by a different set of markers, and railway stations with a different one. Hence, the Style-function component of Openlayers was utilised :

Screenshots :

OSM in IIAB

OSM in IIAB

Detailed blogpost for this phase can be found here.

Phase II :

Overlay on Map with additional information:

An additional overlay layer was added to the map base, that would display information about that point on clicking it. The information is sourced from Wikidata and any sort of information available there would be visible on the overlay. Most useful ones being the image( if available), the coordinates of that point, and description of the marked pointer, as given in Wikidata. The overlay content is stored offline, and is extracted from Wikidata during query generation. The points beneath the overlays was not visible clearly. So that was fixed. Instead of having a one-size-fits-all strategy for all sorts of markers, a separate icon library was put together, and was linked to the automation script. So instead of hard-coding values, the query type entered during generating queries from the automation script, would decide the icon file to be chosen. A template structure containing all the available queries was made that contains the details and the associated files for any given query.

Automation Script for users to generate Wikidata queries :

Prior to Phase 2, we were using the open source tool geojson.io, to generate GeoJSON files from the SPARQL queries. It was essential to develop a functionality to do that conversion as part of the process, without the need for an external tool. Hence, a Python Script was developed that would take in arguements (via Argparse) consisting of values for anything that could parameterize a given query. This included values for latitude, longitude, radius , central coordinate and limiting the number of queries for a particular region.With the script in place, the following command generates a GeoJSON file and saves it in the specified directory:

python3 sparql-to-geojson-final.py railway-station railway 58.9656 -66.5525 50 50

Phase 3

Admin Console page for Users to download Wikidata Query:

Finally, with an overlay enhancement feature on the map, Phase 3 dealt with developing a User Interface for embedding an Overlay for a desired location on the map. The UI includes a form for taking in the latitude and longitude values for any given location. Alongside that, there are fields for entering desired values for radius, query limit and output file name for the generated query. For any given set of user inputs, there are validation rules for legal values. For example, checking the latitude and longitude for being in the correct range, checking for empty fields,etc. Once the correct values are entered, the user presses the Submit button and a Python command for the entered values is generated. This Python command is required to be pasted in the Terminal,upon which the resultant GeoJSON file are copied to the correct location, and are visible on the map area.

Integrating Map for Users to select Latitude Longitude by clicking on Map

Once the User Interface was developed, an embed of a clickable world map was added to better enhance the experience. So now, instead of entering the latitude and longitude values manually, the user can conveniently click on the desired location on the map, and get the values in the respective coordinate fields.

Screenshots :

Satellite region for selecting Latitude Longitude values

Satellite region for selecting Latitude Longitude values

Upcoming Changes

For power users, having a text field in the Admin Console page, for directly entering the SPARQL query, would be a good idea. This would enable them to take control of the queries to a greater extent. This is a stretch goal and I plan to integrate this feature in the month of September.

Code Submission :

Github Repository: https://github.com/iiab/maps

Working Repository : https://github.com/siriuswapnil/maps/tree/7.2-devel-osm-working

Phase 1 Commits :

https://github.com/iiab/maps/pull/21/commits/6071466427fb5afed0a9fce6ba4bcd049ef35449

Phase 2 Commits:

https://github.com/iiab/maps/pull/21/commits/86f10a711f0d07aa2c8a4fbcf12c3b92b7b0ead0

Phase 3 Commit :

https://github.com/iiab/maps/pull/21/commits/db6e136467e49e5c6d925371e8eb1e0eb5b5c943

Final PR to original codebase :

https://github.com/iiab/maps/pull/21

Conclusion

All in all, Google Summer of Code 2020 gave me a wonderful opportunity to work on an impactful product and make my contribution count. It has certainly made me a better developer and has given a new dimension to open source software for me. My mentors Tim and George were instrumental in making this program a success for me and guiding me along the way. I had a great time being a part of Wikimedia Foundation and hope to be a long term contributor various projects in and out of the organisation.