Extension:FileAnnotations/Design

This extension lets you add annotations to files. It currently only supports images but can later be used to add annotations to audio and video files too. It is an improvement on the ImageAnnotator gadget and currently has a similar feature set.

Goal

Text and photos and videos are great, but today we don't have the same kind of pervasive hyperlinking in media that makes disappearing down a Wikipedia clicking journey both fun and educational. - From T133526

To achieve this FileAnnotations might benefit by being more structured than ImageAnnotator. Commons has guidelines on what kind of annotations are considered good and what format makes sense given a particular context. Encouraging these guidelines through the user interface, and making it easier to do the right thing, while still keeping the contribution process simple is important.

Users and use cases

Personas are we targeting and what their use cases might be.

Active reader

Discover the feature and figure out how to use it .
Use the annotations for a visual learning experience for her students.
Get encouraged to add annotations of her own as the contribution doesn't require much time or effort.

New editor

Be able to contribute without learning about too many policies.
Share his know about modern design without the hassle of creating new articles.
Get guidance on what annotations work best.
See his annotations on an article, without having to edit the article. This might be motivate him to get deeper into editing.

Active editor

Quickly tag event photos with relevant bands as links.
Monitor and revert any vandalism on annotations.
Improve annotations by others, by turning text annotations to links, for example.

What is a good annotation?

The guidelines on Commons on what makes a good annotation are helpful for users, but we need to find guidelines on what is helpful for the entire system. I feel that in some cases a link to Wikidata item would serve better than a Wikipedia article or a commons category. The way we display the link to the Wikidata item can make us reach closer to our goal too (I am thinking something like Popups and not just a plain link).

Current state

Type of Annotation	Type of Link	Remarks	Wikidata?
Thing	Commons Category
Person	English Wikipedia	Apart from identifying the person, the annotation also has information about the event.
Detail in photo	None (text)		No
Text in photo	None (text)		No?
Text in photo	None (text)		No?
Zoom for detail	Commons file	If this were to be a Wikidata item, the Popup could have a small description and a link to see more photos
Zoom for detail	Commons file	It seems that images from Commons are most suited here. A Wikidata item is too general for this.	No
Thing	English Wikipedia	This one has a wide variety. Some are plain links to Wikipedia articles, some have extra information, like "Foot from Monty Python". In such cases Monty Python could be shown as a Hovercard and the rest as plain text. Much like chat applications do with pasted links.
Thing	Plain text	These should really be links...to something...
Thing	English Wikipedia	I couldn't find a WIkidata item for this. Could editing a note like this be an opportunity to add it?
Place	English Wikipedia + External links + Polski and English where available
People	Text + Link to User pages (not just on Commons)	All users shouldn't have Wikidata entries, right?

Annotation types

Given the current use cases we can roughly divide the types into the following:

Type	Source	Remarks	Do we want it?
Image	Commons		Yes
Category	Commons Category	Will structured meta-data have any affect on this?	Yes
Thing	Wikidata or Wikipedia item	While adding we need to show the user only the Wikidata result if there is one. Its easy enough to find out WP entries from there, but not the other way around. Once a Wikipedia article is added, adding more languages needs to be supported while editing.	Yes
External link			?
Location	Some geo template?	This is not for the location of the image (that could already be in the EXIF data) but the location of places in it. Might be helpful in skyline type photos.	?
Text		This is specifically for transcribing text in images. Not sure if it needs to differ from the Wikitext option.	Yes
Wikitext		We could either have this as a separate type, or have the "[[ ]]" icon for each type and let power users edit the Wikitext of pre-decided types too.	Yes, but in what form?

These types need not be explicitly chosen by the editor but can be inferred and then displayed accordingly. The interface however could however encourage links over text.

Possible solutions

Discovery

How will a reader find out that there are annotations on a particular file?

Current state (gadget/extension): Text note at the bottom of the file.
Current state (gadget): Icon on top the thumbnail indicating the file has annotations.
Flash the annotation rectangles for a short period of time after the page is done loading.
Flash the annotation rectangles as the image thumbnail scrolls into view in an article.
Load the annotations and keep showing them till the user doesn't mouse-in-out once.
Show a list of annotations on the page, separate from the image.

Viewing annotations

TextExtracts and PageImages for Wikipedia articles (like Hovercards).
Title, Image and Description for Wikidata items.
- See more images link, where applicable.
- See on map link, where applicable.
- Read more on Wikipedia.
Image from commons with a frame.
If its a text with a link like, "Foot from " then the Wikipedia page hovercard can be embedded below (similar to link embedding in chat applications.
An embedded map if the annotation has a geo template.

Encourage Annotating

How can we encourage annotating files, both readers and editors?

Show notification to add annotations based on the category of the image. Eg- people, map etc.
Are we running any face recognition or focal point recognition services? If yes, show a notification with a rectangle already around the face/focal point.

Annotating

How should the actual annotating process work?

Current state: Lets you add wikitext annotations.
Should we allow for structured annotations? A tab for each annotation type? It could save in wikitext like it already does, and infer the type later again based on the type of the link.
Don't allow anonymous users to add plain-text annotations?

En:Monty Python

Editing

Be able to see diffs and watchlist annotation pages (maybe automatically when you watchlist the File page).
Have links to quickly convert plain wikitext annotations into links.
Have links to quickly convert Wikipedia or Commons links to Wikidata links.

UI mockups for possible solutions
We could show the different annotation types with a description first and then open a different dialog based on type. I am not to keen on this as it makes the process longer, but it might be easier for beginners to understand what each type is good for.
We could alternatively open a dialog with tabs, similar to the "Insert media" dialog in VE, and give tabs for each type. The Wikitext tab could always be populated, no matter the type, so that power users can edit if they need to.
Flash annotations on file after the page has loaded
Alternatively, the annotations could remain visible till the user doesn't hover and un-hover over the image once
While on an article the annotation boxes could flash as you scroll by images
When images are in certain categories like people, places, we can encourage the reader to add annotations
When an annotation is a Commons category, it could show thumbnails of images from that category and a link to see more
When the annotation is to a Wikipedia article we could show a Hovercards-like interface
When the annotation has a geo template we can directly render a map, that can be linked to OSM or our own maps
When the annotation is to a Wikidata item, we can show the main image and description along with anything we can link to

Research and User testing