Social20: a groundtruth set for social image retrieval

Social20

The Social20 is a groundtruth set for tag-based social image retrieval, which was created as follows.

Queries: We select 20 diverse visual concepts as queries.
Labeling criterion: We consider a concept and an image relevant if the concept is clearly visible in the image and we shall relate the concept to the visual content easily and consistently with common knowledge. Therefore, toys, cartoons, painting, and statues of the concept are treated as irrelevant.
Data set: For each concept, we randomly select 1000 examples from images labeled with the concept in the Flickr-3.5M collection, and relabel them according to our labeling criterion.

Annotation.zip: Annotation files.
ImageData: 19,971 JPEG images, each has a maximum width or height of 500 pixels (see here for details). Copyright of each image is retained by its owner. The usage of the images is strictly limited to benchmarking research.
1. JPG files part-1 (~1GB)
2. JPG files part-2 (~1GB)
id.userid.tags.txt: Each line represents an image record, with photo-id, user-id, and tags separated by tab.
SearchResults.zip: For the ease of comparison, we provide two runs: TagBaseline created by retrieval without tag relevance learning, and TagRelevance created by retrieval with tag relevance learning (see our T-MM'09 paper for details).

We thank Arjan Setz for his contributions in creating the dataset.