Social Media Retrieval
ASCI A21: Multimedia Retrieval: Social Media Retrieval
Due to the subjective nature of social tagging, estimating the relevance of a social tag with respect to the multimedia content it is describing is a fundamental problem in social media retrieval. This lab session is about tag relevance estimation in the context of social image retrieval. Given a specific visual concept and images labeled with the concept by social tagging, the goal of social image retrieval is to rank images relevant to the concept in front of images irrelevant to the concept.
Assignments: When submitting the result of Assignment 5, please describe in brief the searchers you choose to combine and how they are combined.
Python API
Get tag relevance scores for an image (from smr11train)
>>> from datareader import TagrelReader
>>> reader=TagrelReader(collection='smr11train', feature='color64')
# Read the tag relevance score of the tag 'tiger' with respect to the image '118490626'
>>> score = reader.get(photoid='118490626', tag='tiger')
Run an image search engine
>>> from imagesearch import ImageSearcher
>>> searcher = ImageSearcher(collection='smr11train')
>>> searchresults = searcher.scoreCollection(concept='tiger')
# The following lines evaluate the search performance, measured by average precision
>>> photoid2label = readAnnotationsFrom(collection='smr11train',annotationName='concepts.txt', concept='tiger')
>>> computeAP(searchresults, photoid2label)
Evaluate an image search engine
>>> from imagesearch import ImageSearcher, evaluateSearchEngine
>>> searcher = ImageSearcher(collection='smr11train') # use your own searcher here
>>> [results, mAP] = evaluateSearchEngine(searcher, collection='smr11train', annotationName='concepts.txt')
Save image search results as html files
>>> import imagesearch
>>> searcher = imagesearch.ImageSearcher('smr11train') # use your own searcher here
>>> resultdir = 'mydir'
>>> imagesearch.submit(searcher, resultdir=resultdir) # search results are stored at resultdir
>>> import htmlwriter
>>> htmlwriter.searchresults2html(collection='smr11train', datadir=resultdir)
# Now you should see html files at resultdir