Determining the aesthetic rating and weather of a location from Flickr photos and metadata

Abstract

The last decade has witnessed an unprecedented rise in the popularity of content sharing networks such as Flickr and Twitter. Shared photos are usually accompanied by metadata such as geo-location, timestamp and tags. These photos contain a digital representation of locations and they convey human behavior patterns, photo trails and tour summaries. The availability of such geographic information in the form of multimedia contents has given rise to interesting applications such as recommendation system, point-of-interest mining and tour planning system, user gender and home location prediction system and event recommendation systems. In this work, we have proposed a method to determine the aesthetic rating of a location and weather condition of an image from social metadata of Flickr photos and content analysis of Flickr images respectively.

The aesthetic rating of a location is the evaluation of aesthetic quality of that location. Tourists, artists and urban planners often seek to rate each location by their aesthetics. Popu-lar recommendation websites such as TripAdvisor generate a relative rating of the locations to provide recommendations to its users about possible locations to visit. However, such rankings are highly dependent on user contributions. Therefore, in this work, we have proposed a method to generate aesthetic rating of a location using social metadata of user captured and shared Flickr photos. A number of empirical features have been defined and computed from the social meta-data of the Flickr photos available at each location. Using these features numerous classifiers have been trained and our classifiers have been able to achieve notable accuracy.

On a different note, weather condition detection and tracking are in practice for a long time. However, most weather detection technologies rely highly on powerful hardware technologies and expertise of weather specialists. With the development of computer vision technologies

4

several attempts have been taken to recognize weather conditions from images. In this work, we leveraged the availability of user tagged images in Flickr to generate a image dataset with weather condition annotations. Using the dataset we have proposed, deep convolution network based solutions to detect weather conditions of a location from user tagged Flickr images.

We conduct comprehensive empirical analysis to investigate the performance of our pro-posed algorithms. We have gathered social metadata of Flickr photos of the locations of two major tourist destinations i.e. Rome and Paris. Our classifiers obtained about 80% accuracy in correctly predicting the aesthetic ratings of locations in Rome and achieved about 71% ac-curacy on Paris dataset. On the other hand we have considered four weather conditions in our weather detection task, namely, sunny, cloudy, rainy and snowy. We have trained several neural networks by varying hyper-parameters. Additionally, we have also applied transfer learning with popular pre-trained neural networks such as VGG16, InceptionV3, InceptionResnetV2 etc. Our classifiers have reported as much as 60% accuracy on our scrapped dataset.