Semantic Image Retrieval

The Objective : The proliferation of digital photos has made efficient automation of image search more challenging than ever. Current image retrieval methods directly compare images by low-level perceptual characteristics such as color and texture. However, humans understand images by describing them with concepts, such as "building," "sky," and "people." This project employs artificial intelligence to identify semantic concepts in images using probabilistic models. No manual annotation of photos is required images are found using content recognition.

Methods/Materials

In this research, the visual feature distribution of each concept was modeled with a Gaussian Mixture Model learned using the Expectation-Maximization algorithm.

The models captured the dominant colors and textures of the concepts.

During retrieval, the models were used to determine the probability of each concept occurring in the images, producing a probability vector called a semantic multinomial (SMN). Images were retrieved by finding the most similar SMNs.

Results

This research has shown that by linking images with their underlying semantic meaning, the system understood the images and significantly improved retrieval accuracy.

It was also shown that a feature set consisting of YCbCr color values and Gabor texture filter responses had consistently better retrieval performance than Discrete Cosine Transform features.

A novel dynamic browser was also developed for exploring image collections based on semantic similarity.

It uses an animated spring graph layout algorithm to create self-organizing clusters of semantic concepts.

This provides a unique way to visualize the inherent semantic relationships in the image database, as well as to search for images by concepts.

Conclusions/Discussion

This technology has far reaching applications beyond organizing family albums. In the medical profession, the ability to quickly correlate unknown MRI images with that of known medical disorders is now within reach. In the area of natural resource exploration for oil, gas, and coal, remotely-sensed images can now be automatically related to images of known natural reserves.

The project used artificial intelligence models to associate visual features (color and texture) with images' underlying meaning, automatically annotating them with concepts like "buildings" and "water." I also created a unique dynamic image browser.

Science Fair Project done By David C. Liu

 

 

<<Back To Topics Page...................................................................................>>Next Topic

Related Projects : Osmosis and Diffusion , Pick-Up the Pace , Power House , Project Zier , Proving Universal Gravitation by Warping Space-Time , Red Clawed Crabs , Reflexes through the Ages , Rotating into the Future of Flight , Run from the Runoff , Semantic Image Retrieval , Sharing Spectrum the Smart Way , Sink or Float , Soy: Carcinogen or Prevention , Spin It to the Limit , Study of Asthma and Swimming