Research Interests

My overall interests are in computer vision, computer graphics, machine learning, and computational photography. More specifically, I am interested in image and video search, organization, manipulation, and enhancement. Below you will find papers, code, and datasets for some of the projects I have worked on. I welcome bug fixes (patches ideally), but I do not have time to give support for the code.

 

Research Projects

Video Event Detection

We developed volumetric features for event detection in crowded videos. We used spatio-temporal 3D shape and flow features to find human actions such as picking up an object from the ground, waving for the bus, and sitting down. The method works despite significant clutter and motion in the video.
Published in ICCV 2005, VS 2007, ICCV 2007, Ph.D. Thesis: CMU-CS-08-113.
Thesis Oral Slides and Videos.
Dataset for ICCV 07 paper. Code for reading and manipulating the dataset.

 

Photo Quality Assessment

We developed high-level semantic features to assess the quality of photographs. The system can classify between photographs taken by professionals versus snapshots taken by amateurs. A web image search engine can use the quality metric to place the best quality photographs near the top, giving the user the most relevant results.
Published in CVPR 2006.
Lecture slides.
Sample Results.

 

Music Identification

We used machine learning techniques to develop an audio fingerprinting algorithm for music identification. Given ten seconds of low-quality music recorded through a phone, our system can robustly identify the song in a large database of songs.
Published in CVPR 2005.
Video Demonstration.
Binary demo code and sample music: [mrdemo.tgz 16MB]
C++ server code: [musicretr-1.0.tgz]
Java GUI code: [mrgui.tgz]

 

Image Retrieval with PCA-SIFT

We developed a more distinctive local image descriptor for image retrieval. We then combined PCA-SIFT with locality sensitive hashing to enable fast and accurate sub-image retrieval over tens of thousands of image. The application can be used for image copyright enforcement.
Published in CVPR 2004, ACM MM 2004.
PCA-SIFT Code (descriptor only). May require netpbm libraries.
Matlab code for training PCA-SIFT.
Dataset for ACM MM paper.

Yan Ke