My Research

  • Particle Filtering for Continuous Curves (represented using the Level Set Method)
    I am now studying implicit representations of continuous curves as zero level sets of a higher dimensional function. These can automatically handle changes in curve length and also can deal with more complicated changes in curve topology over time. Level set methods have been used successfully in segmentation and registration problems in medical images. We are working on a particle filtering algorithm for tracking deforming objects using a level set representation of the curve. The level set representation is infinite dimensional and hence the main challenge here is how to generate random samples from a stochastic state space model for the curve deformation. Most past work on shape filtering, including my own work, is for parametric representations of curves, which are finite dimensional and hence it is easy to add noise in the deformation model. I would like to explore using a time-varying finite dimensional approximation of the curve as the state vector. Both the state space dimension and the basis are allowed to change slowly with time. The observation is still represented using the infinite dimensional level set representation. The finite dimensional basis of the state space (both the dimension and the basis) is updated whenever the current approximation is unable to track the observations with sufficient accuracy. Two possibilities for approximating the curve are time-varying number of landmarks or time-varying principal components of possible variations of the curve.

    I am also studying existing work on Riemannian distances on the space of continuous curves and finding practical algorithms to evaluate mean and principal subspaces w.r.t. these distances. This can be used for shape classification and clustering in medical image analysis.

  • Change Detection in Partially Observed Nonlinear Dynamic Systems With  Unknown Change Parameters
    We study the change detection problem in partially observed nonlinear dynamic systems. We assume that the change parameters are unknown and the change could be gradual (slow) or sudden (drastic). For most nonlinear systems, no finite dimensional filters exist and approximation filtering methods like the Particle Filter are used. Even when change parameters are unknown, drastic changes can be detected easily using the increase in tracking (output) error or the negative log of observation likelihood (OL). But slow changes usually get missed. We propose in this paper, a statistic for slow change detection which turns out to be the same as the Kerridge Inaccuracy between the posterior state distribution and the normal system prior. We show asymptotic convergence (under certain assumptions) of the  modeling and particle filtering errors in its approximation using a particle filter optimal for the unchanged system. We also demonstrate using the bounds on the errors that our statistic works in situations where observation likelihood (OL) fails and vice versa.
            Simulation results for using ELL and OL for change detection are shown first for simulated examples and for a real abnormal activity detection problem described below. We also discuss below application of ELL for activity segmentation. ACC'04 Paper
            We generalize the above problem and study the errors in particle filtering with incorrect system model parameters. We quantify the "incorrectness" in the model  by a distance metric between the correct and incorrect state transition kernels (referred to as system model error per time step). The total error in approximating the posterior distribution of the state given noisy observations, can be split into modeling error and particle filtering error in tracking with the incorrect model. We show that the bound on both errors is a monotonically increasing function of the error in the system model per time step. The bound on the particle filtering error blows up very quickly since it has increasing derivatives of all orders.We apply this result to bounding the errors in approximating our statistic for slow change detection, where it implies that the approximation of the change detection statistic is more accurate for slower changes.   ICASSP'04 Paper  
     
  •  "Shape Activities": Dynamic Stochastic Models for Moving/Deforming Shapes with Application to Abnormal Activity Detection
    •  
    The aim is to model "activity" performed by a group of moving and interacting objects (which can be people or cars or different rigid components of the human body) and use the models for abnormal activity detection. We treat the objects as point objects (referred to as `landmarks' in shape theory literature) and model their changing configuration as a moving and deforming "shape" using ideas from Kendall's shape theory for discrete landmarks. A continuous state HMM (Hidden Markov Model) which takes the objects' configuration as the observation and the shape and motion as the hidden state, is defined to represent an activity and called a "shape activity". Particle filters are used to track the HMM i.e. estimate the hidden state (shape, motion) given observations. An abnormal activity is then defined as a change in the shape activity model, which could be slow or drastic and whose parameters are unknown and strategies for change detection described above are used.
                I have applied this framework to model the "activity" of passengers deplaning and walking towards an airport terminal. Here, the configuration of passengers in an image frame defined the shape at the current time. For this application, a stationary shape activity (SSA) model was sufficient. Shape variation around a constant mean shape was modeled by a linear Gauss-Markov model in the tangent space to the shape manifold at the mean shape. I am also trying to apply this framework to modeling human actions in which case each rigid component of the human body is a landmark. A non-stationary shape activity (NSSA) model is required in this case. Since the mean shape is time-varying, one needs to define the time derivative of shape (or shape "velocity") which is defined in the tangent space to the shape manifold at the current shape. NSSA is a more flexible model which is able to track abnormalities (unmodeled shape changes) better and yet detect them using ELL.
            Now certain other kinds of activities or a sequence of different activities could be modeled more accurately by piecewise stationary shape activity (PSSA) models where the mean shape is set as the current shape either at pre-decided time instants or at change times decided on the fly using ELL. The PSSA model is more specific than NSSA but more general than SSA. The activity segmentation problem - breaking a long sequence into stationary pieces whenever a change is detected, can be tackled using PSSA with unknown change times. Once segmented, parameters of each stationary model can be learnt (unsupervised training).
            Also, our approach is sensor independent, the observation vector of object locations could be obtained by motion detection on a video sequence or using an infra-red, radar or acoustic sensor. The framework has potential applications in robot formation control, which I am very interested in exploring. Also, the particle filtering framework can be adapted easily for multi-sensor fusion (combining observations from multiple sensors to get more reliable estimates of shape).    Papers  
     
  • Principal Component Null Space Analysis for Image and Video Classification
     We present a new classification algorithm, Principal Component Null Space Analysis (PCNSA), which is designed for "apples from oranges" type classification problems like object recognition where different classes have unequal and non-white noise covariance matrices. PCNSA first obtains a principal components subspace (PCA space) for the entire data in order to maximize the between-class variance. In this PCA space, it finds for each class 'i', an M_i dimensional subspace along which the class's intra-class variance is the smallest. We call this subspace an Approximate Null Space (ANS) since the lowest variance is usually ``much smaller'' than the highest. A query is classified into class 'i' if its distance from the class's mean in the class's ANS is a minimum.
            We derive tight upper bounds on classification error probability. We use these expressions to compare classification performance of PCNSA with that of Subspace Linear Discriminant Analysis (SLDA). We propose a practical modification of PCNSA called progressive-PCNSA that also detects `new' (untrained classes). Finally, we provide a brief experimental comparison of PCNSA, progressive-PCNSA and SLDA for three image classification problems - object recognition, facial feature matching and face recognition under large pose/expression variation. We also show application of PCNSA to two classification problems in video - an abnormal activity detection problem and an action retrieval problem.  Papers
     
  • Infra-red Image Compression and Characterization in the Wavelet Domain (June'00-Dec.'00):
    This involves characterization of the mutual information of wavelet subbands of IR images and development of compression algorithms for 2D predictive DPCM in which the predictor coefficients are chosen to maximize the correlation between predicted and actual pixel values assuming a joint probability distribution model for the images obtained from the characterization. I have experimented with combining non-iterative zerotree coding with 2D predictive DPCM (assuming  the standard two dimensional first order Gauss-Markov model for the images) for compression of the wavelet subbands and only DPCM for the low frequency subband. Papers

    •  
  • Hardware Compatible and Realtime Algorithms for Object Detection, Best View Selection and Compression of Moving Objects in IR Image Sequences (Jan'00-May'00) :
    An end to end system for target chip detection, best view selection and compression in surveillance applications was developed. Moving target detection was done by segmenting the image function using measures of its local singularity, the measure used was derivatives of Gaussian along X and Y axis. Since the available transmission bandwidth was very small, a single image chip had to be selected for transmission. Three different approaches were attempted :- Classification in eigenspace, Focus of expansion estimation and Size based methods. Various algorithms for fast image compression were compared:- Laplacian-Gaussian pyramid versus Haar wavelets for subband decomposition, Geometric Vector Quantization versus Scalar Quantization for quantizing the wavelet subbands and 2D-DPCM for quantizing the low pass coefficients. Report

    •  
  •  Gesture-Driven Control of Spaces and Objects in Collaborative Augmented Reality (Aug-Dec 1999): Worked on developing a gesture based interface for controlling objects in a virtual world. Completed segmentation of a gloved hand at every frame . The centroid of the segmented hand gave the x and y coordinates of hand position and scale information was used to estimate the z coordinate. The motion of objects in the 3-D virtual world (interface developed using Open Inventor) followed the hand motion.
    •  

    Graduate Course Projects

  • Term paper on Convolutional Coding and its Relation to Minimal Realization of LTI Systems
  • Testing various approaches for Image Scaling and obtaining the maximum scale-up limit for each approach
  • Design of MPEG-2 based Video Encoder and Decoder
  • Design of Perfect Reconstruction QMF banks and using them for Subband Filter Bank based Signal and Image Compression
  • Implementing the Levinson-Durbin Recursion Algorithm for estimating All-Pole filter or AutoRegressive model coefficients and using it for Signal Modeling and Compression


    • Undergraduate Major Project

  • A 2-D Hand Gesture Recognition System for Robot Control (Aug'98-May '99) : This involved skin color based hand segmentation, static hand shape recognition using PCA and a contour discriminant based approach, Kalman filter based hand tracking to obtain scaling and translation at every frame and Hidden Markov model (HMM) based dynamic gesture recognition using the static shape , scale and translation parameters as observations. This lead to a journal paper in the journal `Pattern Recognition':  Paper

    •   
    Talks