DESCRIPTION: This course addresses the design and implementation of high performance, embedded video processing systems. System design issues such as imager and VLSI technology, processor, memory, and I/O architectures, execution parallelism, image representations, and algorithms for scene recognition will be addressed in the context of a low-cost embedded surveillance system. Projects will include design and evaluation of a PC-based system implementation for specified surveillance applications (e.g., background/ foreground modeling, blob identification, object tracking, etc.).

COURSE OBJECTIVES: This course builds on prerequisite undergraduate material to apply recent research results in automatic scene analysis to embedded systems. Graduate students who complete the course will have a systems-level grasp of the complex interactions of algorithms, computing architectures, and technology. Example objectives include:

• impact of sensor quality and frame rate on processing requirements
• availability and exploitation of explicit parallelism in video surveillance algorithms
• color representation impact on storage and processing requirements
• impact of size-weight-power-cost constraints on system performance.

PROJECT DESCRIPTION: Students work in two person teams to design and implement a basic automatic surveillance system. The target system detects and tracks a reference object through a dynamic scene. Due to time limitations, systems are prototyped using PCs and USB webcams. The projects address different aspects of the overall system: (1) image capture and preprocessing, (2) background modeling, (3) object tracking, and (4) articulated motion analysis (e.g., tracking people, computing pedestrian flow, and activity classification). Design teams calibrate and evaluate their systems in the field. Each project is evaluated on outdoor scenes in class-wide demonstration/discussion periods.

PROJECTS

READINGS

PREREQUISITES: graduate standing

TEXTBOOKS (recommended):
Multimedia Technology for Applications, Eds. Sheu and Ismail, 1998 [ISBN 0-7803-1174-4],
Machine Vision, Snyder and Qi, 2004 [ISBN 0-521-83046-X],
and recent research/survey papers.

ASSESSMENT: final exam (25%), 3-4 projects (75% total)

CODE OF CONDUCT:

• Collaboration: The first project is to be completed individually. The remaining projects will be performed in 2-person teams. Students are encouraged to discuss project assignments, systems issues, and algorithms with other students in the class, particularly during group class discussions. However, sharing or copying code between teams is prohibited.
• Scholarship: You must credit the source of all algorithms or code implementations you use in your projects that did not originate with you or your project partner. This should be included as citations in the project documentation (e.g., if you use the Stauffer-Grimson mixture-of-Gaussians background modeling algorithm, provide a reference to it in the project documentation).
• Field Work: In all activities associated with this class, particularly in gathering videos for test input data, do nothing to harm yourself or others (including risking personal injury, causing embarrassment, or invading privacy).
• Final Exam: The final exam is to be completed individually with no collaboration or interaction during the exam period.

All conduct in this course will be governed by the Georgia Tech honor code. Additionally, it is expected that students will respect their peers and the instructor such that no one takes unfair advantage of anyone else associated with the course. Any suspected cases of academic dishonesty or nonacademic misconduct will be reported to the Dean of Students for further action.

INSTRUCTORS: Linda Wills and Scott Wills
OFFICES: Klaus Advanced Computing Building 3310 and 3312
PHONE: (404) 894-4565 and (404) 894-7469
E-MAIL: linda.wills@ece.gatech.edu and scott.wills@ece.gatech.edu

DETAILED TOPICAL OUTLINE:

1. Intro to Embedded Video Surveillance Systems
   • course objectives
   • summary of prerequisite material (what is needed for this class)
   • schedule and project team formation
   • anatomy of an embedded video processing system
2. Imagers (1 week)
   • physics of CCD imagers
   • adding color (Beyer patterns, alternatives)
   • array design: resolution, sensitivity, dynamic range, noise
   • optical design: focal length, aperture, field of view
3. Front-End Conversion and Preprocessing (1 week)
   • analog to digital conversion
   • gain control
   • noise filtering
   • color correction
   • using COTS cameras
   
4. Processing (1.5 weeks)
   • pixel representations (grayscale, RGB, HSI, YCrCb)
   • operation data types (integer, floating point, packed words)
   • parallel execution in COTS processors
   • parallel execution in FPGAs
   • energy efficiency: clock rate versus parallelism
5. Local Storage (1 week)
   • cache performance for stream processing
   • effective memory access patterns for DRAM
   • caching versus recomputation tradeoffs
   • static versus dynamic storage allocation
6. Transmission (0.5 weeks)
   • effectiveness of compression
   • disk storage versus network communication
   • bandwidth roundup: disks, wired networks, wireless
7. Image Filtering and Normalization (1 week)
   • median filtering in color images
   • contrast, brightness, and saturation
   • variations in outdoor scenes (e.g., cloud effects, shadows, illumination variations)
   
8. Background Modeling (1.5 weeks)
   • single mode versus multi-modal
   • adaptivity and selectivity in change detection
   • representation issues
   • mixture of Gaussian models
   • non-parametric models
   • occlusion problem
9. Activity Modeling and Recognition (1.5 weeks)
   • correspondence problem
   • blob detection
   • object recognition and tracking
   • classifying activity sequences
   • anomaly detection
10. System Integration (2 week)
    • functional specification
    • performance requirements
    • size-weight-power-cost
    • calibration and testing

    Click here

    Questions and comments to Scott Wills or Linda Wills
    last revised at 4:41 p.m. on 4 January 2008.