CS782 - Pattern Recognition and Applications - Fall'2020

• Study of recent advances in development of statistical pattern recognition algorithms, approximation, and estimation techniques. Topics include statistical estimation theory (decision rules and Bayes error), classifier design, parameter estimation, feature extraction (for representation and classification), clustering, statistical learning theory, support vector machines and other kernel methods, and various applications.  Additional topics include nonparametric density estimation, nonparametric classifier design, machine learning for pattern recognition, and evolutionary computation for pattern recognition.

• Prerequisites: CS 610 – Data Structures and Algorithms

• K. Fukunaga, Introduction to Statistical Pattern Recognition, 2nd edition, Morgan Kaufmann, 1990.
• C.M. Bishop, Pattern Recognition and Machine Learning, Springer, 2006.
• R.O. Duda, P.E. Hart, and D.G. Stork, Pattern Classification, 2nd edition, John Wiley & Sons, 2001.
• V. N. Vapnik, The Nature of Statistical Learning Theory, 2nd edition, Springer, 2000.
  
• Selected papers and handouts.
  

1. Introduction
• Pattern Recognition Fundamentals
  
• Formulation of Pattern Recognition Problems
• Major Components of a Pattern Recognition System
• Related Fields: Machine Learning, Neural Networks, Statistical Learning Theory
2. Bayes Decision Theory - The Bayes Decision Rule for Minimum Error
• Posterior Probability Form, Likelihood Ratio Form, Discriminant Function Form
• The Bayes Error
• Quadratic Discriminant Analysis (QDA)
• Linear Discriminant Analysis (LDA)
3. Bayes Decision Theory - Other Decision Rules
• The Bayes Decision Rule for Minimum Cost
• The Neyman-Pearson Decision Rule
• The Minimax Decision Rule
  
4. Bayes Decision Theory - the Bayes Error
   
• Error Probability and the Bayes Error
  
• Upper Bounds on the Bayes Error
• Chernoff Distance and Bhattacharyya Distance
  
5. Parametric Classifier Design
• The Bayes Classifier
• Linear Classifier Design and Examples
  
• Quadratic Classifier Design
• Piecewise Classifier Design
  
6. Parameter Estimation
• Maximum-Likelihood Estimation
• Bayesian Estimation
7. Feature Extraction and Mapping for Representation
• Optimal Feature Representation Methods
• Principal Component Analysis
  
• Similarity Measures and Optimal Feature Representation
8. Feature Extraction and Mapping for Classification
• Optimal Feature Classification Methods
• Discriminant Analysis
• Similarity Measures and Discriminant Analysis
  
9. Statistical Learning Theory (SLT)
   
• Structural Risk Minimization (SRM)
  
• Support Vector Machines (SVM)
• More Kernel Methods - Kernel PCA, Kernel Fisher Analysis (KFA)
  
10. Clustering
    
• Parametric Clustering
• Nonparametric Clustering
11. Nonparametric Density Estimation (optional)
    
• Parzen Density Estimation
• KNN Density Estimation
• Expansion by Basis Functions
  
12. Nonparametric Classifier Design (optional)
• Parzen Approach and its Error Estimation
• KNN Approach and its Error Estimation
  

Midterm exam or short presentation 20%

Project and presentation (topics are related to our course Contents) 40%

Final exam or term paper 30%

Class attendance and participation 10%

• “Academic Integrity is the cornerstone of higher education and is central to the ideals of this course and the university. Cheating is strictly prohibited and devalues the degree that you are working on. As a member of the NJIT community, it is your responsibility to protect your educational investment by knowing and following the academic code of integrity policy that is found at: http://www5.njit.edu/policies/sites/policies/files/academic-integrity-code.pdf.   
  
  Please note that it is my professional obligation and responsibility to report any academic misconduct to the Dean of Students Office. Any student found in violation of the code by cheating, plagiarizing or using any online software inappropriately will result in disciplinary action. This may include a failing grade of F, and/or suspension or dismissal from the university. If you have any questions about the code of Academic Integrity, please contact the Dean of Students Office at dos@njit.edu”

The “Best Practices” document @ http://www5.njit.edu/provost/sites/provost/files/lcms/docs/Best_Practices_related_to_Academic_Integrity.pdf.