CS 4501: Introduction to Computer Vision

Course Summary

Lectures: TuTh 3:30 - 4:45 PM, Room: Rice 130

Office hours (Connelly): TuTh 1:00 - 2:00 PM, Rice 510, or by appointment via email
Office hours (TA Zachary Verham, Masters student): TuTh 2:00 - 3:00 PM, Rice 224, or by appointment via email, email is zdv8rb at virginia.
Office hours (TA Muthu Chidambaram, undergraduate student): Wed 11 AM-Noon, Fri 2-3pm, Rice 224, email is mc4xf at virginia.

Course Description

Computer vision is a topic within computer science that deals with how computers can gain understanding from images and video. Computer vision attempts to reverse-engineer what the human visual system can do. The understanding extracted by computer vision could include detection of human faces, recovering 3D geometry or depth from photographs, finding correspondences between objects that have moved within a photograph sequence or video, or labelling the different regions within a photo according to categories such as road, person, or car. The course will introduce both core topics in computer vision as well as ongoing research areas such as using deep learning for image understanding. The course will consist of lectures, programming assignments, quizzes, and a final project. Topics include:


• Image formation
• Edge and feature detection
• Segmentation
• Face detection
• Optical flow
• Stereo scene reconstruction
• Deep learning for image classification, segmentation, object detection
• Computer vision and natural language
• Autonomous driving

Textbook

• Computer Vision: A Modern Approach by Forsyth and Ponce
• Multiple View Geometry by Richard Hartley and Andrew Zisserman
• Computer Vision: Algorithms and Applications by Szeliski
• Deep Learning by Goodfellow, Bengio and Courville. See also [2].

Prerequisites

Requires experience writing code independently (for projects). We plan to use the Python language for the programming assignments, although other languages can be used if desired by the student. Students are expected to have completed CS 2150, or have sufficient experience programming, and have some knowledge of linear algebra. Experience with computer graphics or statistics will be useful, but not required. If department computing resources are used then knowledge of basic Linux commands will be helpful (e.g. ssh, cd, pwd, text editors, ...).

Grading

Grading will be based on:


• Quizzes at the end of class (25%) covering topics from between the previous quiz day (inclusive) and the current quiz day (exclusive);
• Three programming assignments (40%);
• A final course project (25%);
• In-class participation (10%).

Quizzes

Programming Policy

For the programming assignments, it is OK to seek the help of your peers or online resources. However, being able to write and debug your own code is very valuable for a computer scientist. Therefore, please submit code that you have individually authored. However, for the final project, you can work in groups.

Late Policy

Programming assignments and other Collab submissions are due at 11:59 PM. Late submissions are marked down 25% per day.

Lectures

Date	Topic	Textbook reading*	Paper reading	Assignments
1. Jan 19	Introduction	FP: Preface, S Ch. 1
2. Jan 24	Image formation	FP: I 1.1, S Ch. 2
3. Jan 26	Filtering and edge detection	FP II.4, S 3.1-3.2
4. Jan 31	Sparse feature detectors: Harris corner, difference of Gaussian	FP II.5, S 4.1
5. Feb 2	Sparse feature descriptors: SIFT. Hough transform, RANSAC, panorama stitching	Lowe 2004 (esp Sect 3-6)
6. Feb 7	Sparse feature descriptors (continued)			Quiz 1 (in class)
7. Feb 9	Linear algebra refresher and transformations	FP: I 1.2, S 2.1
8. Feb 14	Camera geometry, calibration (same slides as stereo)	FP: I 1.2, S 6.2-6.3
9. Feb 16	3D vision overview, stereo	S 11.1, 11.3, 11.4, blog		Project 1 due (edge and corner detector)
10. Feb 21	Basics of machine learning, supervised and unsupervised learning, classifiers	GBC 5.1-5.8
11. Feb 23	Vicente Ordonez: data-driven methods for vision
12. Feb 28	Structure from motion (presented from Photo Tourism slides 1-30)	S 7.1-7.2, 7.4, FP 8.1		Quiz 2 (in class)
13. Mar 2	Optical flow, face detection / PCA / eigenfaces	S 8.4, 14.2.1, FP 10.6.1
Spring recess: March 4 - 12
14. Mar 14	Intro to neural networks	GBC 6-6.2
15. Mar 16	Training neural networks (same as previous slides)	GBC 8-8.3		Project 2 due (stereo matching, panorama stitching) on Mar 17
16. Mar 21	More neural networks: CNNs, Keras	GBC 9-9.4		Quiz 3 (in class)
17. Mar 23	Details of training neural nets	GBC 8.4, 8.7.1
18. Mar 28	RNNs	GBC 10-10.2
19. Mar 30	Final project proposals I
20. Apr 4	Final project proposals II			Final project proposal document due
21. Apr 6	Reinforcement learning	DQN (video), VisualDoom winner (video), AlphaGo		Project 3 due (neural networks)
22. Apr 11	Object localization, detection, semantic segmentation	FCN, SegNet, YOLO v1
23. Apr 13	(Finish previous topic)
24. Apr 18	Visualizing CNNs, style transfer, inpainting	Deep dream, Gatys et al. 2016, Barnes et al. 2009
25. Apr 20	Computer vision + natural language	Vinyals et al. blog, paper, iBOWIMG		Quiz 4 (in class)
26. Apr 25	Applications: AR/VR	LSD-SLAM
27. Apr 27	Applications: autonomous driving, medical image diagnosis	End-to-end self-driving
28. May 2	Final project presentations (3 hour class, 3:30 to 6:30, CHM 304)
29. May 7				Final project writeup+code due, 11:59 PM

Final Project Sample Topics

Final Project Writeup and Code

• Would you like your final writeup to be posted on the course website?
• A description of the goal of the project.
• A list of students who were part of the project, and what contribution was made by each student.
• A description of which milestones were achieved and which challenges if any were encountered.
• Figures for results, e.g. performance and confusion matrices of classifiers, images produced for graphics projects, other visualizations, etc.
• Explanations or figures for the architectures of neural networks used (if appropriate)
• Any other observations you would like to make.