CSCI 252: Neuromorphic Computing
General Information
 |
 |
 |
Professor: Simon D. Levy Lecture: MWF 12:15-1:15pm Parmly 405 Office: Parmly 407B E-mail: simon.d.levy@gmail.com Office Hours: MWF 2:15-2:45 / 3:45-4:30, and by appointment |
Textbook: There is no required textbook for this course. In my experience, textbooks on this topic are either (1) very expensive; (2) unnecessarily mathematical; (3) incomplete; or (4) some combination of the above. We will rely instead on primary source material – both classic and contemporary – available online.
For those interested in a lively introduction to neuroscience, I recommend David E. Presti’s book. The lecture notes on Self-Organizing Maps and the Hopfield Network come from Judith Dayhoff’s excellent (but unfortunately out-of-print) introductory textbook. For the final topic in our course (biologically realistic modeling) I will be drawing heavily from Chris Eliasmith’s How to Build a Brain.
Course Objectives
By the end of the course you will be able to
-
- Explain the history of neural network research and critique arguments for and against neural networks, both practical and theoretical.
- Implement popular neural-network algorithms in Python.
- Look at a computational problem and determine whether / which one of these approaches is appropriate for solving it.
- Describe the goals of neuromorphic hardware movement and write programs for simple neuromorphic devices.
We will take an “architecture-of-the-week” survey approach to the material, attempting to cover one kind of neural net each week.
CSCI 252 will not cover the currently hot field of Deep Learning, for which we offer another course. CSCI 252 will however serve as excellent preparation in the skills and concepts used in the more advanced course.
Professionalism
At this point in your CS career you probably realize that what matters most is the quality of the work (software) you create and the impression you make as a serious student and software professional. Hence:
- If your code produces a syntax error or fatal runtime error in Python3 you’ll get a zero on that part of the assignment, as well as any other parts that depend on that program. You will get no credit for a program that runs under Python2 but not Python3.
- No late work will be accepted, or make-up exams given, without a note from the Dean’s office. You will get no credit for an assignment emailed to me, or submitted through anything other than github.
The best way to avoid the “But the program worked when I ran it!” problem is to submit your code early to github, clone it from github into an empty folder, and run it yourself.
Grading
- Programming assignments, submitted through github. Though there is no official lab for this course, you will spend much of your time programming (50%).
- Two one-hour in-class exams (30%).
- Comprehensive final exam, during exam week (20%)
These percentages are flexible. If you have a bad day and don’t do well on an exam or assignment, I will try and weight your other work more heavily.
The grading scale will be 93-100 A; 90-92 A-; 87-89 B+; 83-86 B; 80-82 B-; 77-79 C+; 73-76 C; 70-72 C-; 67-69 D+; 63-66 D; 60-62 D-; below 60 F.
Honor System
The exams will be done without books or notes and without assistance from other people. You can discuss your programming assignments with other students in the class, but you should not share code. Talk to me before asking another student for help!
Final Exam
The final exam for this course will be given during the final exam week. You can take this exam during any of the regularly scheduled exam periods that week. You must supply an exam envelope to the instructor or the department administrative assistant no later than noon on the last day of class. You must specify a provisional day and time on the envelope, which you are free to change on the clipboard provided outside the door of Parmly 407 any time that week. Email or phone requests to reschedule will not be accepted.
The exam will be given in Parmly 405, and you should arrive promptly before the appointed time. If you are more than 15 minutes late, you will have to reschedule your exam. If you are more than 15 minutes late to the last exam period on Friday morning, you will receive a grade of 0 on your exam.
Students who have approved academic accommodations must make arrangements to use those accommodations directly with the instructor no later than the last day of class. Students approved for extra time will receive that time at the tail end of the morning exam period or before the beginning of the afternoon exam period (for example, ending at 1:30 PM for a morning exam or beginning at 12:30 PM for an afternoon exam). Students approved for a low-distraction testing location should reserve that space during the last week of classes (following instructions distributed by Director of Disability Resources Lauren Kozak.
Accommodations
Washington and Lee University makes reasonable academic accommodations for qualified students with disabilities. All undergraduate accommodations must be approved through the Title IX Coordinator and Director of Disability Resources, Elrod Commons 212, (540) 458-4055. Students requesting accommodations for this course should present an official accommodation letter within the first two weeks of the term and schedule a meeting outside of class time to discuss accommodations. It is the student’s responsibility to present this paperwork in a timely fashion and to follow up about accommodation arrangements. Accommodations for test-taking must be arranged with the professor at least a week before the date of the test or exam, including finals.
Programming assignments
Programming assignments should be submitted to the private repository I have set up for you on github, using a separate folder for each assignment. Your repository name is https://github/com/simondlevy/<username>, where <username> is your W&L network user name (for example, smithj22). Each assignment is due 11:59 PM on the day posted in the schedule below.
The computers in the CS Department should all have NumPy and matplotlib installed, but I encourage you install them your laptop or home computer and work on these assignments anywhere you like. Experience has taught me that the students who do best in this course are the ones who start the assignments early and take advantage of office hours for help.
Tentative Schedule of Lectures and Exams
|
Monday |
Wednesday |
Friday |
|
| 06 Sep Week 0 |
Course Overview; Philosophical Foundations Reading: Dragonfly Brains (2021) |
||
| 13 Sep Week 1 |
Reading: The Appeal of Parallel Distributed Processing (1986) | Dot Product Lecture Notes | Dot Product Lecture Notes |
| 20 Sep Week 2 | Architecture #1: Kohonen’s Self-Organizing Map
Due: Assignment #1 |
Lecture notes
Reading: Törmä 1994 Kohonen SOM, continued |
Reading: Zhao, Li, Kohonen 2010 |
| 27 Sep Week 3 |
Architecture #2: The Hopfield Network
Reading: Hopfield 1982 |
Reading: Hopfield & Tank 1985 | Student’s don’t know …
Due: Assignment #2 |
| 04 Oct Week 4 |
Architecture #3: Sparse Distributed Memory
Reading: Denning 1989 |
Lecture notes
Reading: Denning 1989 |
Pre-exam review |
| 11 Oct Week 5 |
In-class Exam #1 | After-exam review | Reading Day; no class |
| 18 Oct Week 6 |
Architecture #5: Latent Semantic Analysis
Reading: Landauer & Dumais (1997) Due: Assignment #3 |
LSA | LSA |
| 25 Oct Week 7 |
Architecture #6: Tensor Products
Reading: Smolensky 1990 |
Tensor Products (continued)
Reading: Gayler (2003) |
Tensor Products II |
| 01 Nov Week 8 |
Due: Assignment #4 | Review for exam | In-class Exam #2 |
| 08 Nov Week 9 |
Exam #2 follow-up
Architecture #7: Biologically realistic neural networks |
Readings: | Due: Assignment #5 |
| 15 Nov Week 10 |
NEF | NEF | NEF |
| 29 Nov Week 11 |
SPA: Lecture notes | Neuromorphic Hardware
Reading: The Physicality of Representation |
Neuromorphic Hardware
Due: Assignment #6 |
| 05 Dec Week 11 |
Group Exercise: Nengo | Group Exercise: Event Camera | Review for final exam |