CSCI 252: Neuromorphic Computing

CSCI 252: Neuromorphic Computing


General Information


Professor: Simon D. Levy
Lecture: MWF 12:15-1:15pm
Parmly 405
Office: Parmly 407B
E-mail: simon.d.levy@gmail.com
Office Hours: MWF 2:15-2:45 / 3:45-4:30, and by appointment

Textbook: There is no required textbook for this course. In my experience, textbooks on this topic are either (1) very expensive; (2) unnecessarily mathematical; (3) incomplete; or (4) some combination of the above. We will rely instead on primary source material – both classic and contemporary – available online.

For those interested in a lively introduction to neuroscience, I recommend David E. Presti’s book. The lecture notes on Self-Organizing Maps and the Hopfield Network come from Judith Dayhoff’s excellent (but unfortunately out-of-print) introductory textbook. For the final topic in our course (biologically realistic modeling) I will be drawing heavily from Chris Eliasmith’s How to Build a Brain.


Course Objectives

By the end of the course you will be able to

    1. Explain the history of neural network research and critique arguments for and against neural networks, both practical and theoretical.
    2. Implement popular neural-network algorithms in Python.
    3. Look at a computational problem and determine whether / which one of these approaches is appropriate for solving it.
    4. Describe the goals of neuromorphic hardware movement and write programs for simple neuromorphic devices.

We will take an “architecture-of-the-week” survey approach to the material, attempting to cover one kind of neural net each week.

CSCI 252 will not cover the currently hot field of Deep Learning, for which we offer another course. CSCI 252 will however serve as excellent preparation in the skills and concepts used in the more advanced course.


Professionalism

At this point in your CS career you probably realize that what matters most is the quality of the work (software) you create and the impression you make as a serious student and software professional. Hence:

  • If your code produces a syntax error or fatal runtime error in Python3 you’ll get a zero on that part of the assignment, as well as any other parts that depend on that program. You will get no credit for a program that runs under Python2 but not Python3.
  • No late work will be accepted, or make-up exams given, without a note from the Dean’s office. You will get no credit for an assignment emailed to me, or submitted through anything other than github.

The best way to avoid the “But the program worked when I ran it!” problem is to submit your code early to github, clone it from github into an empty folder, and run it yourself.


Grading

  • Programming assignments, submitted through github. Though there is no official lab for this course, you will spend much of your time programming (50%).
  • Two one-hour in-class exams (30%).
  • Comprehensive final exam, during exam week (20%)

These percentages are flexible. If you have a bad day and don’t do well on an exam or assignment, I will try and weight your other work more heavily.

The grading scale will be 93-100 A; 90-92 A-; 87-89 B+; 83-86 B; 80-82 B-; 77-79 C+; 73-76 C; 70-72 C-; 67-69 D+; 63-66 D; 60-62 D-; below 60 F.


Honor System

The exams will be done without books or notes and without assistance from other people. You can discuss your programming assignments with other students in the class, but you should not share code. Talk to me before asking another student for help!


Final Exam

The final exam for this course will be given during the final exam week. You can take this exam during any of the regularly scheduled exam periods that week. You must supply an exam envelope to the instructor or the department administrative assistant no later than noon on the last day of class. You must specify a provisional day and time on the envelope, which you are free to change on the clipboard provided outside the door of Parmly 407 any time that week. Email or phone requests to reschedule will not be accepted.

The exam will be given in Parmly 405, and you should arrive promptly before the appointed time. If you are more than 15 minutes late, you will have to reschedule your exam. If you are more than 15 minutes late to the last exam period on Friday morning, you will receive a grade of 0 on your exam.

Students who have approved academic accommodations must make arrangements to use those accommodations directly with the instructor no later than the last day of class. Students approved for extra time will receive that time at the tail end of the morning exam period or before the beginning of the afternoon exam period (for example, ending at 1:30 PM for a morning exam or beginning at 12:30 PM for an afternoon exam). Students approved for a low-distraction testing location should reserve that space during the last week of classes (following instructions distributed by Director of Disability Resources Lauren Kozak.


Accommodations

Washington and Lee University makes reasonable academic accommodations for qualified students with disabilities. All undergraduate accommodations must be approved through the Title IX Coordinator and Director of Disability Resources, Elrod Commons 212, (540) 458-4055. Students requesting accommodations for this course should present an official accommodation letter within the first two weeks of the term and schedule a meeting outside of class time to discuss accommodations. It is the student’s responsibility to present this paperwork in a timely fashion and to follow up about accommodation arrangements. Accommodations for test-taking must be arranged with the professor at least a week before the date of the test or exam, including finals.


Programming assignments

Programming assignments should be submitted to the private repository I have set up for you on github, using a separate folder for each assignment.  Your repository name  is https://github/com/simondlevy/<username>, where <username> is your W&L network user name (for example, smithj22).  Each assignment is due 11:59 PM on the day posted in the schedule below.

The computers in the CS Department should all have NumPy and matplotlib installed, but I encourage you install them your laptop or home computer and work on these assignments anywhere you like. Experience has taught me that the students who do best in this course are the ones who start the assignments early and take advantage of office hours for help.


Tentative Schedule of Lectures and Exams

Monday

Wednesday

Friday

06 Sep
Week 0
Course Overview;
Philosophical FoundationsReading: Dragonfly Brains (2021)
13 Sep
Week 1
Reading: The Appeal of Parallel Distributed Processing (1986)Lecture Notes Dot Product Lecture Notescosine.py

eucdist.py

Dot Product Lecture Notes
20 Sep Week 2 Architecture #1: Kohonen’s Self-Organizing MapDue: Assignment #1 Lecture notesReading: Törmä 1994

Kohonen SOM, continued

Reading: Zhao, Li, Kohonen 2010Lecture notes
27 Sep
Week 3
Architecture #2: The Hopfield NetworkLecture notes

Reading: Hopfield 1982

Reading: Hopfield & Tank 1985 Student’s don’t know …Due: Assignment #2
04 Oct
Week 4
Architecture #3: Sparse Distributed MemoryLecture notes

Reading: Denning 1989

Lecture notesReading: Denning 1989 Pre-exam review
11 Oct
Week 5
In-class Exam #1 After-exam review Reading Day; no class
18 Oct
Week 6
Architecture #5: Latent Semantic AnalysisLecture Notes

Reading: Landauer & Dumais (1997)

Due: Assignment #3

LSA LSA
25  Oct
Week 7
Architecture #6: Tensor ProductsReading: Smolensky 1990

Lecture Notes

Tensor Products (continued)Reading: Gayler (2003) Tensor Products IIReading: Kanerva 2008
01 Nov
Week 8
Due: Assignment #4 Review for exam In-class Exam #2
08  Nov
Week 9
Exam #2 follow-upArchitecture #7: Biologically realistic neural networks Readings:Eliasmith et al. 2012

Stewart 2012

NEF: Lecture notes

Due: Assignment #5
15 Nov
Week 10
NEF NEF NEF
29  Nov
Week 11
SPA: Lecture notes Neuromorphic HardwareReading: The Physicality of Representation Neuromorphic HardwareDue: Assignment #6
05 Dec
Week 11
Group Exercise: Nengo Group Exercise:            Event Cameradvs-client.zip Review for final exam