Neural Networks FFR135

(7.5 credit units)
Bernhard Mehlig (lectures, examiner) Office: S3050
Marina Rafajlovic  (examples classes) Marina.Rafajlovic(at)physics.gu.se, office S3014
Johan Fries (examples classes) Johan.Fries(at)physics.gu.se, office S3047

Course representatives

N.N.

Schedule

The schedule in TimeEdit can be found here.
Consultation hours Marina Rafajlovic: Thursdays 15:15-16:15
Consultation hours Johan Fries: Tuesdays 15:15-16:15

Overview

The course introduces the use of neural network models in learning and optimisation.

Plan

Lecture 1      Introduction

Lecture 2      Pattern recognition: deterministic Hopfield model, Hebb's rule (LN pp. 1-17)
Lecture 3      One-step error probabilityl, storage capacity, energy function (LN pp. 18-27)
Lecture 4      Stochastic Hopfield model, mean-field theory for stochastic Hopfield model (LN pp. 32-47)
Lecture 5      Phase diagram for stochastic Hopfield model, error probability (LN pp. 48-60)

Lecture 6      Supervised learning: introduction to simple perceptrons (LN p. 79-85)
Lectures 7,8  Backpropagation (LN pp. 86-112)    
Lecture 9      Conjugate gradient methods (LN pp. 113-119)
Lecture 10    Performance of multilayer networks (LN pp. 124-141)

Lecture 11    Unsupervised learning: simple Hebbian learning (LN pp. 142-150)
Lecture 12    Oja's rule (LN pp. 151-157)
Lecture 13    Linear stability analysis (LN pp. 152,153)
Lecture 14    Simple competitive learning  (LN pp. 158-169)
Lecture 15    Topographic maps (LN pp. 170-174)
Lecture 16    Kohonen's algorithm (LN pp. 175-191)

Lecture 17   Radial-basis function networks (LN pp. 192-202)

Lecture 18   Recurrent networks (LN pp. 203-216)

Lecture 19   Optimisation, Monte-Carlo algorithms, simulated annealing (pp. 66-78)

Literature

B. Mehlig, Lecture notes for FFR 135

O. Mogren, slides

I. Goodfellow, Y. Bengio, A. Courville, Deep Learning MIT Press
Y. LeCunn, Y. Bengio & G. Hinton, Deep learning, Nature 521 (2015) 436–444

Further literature

S. Haykin, Neural Networks: a comprehensive foundation, 2nd ed., Prentice Hall, New Jersey (1999)

Examination

Credits for this course are obtained by solving the homework problems (solutions of examples and programming projects) and by a written examination. There are two sets of homework which are graded. Each set gives at most 6 points. The exam gives at most 12 points, resulting in a maximum of 24 points. The grades are determined as follows.

Chalmers: 3: 14-17p, 4: 17.5-21.5p, 5: 22-24p
GU: G: 14-19.5p, VG: 20-24p
ECTS: C: 14-17p, B: 17.5-21.5p, A: 22-24p

Rules for homework problems

Same rules as for written exams apply: it is not allowed to copy any material from anywhere unless reference is given.

Group work. You are encouraged to collaborate and you are allowed to hand in the solutions to the first problem set in groups of two.  You may also submit your solutions to the second problem set in groups of two, but not  with the same partner.

Format. The format of the solutions must be as follows. On each sheet there are six questions giving 1p each. Separately for each 1p-question you must submit at most one A4 page with 12pt single-spaced text, and with 2cm margins. LateX template. Each page may contain at most one Figure or one Table with the corresponding Figure or Table caption, in addition to the text discussing the results shown in the Figure/Table. It is not necessary to write a full page for each problem, but you must explain/describe what you have done and clearly state your answers/results to the questions and your conclusions. If appropriate you must discuss possible errors and inaccuracies in your results. If you are asked to plot results/make graphs, you do this in a Figure with axis labels and tic labels. All symbols and lines must be explained in the Figure or in a caption. The Figure may consist of separate panels. Refer to them as 'left panel', 'right panel', 'bottom panel', etc. (or alternatively label them 'a', 'b',...). Program code must be appended as text at the end of your report in the web submission, but not to paper submission.

URKUND. The examples sheets are processed by URKUND. Your solutions must be submitted before the deadline as PDF files electronically to This email address is being protected from spambots. You need JavaScript enabled to view it. , and as hardcopies to the letter box on floor 3 of Soliden/Physics. Information about the URKUND system can be found here. Further Instructions for the email to URKUND: Subject =  [FFR135] (Chalmers). Then attach the PDF file with filename firstname-lastname-ps1.pdf for the first problem sheet, or firstname1-lastname1_firstname2-lastname2-ps1.pdf  if you submit as a pair.

Deadlines are  sharp. Both electronic and paper versions must be handed in before the deadline.  At most a score of 4 can be obtained for a problem set submitted after the deadline. If you are on travel and absolutely cannot turn in a paper copy on time, you can email  your work to one of the teaching assistants, PDF only. It must however still be received before the deadline.

Sheet 1 [pdf,training_set,validation_set] Deadline: September 30 12:00 (lunch).
Sheet 2 [pdf,wine.data.txt,explanation.txt,task3.txt] Deadline: October 28 23:59.

Rules for written exam

Link to test exam. Marina's solutions: link1     
link2

The exam covers the material in the lecture notes as well as in the homework problems. No books, lecture notes, personal notes, or calculators allowed. The only allowed material is Mathematics Handbook for Science and Engineering, Lennart Råde and Bertil Westergren. Cremona. Any edition of this handbook is allowed.

Date for written exam: TBA
Time:  TBA
Place: TBA
Deadline for registration for exam: TBA
Duration of exam: TBA

If the date & time of the exam collides with another exam you must take, then you must follow the steps outlined here.

Summary of changes since course was given last

None.