## Neural Networks FFR135

- Details
- Last Updated on Monday, 19 December 2016 22:11

(7.5 credit units)

Bernhard Mehlig (lectures, examiner) Office: S3050

Marina Rafajlovic (examples classes) Marina.Rafajlovic(at)physics.gu.se, office S3014

Johan Fries (examples classes) Johan.Fries(at)physics.gu.se, office S3047**New:** Re-exam neural networks.

Date: December 22, 2016

Time: 14:00-18:00

Place: Samhällsbyggnaden

For questions please contact the exam administration at tentamen.stodet(at)chalmers.se.

** New: **Marina's solution to test exam (below), corrected 20 October 12:22

** New: **Lecture on deep learning by Olof Mogren on October 6.

**New**: Check exam date/time/place. Colliding exam? See below, bottom of this page. Don't forget to register for exam.

### Course representatives

Alma Ottedag, alma.ottedag(at)gmail.com

Anton Älgmyr, algmyr(at)student.chalmers.se

Christoffer Dahlén, christoffer.dahlen(at)gmail.com

Eleftherios Filippakis, elefil(at)student.chalmers.se

Contact them if you have any comments or suggestions about the course. After the course is finished everyone must fill out the course evaluation form.

### Schedule

The schedule in TimeEdit can be found here.

Consultation hours Marina Rafajlovic: Thursdays 15:15-16:15

Consultation hours Johan Fries: Tuesdays 15:15-16:15

### Overview

The course introduces the use of neural network models in learning and optimisation.

### Plan

Lecture 1 **Introduction**

Lecture 2 **Pattern recognition**: deterministic Hopfield model, Hebb's rule (LN pp. 1-17)

Lecture 3 One-step error probabilityl, storage capacity, energy function (LN pp. 18-27)

Lecture 4 Stochastic Hopfield model, mean-field theory for stochastic Hopfield model (LN pp. 32-47)

Lecture 5 Phase diagram for stochastic Hopfield model, error probability (LN pp. 48-60)

Lecture 6 **Supervised learning**: introduction to simple perceptrons (LN p. 79-85)

Lectures 7,8 Backpropagation (LN pp. 86-112)

Lecture 9 Conjugate gradient methods (LN pp. 113-119)

Lecture 10 Performance of multilayer networks (LN pp. 124-141)

Lecture 11 **Unsupervised learning**: simple Hebbian learning (LN pp. 142-150)

Lecture 12 Oja's rule (LN pp. 151-157)

Lecture 13 Linear stability analysis (LN pp. 152,153)

Lecture 14 Simple competitive learning (LN pp. 158-169)

Lecture 15 Topographic maps (LN pp. 170-174)

Lecture 16 Kohonen's algorithm (LN pp. 175-191)

Lecture 17 ** Radial-basis function networks **(LN pp. 192-202)

Lecture 18

**(LN pp. 203-216)**

**Recurrent networks**Lecture 19

**Optimisation**, Monte-Carlo algorithms, simulated annealing (pp. 66-78)

### Literature

B. Mehlig, Lecture notes for FFR 135

O. Mogren, slides

I. Goodfellow, Y. Bengio, A. Courville, Deep Learning MIT Press

Y. LeCunn, Y. Bengio & G. Hinton, Deep learning, Nature **521** (2015) 436–444

### Further literature

S. Haykin, Neural Networks: a comprehensive foundation, 2nd ed., Prentice Hall, New Jersey (1999)

### Examination

Credits for this course are obtained by solving the homework problems (solutions of examples and programming projects) and by a written examination. There are two sets of homework which are graded. Each set gives at most 6 points. The exam gives at most 12 points, resulting in a maximum of 24 points. The grades are determined as follows.

Chalmers: 3: 14-17p, 4: 17.5-21.5p, 5: 22-24p

GU: G: 14-19.5p, VG: 20-24p

ECTS: C: 14-17p, B: 17.5-21.5p, A: 22-24p

### Rules for homework problems

Same rules as for written exams apply: it is not allowed to copy any material from anywhere unless reference is given.

*Group work.* You are encouraged to collaborate and you are allowed to hand in the solutions to the first problem set in groups of two. You may also submit your solutions to the second problem set in groups of two, but not with the same partner.

*Format.* The format of the solutions must be as follows. On each sheet there are six questions giving **1**p each. Separately for each **1**p-question you must submit at most one A4 page with 12pt single-spaced text, and with 2cm margins. LateX template. Each page may contain at most one Figure or one Table with the corresponding Figure or Table caption, in addition to the text discussing the results shown in the Figure/Table. It is not necessary to write a full page for each problem, but you must explain/describe what you have done and clearly state your answers/results to the questions and your conclusions. If appropriate you must discuss possible errors and inaccuracies in your results. If you are asked to plot results/make graphs, you do this in a Figure with axis labels and tic labels. All symbols and lines must be explained in the Figure or in a caption. The Figure may consist of separate panels. Refer to them as 'left panel', 'right panel', 'bottom panel', etc. (or alternatively label them '**a**', '**b**',...). Program code must be appended as text at the end of your report in the web submission, but not to paper submission.

*URKUND.* The examples sheets are processed by URKUND. Your solutions must be submitted before the deadline as PDF files electronically to
This email address is being protected from spambots. You need JavaScript enabled to view it.
, and as hardcopies to the letter box on floor 3 of Soliden/Physics. Information about the URKUND system can be found here. Further Instructions for the email to URKUND: Subject* = [FFR135] (Chalmers). T*hen attach the PDF file with filename **firstname-lastname-ps1.pdf ** for the first problem sheet, or ** firstname1-lastname1_firstname2-lastname2-ps1.pdf **if you submit as a pair.

*Deadlines *are sharp. Both electronic and paper versions must be handed in before the deadline. At most a score of 4 can be obtained for a problem set submitted after the deadline. If you are on travel and absolutely cannot turn in a paper copy on time, you can email your work to one of the teaching assistants, PDF only. It must however still be received before the deadline.

Sheet 1 [pdf,training_set,validation_set] Deadline: September 30 12:00 (lunch).

Sheet 2 [pdf,wine.data.txt,explanation.txt,task3.txt] Deadline: October 28 23:59.

### Rules for written exam

Link to test exam. Marina's solutions: link1

link2

The exam covers the material in the lecture notes as well as in the homework problems. No books, lecture notes, personal notes, or calculators allowed. The only allowed material is *Mathematics Handbook for Science and Engineering, *Lennart Råde and Bertil Westergren. Cremona. Any edition of this handbook is allowed.

Date for written exam: Oct. 22 (2016)

Time: 14:00

Place: Lindholmen-salar.

Deadline for registration for exam: Oct. 6 (2016)

Duration of exam: 4h

If the date & time of the exam collides with another exam you must take, then you must follow the steps outlined here.

### Summary of changes since course was given last

Examination form changed.