Homework 2 - More ANN Fundamentals
Due Monday, September 12, 2016
NOTE: This assignment, like others in this class, is due at the
beginning of the class period. This means that if you are
even a minute late, you lose 20%. If you are worried about
potentially being late, turn in your homework ahead of time.
Do this by submitting them to me during office hours or by sliding it
under my office door. Do not send assignments to me through email or
leave them in my departmental mail box.
1. Motivation
Artificial neurons (ANs) can be combined in many ways to compute more
complex functions than could be computed by a single AN. The most
fundamental way is by combining ANs into a layered, feedforward neural
network (FFNN). Likewise, FFNNs can learn in many ways but the most
fundamental way is supervised learning. Moreover, FFNNs may be used for
many tasks but the two most fundamental are classification and function
approximation, of which classification is the easier to visualize. These
ANN fundamentals — FFNNs, supervised learning, and classification
— are the topics of this homework.
2. Goal
The goal of this assignment is to give you experience with basic FFNNs,
supervised learning using backpropagation of error, and classification.
3. Assignment
Complete the following exercises:
Part 1 — FFNN Representation
- Consider a two-layer FFNN — that is, one with two layers of
computational elements (ANs) — used for classification in a 2D
space with augmented vectors. The ANs in this FFNN are all SUs and their
activation functions are identical to fAN as given in
Homework 1, 1.1. There are three ANs in the hidden layer and one in the
output layer. Given the weights v1,1=−1.0,
v2,1=1.0, v3,1=1.0,
v1,2=0.6, v2,2=−0.9,
v3,2=0.6, v1,3=−0.2,
v2,3=0.6, v3,3=−0.3,
w1=−1.0, w2=0.0,
w3=−0.8, and w4=0.0,
draw this FFNN.
- Draw the decision region encoded by this FFNN. Be sure to
indicate the γ1 side of the region. Be sure to indicate which portion of the decision region is due to each hidden layer AN.
- Add the following points on the graph you just drew and
label the class of each according to the AN.
- (−2, −3)
- (3, −3)
- (0, 0)
- (0, −2)
- (−4, −3)
- (1, 3)
- (−3, 4)
- Explain the significance of the value of
w4.
- Explain the significance of the value of
w2.
- Explain the significance of the relative values of
w1 and w3.
- Explain how the decision region for this FFNN would change if
the value of w2 were changed to 0.3 rather than 0.0
and explain which points, if any, from those above would be
classified differently and which would be classified the same. Be sure
to discuss the relative values of w1,
w2, and w3
- Explain how the decision region for this FFNN would change if
γ2 were changed to −1 rather than 0 and
explain which points, if any, from those above would be classified
differently and which would be classified the same. (For this
hypothetical, use a value of 0.3 for w2.)
- Explain how the decision region for this FFNN would change if
fAN for the hidden layer ANs were changed to be a
linear activation function, in particular, if
fAN(net)=net for the hidden layer ANs.
(For the output AN, fAN remains unchanged in this
hypothetical.) Explain which points, if any, from those above
would be classified differently and which would be classified the same.
(For this hypothetical, use a value of 0.0 for w2 and
a value of 0 for γ2.)
Part 2 — FFNN Learning
- Consider the FFNN given above in 1.1 but with sigmoidal activation
functions with λ=1 for each AN and η = 0.5. Here the
target value for γ1 is 0.9 and for γ2
is 0.1. Explain how its weights would be updated, using the
backpropagation algorithm we covered in class, if presented with the data
item (−1.0, −1.0) γ1. Show your work.
Keep track of four significant digits.
- Calculate the output value of the FFNN above if, after
learning on (−1.0, −1.0) γ1, you were to
present this data item to the FFNN again. Show your work. Keep
track of four significant digits.
- Explain whether the error value for the input (−1.0,
−1.0) γ1 increased or decreased due to
learning.
4. What to Turn In
Turn in a neatly handwritten copy of your answers to the exercises for
this assignment. The diagrams should be drawn on engineering or graph
paper. You may also turn in a scanned electronic copy of this assignment
as a backup in case your paper copy is misplaced.