Advanced deep learning and neural network

1




Important Questions and Answers
CP25C11-Advanced Deep Learning and Neural Networks
Syllabus
1. Neural Networks: Introduction – Perceptron – Multilayer Feed forward networks – Back
propagation – Activation functions – Loss Function – Regularization: Data Augmentation -
Noise Robustness – Early Stopping – Bagging – Dropout – batch normalization.
Activity: Understanding the working principle of Neural Networks.
2. CNN and RNN: Convolutional Neural Networks – Convolution Operation – Architecture
Overview – Input layers – Convolutional layers – pooling layers – fully connected layers
Recurrent Neural networks – LSTM – Bidirectional RNNs – RNN Language model – Word
Level RNN - Deep Recurrent Networks – Recursive Neural Networks.
Activity: Reproduction of research paper related to CNN and RNN.
3. Optimization and Deep Networks: Optimization in deep learning – Non-convex
optimization for deep networks- Stochastic Optimization- Generalization in neural networks
- Spatial Transformer Networks Tuning Deep Networks – Basic Concepts – Matching input
data and network architecture – relating model goal and output layers – Working layer count,
parameter count and Memory – weight initialization strategies – using activation functions –
applying loss function – Dealing with overfitting.
Activity: Simple application using deep neural network with three layers.
4. Encoder / decoder Models and Regressive Models: Encoder Decoder Models - Attention
Mechanism - Attention over images – Hierarchical Attention – Variational auto encoders –
Autoregressive models – NADE – MADE - PixelRNN – Generative Adversarial Networks
(GANs) – Graph Convolution Network – Deep Belief Network.
Activity: Design of any Regressive, Generative and Graph Models.
V.




5. Advanced Applications of Neural Networks: Object Detection: RCNN, Faster RCNN, and
U
m




Yolo – MobileNet – DarkNet – Object Tracking - Audio WaveNet - Natural Language
ap




Processing Word2Vec Joint Detection - Face Recognition - Scene captioning Language
at




Transformer Models.
hy




Activity: Design of face recognition model.
,Pr




6. Advanced applications of Deep learning: Deep learning for Computer Vision – Deep
of
./
EC
N
ot
es

, 2


learning for Time Series – Deep learning for text – Generative Deep learning.
Suggested Activity: medical image segmentation and time series data analysis.
S.No. Description
Unit-I
Neural Networks
Basic Concepts
S.No. Topic Basic Concepts
Computational models inspired by the human brain. They consist of
1 Neural Networks
interconnected neurons that learn patterns from data.
Simplest neural network with input values, weights, bias and
2 Perceptron
activation function. Used for binary classification.
Multilayer Feed Consists of input, hidden and output layers. Information flows only in
3
Forward Network the forward direction.
Learning algorithm that calculates output error and updates weights
4 Back Propagation
using gradient descent.
Introduce non-linearity into neural networks. Examples: Sigmoid,
5 Activation Functions
Tanh, ReLU and Softmax.
Measures the difference between predicted output and actual output.
6 Loss Function
Examples: Mean Squared Error and Cross-Entropy Loss.
Techniques used to reduce overfitting and improve the generalization
7 Regularization
ability of neural networks.
Increases training data by rotation, flipping, cropping and adding
8 Data Augmentation
variations to existing data.
Improves network performance when input data contains noise or
9 Noise Robustness
unwanted disturbances.
Stops training when validation error starts increasing to prevent
10 Early Stopping
overfitting.
Trains multiple models using different bootstrap samples and
11 Bagging
combines their predictions to improve accuracy.
Randomly disables some neurons during training to prevent over-
12 Dropout
dependence on particular neurons.
Normalizes layer inputs in mini-batches to improve training speed and
V.




13 Batch Normalization
U




stability.
m




Important Equations (Plain Text – Copy to Word)
ap
at
hy




Equation
,




Concept
Pr




Perceptron y = f(Σ(wi × xi) + b)
of
./
EC
N
ot
es

, 3


Weighted Sum z = Σ(wi × xi) + b
Sigmoid f(x) = 1 / (1 + e^(-x))
Tanh f(x) = (e^x - e^(-x)) / (e^x + e^(-x))
ReLU f(x) = max(0, x)
Softmax P(yi) = e^(zi) / Σe^(zj)
Mean Squared Error MSE = (1/N) × Σ(yi - ŷi)²
Gradient Descent w(new) = w(old) - η × (∂L/∂w)
Back Propagation δ = ∂L/∂z
L2 Regularization L(total) = L + λΣ(wi²)
Dropout h' = h × m / (1 - p)
Batch Normalization x_norm = (x - μ) / √(σ² + ε)
Important Points for Examination

1. Perceptron: Basic building block of neural networks.
2. Back Propagation: Used to minimize prediction error by updating weights.
3. Activation Function: Helps the network learn complex non-linear relationships.
4. Loss Function: Measures prediction error.
5. Regularization: Prevents overfitting and improves generalization.
6. Dropout and Batch Normalization: Improve network training and performance.

Exam tip: Give special importance to Perceptron, Feed Forward Networks, Back Propagation,
Activation Functions, Loss Functions and Regularization techniques.
Multiple Choice QA
1. What is the basic processing unit of an artificial neural network?

A) Decision Tree
B) Perceptron
C) Database
D) Compiler

Answer: B) Perceptron
V.
U




Reason: A perceptron is the fundamental building block of a neural network. It receives inputs,
m




multiplies them by weights, adds a bias and produces an output using an activation function.
ap
at
hy




Equation (plain text):
,Pr
of
./
EC
N
ot
es

, 4



y = f(w1x1 + w2x2 + ... + wnxn + b)

Where:

 x = Input values
 w = Weights
 b = Bias
 f = Activation function

2. Which algorithm is commonly used to train multilayer feed forward neural networks?

A) K-Means Algorithm
B) Decision Tree Algorithm
C) Back Propagation Algorithm
D) Apriori Algorithm

Answer: C) Back Propagation Algorithm

Reason: Back propagation calculates the error between predicted and actual outputs and propagates
the error backward to update weights and biases.

Equation (plain text):

Weight Update: w(new) = w(old) - η × (∂L/∂w)

Where:

 η = Learning rate
 L = Loss function
 ∂L/∂w = Gradient of loss with respect to weight

3. Which activation function produces an output between 0 and 1?

A) ReLU
V.




B) Sigmoid
U
m




C) Tanh
ap




D) Linear
at
hy




Answer: B) Sigmoid
,Pr
of




Reason: The sigmoid activation function converts input values into a range between 0 and 1. It is
./
EC
N
ot
es