Convolution Neural Network study material

 

Convolution Neural Network https://www.analyticsvidhya.com/blog/2021/05/convolutional-neural-networks-cnn/

Convolution Operation

In mathematics, convolution is a mathematical operation on two functions that produces a third function that expresses how the shape of one is modified by the other. The term convolution refers to both the result function and to the process of computing it. 

Max-pooling operation, questions based on these operations.

Benefits of Convolution operation

Convolutions in Neural Networks apply filters to extract features from actual data. A filter could be related to anything, for pictures of humans, one filter could be associated with seeing noses, another with eyes, and so on. Each feature extracted from input data will be in form of activation maps

Training process of convolution Neural Network.

These are the steps used to training the CNN (Convolutional Neural Network).

1. Steps:
2. Step 1: Upload Dataset.
3. Step 2: The Input layer.
4. Step 3: Convolutional layer.
5. Step 4: Pooling layer.
6. Step 5: Convolutional layer and Pooling Layer.
7. Step 6: Dense layer.
8. Step 7: Logit Layer.

Convolution Neural Net Architecture

AlexNet

VGG https://towardsdatascience.com/vgg-neural-networks-the-next-step-after-alexnet-3f91fa9ffe2c

VGG Neural Networks. While previous derivatives of AlexNet focused on smaller window sizes and strides in the first convolutional layer, VGG addresses another very important aspect of CNNs: depth. Let’s go over the architecture of VGG:

• Input. VGG takes in a 224x224 pixel RGB image. For the ImageNet competition, the authors cropped out the center 224x224 patch in each image to keep the input image size consistent.
• Convolutional Layers. The convolutional layers in VGG use a very small receptive field (3x3, the smallest possible size that still captures left/right and up/down). There are also 1x1 convolution filters which act as a linear transformation of the input, which is followed by a ReLU unit. The convolution stride is fixed to 1 pixel so that the spatial resolution is preserved after convolution.
• Fully-Connected Layers. VGG has three fully-connected layers: the first two have 4096 channels each and the third has 1000 channels, 1 for each class.
• Hidden Layers. All of VGG’s hidden layers use ReLU (a huge innovation from AlexNet that cut training time). VGG does not generally use Local Response Normalization (LRN), as LRN increases memory consumption and training time with no particular increase in accuracy.

ResNet.
Autoencoders- link

Architecture of Autoencoders,

Training of Autoencoders  

Different type  of Auto encoders -https://iq.openÍgenus.org/types-of-autoencoder/

1. Under complete
2. Sparse
3. Denoising Autoencoders.

Principal Component Analysis -Í

Principal Component Analysis (PCA) is an unsupervised, non-parametric statistical technique primarily used for dimensionality reduction in machine learning.

Assumptions

PCA is based on the Pearson correlation coefficient framework and inherits similar assumptions.

1. Sample size: Minimum of 150 observations and ideally a 5:1 ratio of observation to features (Pallant, 2010)
2. Correlations: The feature set is correlated, so the reduced feature set effectively represents the original data space.
3. Linearity: All variables exhibit a constant multivariate normal relationship, and principal components are a linear combination of the original features.
4. Outliers: No significant outliers in the data as these can have a disproportionate influence on the results.

PCA and its comparison with autoencoders (Lecture 29 posted on November 24)Comparison

1. PCA is essentially a linear transformation but Auto-encoders are capable of modelling complex non linear functions.
2. PCA features are totally linearly uncorrelated with each other since features are projections onto the orthogonal basis. But autoencoded features might have correlations since they are just trained for accurate reconstruction.
3. PCA is faster and computationally cheaper than autoencoders.
4. A single layered autoencoder with a linear activation function is very similar to PCA.
5. Autoencoder is prone to overfitting due to high number of parameters. (though regularization and careful design can avoid this)

Optimization in Deep Learning https://medium.com/@minions.k/optimization-techniques-popularly-used-in-deep-learning-3c219ec8e0cc