filmov
tv
Coshnets: Improving Convolutional Neural Networks with Complex Numbers. ML techniques
Показать описание
To prepare for my upcoming look into natural gradients, I thought I'd go back to the idea that introduced me to orthognality, different decision boundaries and more. Let's talk about the amazing CoshNet and how it uses Complex Valued Functions to improve Convolutional Neural Networks
CNNs have been the go-to for Computer Vision forever. Their strength relies on the iterative feature extraction process. The lower layers of this process extracts low-resolution features (edges, color changes etc) but stacking enough layers gives you access to some very high-resolution feature maps.
However, this comes with a problem. Feature Extraction in CNNs are very expensive. Considering that the lower layers are just extracting simple features, it should be possible to pull out the information by using a fixed function. This is the idea behind the CoshNet, which uses a fixed complex function for lower-level feature extraction. CoshNet shows stronger generalization, better performance, lower costs, more stability, and strong adversarial robustness compared to traditional CNNs.
Paper Details
Name: CoShNet: A Hybrid Complex Valued Neural Network using Shearlets
In a hybrid neural network, the expensive convolutional layers are replaced by a non-trainable fixed transform with a great reduction in parameters. In previous works, good results were obtained by replacing the convolutions with wavelets. However, wavelet based hybrid network inherited wavelet's lack of vanishing moments along curves and its axis-bias. We propose to use Shearlets with its robust support for important image features like edges, ridges and blobs. The resulting network is called Complex Shearlets Network (CoShNet). It was tested on Fashion-MNIST against ResNet-50 and Resnet-18, obtaining 92.2% versus 90.7% and 91.8% respectively. The proposed network has 49.9k parameters versus ResNet-18 with 11.18m and use 52 times fewer FLOPs. Finally, we trained in under 20 epochs versus 200 epochs required by ResNet and do not need any hyperparameter tuning nor regularization.
Develop your skills-
Use the discount 20% off for 1 year-
Using this discount will drop the prices-
800 INR (10 USD) → 640 INR (8 USD) per Month
8000 INR (100 USD) → 6400INR (80 USD) per year (533 INR /month)
Reach out to me
Use the links below to check out my other content, learn more about tutoring, reach out to me about projects, or just to say hi.
Small Snippets about Tech, AI and Machine Learning over here
CNNs have been the go-to for Computer Vision forever. Their strength relies on the iterative feature extraction process. The lower layers of this process extracts low-resolution features (edges, color changes etc) but stacking enough layers gives you access to some very high-resolution feature maps.
However, this comes with a problem. Feature Extraction in CNNs are very expensive. Considering that the lower layers are just extracting simple features, it should be possible to pull out the information by using a fixed function. This is the idea behind the CoshNet, which uses a fixed complex function for lower-level feature extraction. CoshNet shows stronger generalization, better performance, lower costs, more stability, and strong adversarial robustness compared to traditional CNNs.
Paper Details
Name: CoShNet: A Hybrid Complex Valued Neural Network using Shearlets
In a hybrid neural network, the expensive convolutional layers are replaced by a non-trainable fixed transform with a great reduction in parameters. In previous works, good results were obtained by replacing the convolutions with wavelets. However, wavelet based hybrid network inherited wavelet's lack of vanishing moments along curves and its axis-bias. We propose to use Shearlets with its robust support for important image features like edges, ridges and blobs. The resulting network is called Complex Shearlets Network (CoShNet). It was tested on Fashion-MNIST against ResNet-50 and Resnet-18, obtaining 92.2% versus 90.7% and 91.8% respectively. The proposed network has 49.9k parameters versus ResNet-18 with 11.18m and use 52 times fewer FLOPs. Finally, we trained in under 20 epochs versus 200 epochs required by ResNet and do not need any hyperparameter tuning nor regularization.
Develop your skills-
Use the discount 20% off for 1 year-
Using this discount will drop the prices-
800 INR (10 USD) → 640 INR (8 USD) per Month
8000 INR (100 USD) → 6400INR (80 USD) per year (533 INR /month)
Reach out to me
Use the links below to check out my other content, learn more about tutoring, reach out to me about projects, or just to say hi.
Small Snippets about Tech, AI and Machine Learning over here
Комментарии