Designing an effective channel attention mechanism implores one to find a lossy-compression method allowing for optimal feature representation. Despite recent progress in the area, it remains an open problem. FcaNet, the current state-of-the-art channel attention mechanism, attempted to find such an information-rich compression using Discrete Cosine Transforms (DCTs). One drawback of FcaNet is that there is no natural choice of the DCT frequencies. To circumvent this issue, FcaNet experimented on ImageNet to find optimal frequencies. We hypothesize that the choice of frequency plays only a supporting role and the primary driving force for the effectiveness of their attention filters is the orthogonality of the DCT kernels. To test this hypothesis, we construct an attention mechanism using randomly initialized orthogonal filters. Integrating this mechanism into ResNet, we create OrthoNet. We compare OrthoNet to FcaNet (and other attention mechanisms) on Birds, MS-COCO, and Places356 and show superior performance. On the ImageNet dataset, our method competes with or surpasses the current state-of-the-art. Our results imply that an optimal choice of filter is elusive and generalization can be achieved with a sufficiently large number of orthogonal filters. We further investigate other general principles for implementing channel attention, such as its position in the network and channel groupings. Our code is publicly available at (https://github.com/hady1011/OrthoNets)

Channel Attention reigns supreme as an effective technique in the field of computer vision. However, the proposed channel attention by SENet suffers from information loss in feature learning caused by the use of Global Average Pooling (GAP) to represent channels as scalars. Thus, designing effective channel attention mechanisms requires finding a s olution t o enhance features preservation in modeling channel inter-dependencies. In this work, we utilize Wavelet transform compression as a solution to the channel representation problem. We first t est wavelet transform as a standalone channel compression method. We prove that global average pooling is equivalent to the recursive approximate Haar wavelet transform. With this proof, we generalize channel attention using Wavelet compression and name it WaveNet. Implementation of our method can be embedded within existing channel attention methods with a couple of lines of code. We test our proposed method using ImageNet dataset for image classification t ask. O ur m ethod o utperforms t he baseline SENet-34, and SOTA FcaNet-34. Our code implementation is publicly available at https://github.com/hady1011/WaveNet-C.

Generative Adversarial Networks GANs have become widely used in Single-view classification tasks. Nowadays, most of the data have multiple views and each view emphasizes a unique feature set of the data. In this paper, we investigate the application of GANs on Multi-view data for the task of clustering and few-shot learning. We propose mvSGAN, a deep learning approach to GAN multi-view clustering, where generator and classifier networks are in a competitive min-max game. A multi-view learning algorithm is implemented with a mini-batch which can handle large data sets. We test the accuracy of our method in clustering real-world data sets. The experimental results show that our method outperforms state-of-the-art research.