摘要 :
In this paper, we propose two novel Adaptive Neural Network Approaches (ANNAs), which are intended to automatically learn the optimal network depth. In particular, the proposed class-independent and class-dependent ANNAs address t...
展开
In this paper, we propose two novel Adaptive Neural Network Approaches (ANNAs), which are intended to automatically learn the optimal network depth. In particular, the proposed class-independent and class-dependent ANNAs address two main challenges faced by typical deep learning paradigms. Namely, they overcome the problems of setting the optimal network depth and improving the model interpretability. Specifically, ANNA approaches simultaneously train the network model, learn the network depth in an unsupervised manner, and assign fuzzy relevance weights to each network layer to better decipher the model behavior. In addition, two novel cost functions were designed in order to optimize the layer fuzzy relevance weights along with the model hyper-parameters. The proposed ANNA approaches were assessed using standard benchmarking datasets and performance measures. The experiments proved their effectiveness compared to typical deep learning approaches, which rely on empirical tuning and scaling of the network depth. Moreover, the experimental findings demonstrated the ability of the proposed class-independent and class-dependent ANNAs to decrease the network complexity and build lightweight models for less overfitting risk and better generalization.
收起
摘要 :
We describe a neural network architecture for learning which (if any) further questions are necessary to make a correct diagnosis, given a set of known preliminary inputs. Question evaluation subnetworks learn when further questio...
展开
We describe a neural network architecture for learning which (if any) further questions are necessary to make a correct diagnosis, given a set of known preliminary inputs. Question evaluation subnetworks learn when further questions are necessary, through positive feedback from misdiagnoses due to lack of necessary information, and negative feedback representing the cost of that information. These activate cutoff units which allow the network to simulate the effects of asking/not asking further questions. The network learns to ask only those further questions necessary to prevent error, and may also be used reach final conclusions once those questions have been answered.
收起
摘要 :
Artificial Neural Networks, notably Convolutional Neural Networks (CNN) are widely used for classification purposes in different fields such as image classification, text classification and others. It is not uncommon therefore tha...
展开
Artificial Neural Networks, notably Convolutional Neural Networks (CNN) are widely used for classification purposes in different fields such as image classification, text classification and others. It is not uncommon therefore that these models are used in critical systems (e.g. self-driving cars), where robustness is a very important attribute. All Convolutional Neural Networks used for classification, classify based on the extracted features found in the input sample. In this paper, we present a novel approach of doing the opposite - classification based on features not present in the input sample. Obtained results show not only that this way of learning is indeed possible but also that the trained models become more robust in certain scenarios. The presented approach can be applied to any existing Convolutional Neural Network model and does not require any additional training data.
收起
摘要 :
Fuzzy neural networks (FNNs) and rough neural networks (RNNs) both have been hot research topics in the artificial intelligence in recent years. The former imitates the human brain in dealing with problems, the other takes advanta...
展开
Fuzzy neural networks (FNNs) and rough neural networks (RNNs) both have been hot research topics in the artificial intelligence in recent years. The former imitates the human brain in dealing with problems, the other takes advantage of rough set theory to process questions uncertainly. The aim of FNNs and RNNs is to process the massive volume of uncertain information, which is widespread applied in our life. This article summarizes the recent research development of FNNs and RNNs (together called granular neural networks). First the fuzzy neuron and rough neuron is introduced; next FNNs are analysed in two categories: normal FNNs and fuzzy logic neural networks; then the RNNs are analysed in the following four aspects: neural networks based on using rough sets in preprocessing information, neural networks based on rough logic, neural networks based on rough neuron and neural networks based on rough-granular; then we give a flow chart of the RNNs processing questions and an application of classical neural networks based on rough sets; next this is compared with FNNs and RNNs and the way to integrate is described; finally some advice is given on development of FNNs and RNNs in future.
收起
摘要 :
Artificial neural networks (ANNs) have been widely used over the last three decades. During this period, many hardware and software solutions have been developed and today a new user entering the field can make a fast trial to thi...
展开
Artificial neural networks (ANNs) have been widely used over the last three decades. During this period, many hardware and software solutions have been developed and today a new user entering the field can make a fast trial to this artificial intelligence solution with commercial software and hardware, instead of developing a solution from scratch thus saving a lot of time. This work aims at helping new and experienced users even further by sharing the ANNs experience in software and hardware collected. This was achieved through a survey questionnaire about present and past used solutions of software and hardware, as well as future prospects for the development of application areas. To further enlighten the reader, a logistic regression (LR) statistical analysis is performed on the obtained results to extract additional details about the answers obtained from the ANN community. The LR statistical analysis verifies whether the researchers with more than 25 years of experience in ANNs use self-written code when compared to those with less years of experience in the area. The LR statistical analysis also verifies whether researchers with less than 25 years of experience in ANNs use some platform to develop their hardware when compared to those who have more years of experience.
收起
摘要 :
To date, the preponderance of techniques for eliciting the knowledge embedded in trained artificial neural networks (ANN's) has focused primarily on extracting rule-based explanations from feedforward ANN's. The ADT taxonomy for c...
展开
To date, the preponderance of techniques for eliciting the knowledge embedded in trained artificial neural networks (ANN's) has focused primarily on extracting rule-based explanations from feedforward ANN's. The ADT taxonomy for categorizing such techniques was proposed in 1995 to provide a basis for the systematic comparison of the different approaches. This paper shows that not only is this taxonomy applicable to a cross section of current techniques for extracting rules from trained feedforward ANN's but also how the taxonomy can be adapted and extended to embrace a broader range of ANN types (e,g., recurrent neural networks) and explanation structures. In addition we identify some of the key research questions in extracting the knowledge embedded within ANN's including the need for the formulation of a consistent theoretical basis for what has been, until recently, a disparate collection of empirical results.
收起
摘要 :
? 2021 Elsevier LtdDeep neural networks unlocked a vast range of new applications by solving tasks of which many were previously deemed as reserved to higher human intelligence. One of the developments enabling this success was a ...
展开
? 2021 Elsevier LtdDeep neural networks unlocked a vast range of new applications by solving tasks of which many were previously deemed as reserved to higher human intelligence. One of the developments enabling this success was a boost in computing power provided by special purpose hardware, such as graphic or tensor processing units. However, these do not leverage fundamental features of neural networks like parallelism and analog state variables. Instead, they emulate neural networks relying on binary computing, which results in unsustainable energy consumption and comparatively low speed. Fully parallel and analogue hardware promises to overcome these challenges, yet the impact of analogue neuron noise and its propagation, i.e. accumulation, threatens rendering such approaches inept. Here, we determine for the first time the propagation of noise in deep neural networks comprising noisy nonlinear neurons in trained fully connected layers. We study additive and multiplicative as well as correlated and uncorrelated noise, and develop analytical methods that predict the noise level in any layer of symmetric deep neural networks or deep neural networks trained with back propagation. We find that noise accumulation is generally bound, and adding additional network layers does not worsen the signal to noise ratio beyond a limit. Most importantly, noise accumulation can be suppressed entirely when neuron activation functions have a slope smaller than unity. We therefore developed the framework for noise in fully connected deep neural networks implemented in analog systems, and identify criteria allowing engineers to design noise-resilient novel neural network hardware.
收起
摘要 :
Deep Learning is a field included in to Artificial Intelligence. It allows computational models to learn multiple levels of abstraction with multiple processing layers. This Artificial Neural Networks gives state-of-art performanc...
展开
Deep Learning is a field included in to Artificial Intelligence. It allows computational models to learn multiple levels of abstraction with multiple processing layers. This Artificial Neural Networks gives state-of-art performance in various fields like Computer Vision, Speech recognition and different domain like bioinformatics. There are mainly three architectures of Deep Learning Convolution Neural Network, Deep Neural Network and Recurrent Neural Network which provides the higher level of representation of data at each next layer. Deep Learning is required to classify high dimensional data like images, audio, video and biological data.
收起
摘要 :
This paper focuses on the powerful concept of modularity. It is descried how this concept is deployed in natural neural networks on an architectural as well as on a functional level. Furthermore, different approaches for modular n...
展开
This paper focuses on the powerful concept of modularity. It is descried how this concept is deployed in natural neural networks on an architectural as well as on a functional level. Furthermore, different approaches for modular neural networks are discussed. By means of these methods, a two-layer modular neural system is introduced. The basic building blocks of the architecture are multilayer perceptions (MP) with the backpropagation (BP) algorithm. This modular network is designed to combine two different approaches of generalization known from connectionist and logical neural networks; this enhances the generalization abilities of the network. Experiments described in this paper show that the architecture is especially useful in solving problems with a large number of input attributes.
收起
摘要 :
Deep neural networks have been applied in many applications exhibiting extraordinary abilities in the field of computer vision. However, complex network architectures challenge efficient real-time deployment and require significan...
展开
Deep neural networks have been applied in many applications exhibiting extraordinary abilities in the field of computer vision. However, complex network architectures challenge efficient real-time deployment and require significant computation resources and energy costs. These challenges can be overcome through optimizations such as network compression. Network compression can often be realized with little loss of accuracy. In some cases accuracy may even improve. This paper provides a survey on two types of network compression: pruning and quantization. Pruning can be categorized as static if it is performed offline or dynamic if it is performed at run-time. We compare pruning techniques and describe criteria used to remove redundant computations. We discuss trade-offs in element-wise, channel-wise, shape-wise, filter-wise, layer-wise and even network-wise pruning. Quantization reduces computations by reducing the precision of the datatype. Weights, biases, and activations may be quantized typically to 8-bit integers although lower bit width implementations are also discussed including binary neural networks. Both pruning and quantization can be used independently or combined. We compare current techniques, analyze their strengths and weaknesses, present compressed network accuracy results on a number of frameworks, and provide practical guidance for compressing networks. (c) 2021 Elsevier B.V. All rights reserved.
收起
原文获取