Selected from Medium
Author:Shagun Maheshwari
Translated by Machine Heart
Contributors: Chen Yunzhu, Lu
This article introduces a new model called XNN (Explainable Neural Network) designed to “open” and explain neural networks. The network consists of three important components: the projection layer, subnetworks, and the combination layer. XNN can also serve as an alternative to complex models, allowing for direct interpretation of how XNN uses input features to make predictions, whether used as a primary model or as a substitute for more complex models.
One of the biggest obstacles to applying artificial neural networks to specific systems is their “black box” nature. XNN (Explainable Neural Network) is a new model designed to “open and explain” the black box of neural networks.
Using artificial neural networks and machine learning algorithms to train machines to learn information like the human brain has become increasingly popular. This allows machines to accurately output predictions for given tasks.
ANN trained on a given task can classify cats and dogs.
As another example, suppose you want to train a machine to determine which bank customers are most likely to leave the bank based on customer credentials.
The neural network will be trained on a large dataset containing each customer’s credentials (e.g., credit scores). It identifies which features in the data have the greatest impact on a customer’s decision to stay or leave by variable selection. The neural network will learn these features and, using them, accurately predict which customers are most likely to leave the bank based on their credentials (features). (See: https://www.linkedin.com/pulse/creating-deep-neural-net-accurately-predict-churn-rate-maheshwari/)
These models are very effective on large datasets, as manual variable selection and feature engineering are challenging in large datasets. Additionally, they offer better and more accurate predictive performance compared to traditional statistical methods.
Black Box
However, one issue with ANNs is that they are black boxes. This means that data analysts or modelers find it difficult to explain the relationship between input features and their responses (outputs). The more complex the neural network, the more neurons and layers are added, making it increasingly difficult to interpret and determine which function within the ANN leads to the output.
Applications
Understanding the black box and having the ability to explain model behavior is crucial, as the use of machine learning models and ANNs needs to be incorporated into fields such as healthcare and finance. It has been shown that machine learning models have the potential to revolutionize these industries, improving the efficiency of certain tasks.
Interestingly, researchers have actually created a machine learning algorithm that identifies breast cancer patterns in images of breast tissue better than human pathologists! The machine learning model can identify breast cancer patterns faster, achieving an accuracy of 89%, compared to the average accuracy of trained human pathologists at 73%! This is just one example of how ANN and machine learning models can provide tools that are more efficient and accurate than traditional human methods.
However, despite the proven accuracy and efficiency of these models, there is still hesitation about incorporating them into fields like healthcare and banking due to their black box nature. The ability to explain models and their behavior is critical in these industries as they deal with high-risk issues and must be under strict regulation.
Explainable Neural Networks
Explainable Neural Networks (XNN) represent a new advancement in machine learning models, aiming to provide interpretable insights into the model, eliminating its black box nature. This means it can provide straightforward explanations regarding the features and the nonlinear transformations learned by the network during the output (prediction) process. Through this model, researchers can clearly explain the relationship between input features and the outputs of complex neural networks, as the structure of the XNN includes mechanisms to interpret this relationship and aids in visualizing the functions learned by the network.
XNN is based on the concept of additive index models, as follows:
f (x) = g1 β1T x+ g2 β2T x+ · · · + gK βKT x
The function on the left can be represented as the sum of K smooth functions gi(·). These smooth functions (i.e., ridge functions) are used for the linear combinations of input features trained in the network (βiT x). This allows the additive index model to provide a flexible framework for approximating any complex function within the network through ridge functions, thereby providing explanations for the features and the nonlinear transformations learned by the network.
Explainable Neural Network Architecture
The explainable neural network provides an alternative formulation of the additive index model as a structured neural network. XNN has built-in explanatory mechanisms that help to explain and understand the internal processes of the model and the functions it learns.
The alternative formulation is as follows:
f (x) = μ + γ1 h1 β1T x+ γ2 h2 β1T x+ · · · + γK hK βKT x
The displacement parameter μ and scale parameter γk are used for model fitting: by regularization, the appropriate number of ridge functions is selected.
The three important components of the XNN structure include:
i) Projection layer (first hidden layer);
ii) Subnetwork (middle part of the diagram below);
iii) Combination layer (last hidden layer).
XNN Structure
Projection Layer
The input layer contains all information fed into the neural network. The input layer is fully connected to the projection layer, where features (information) are passed. The projection layer consists of K nodes (each corresponding to a ridge function). The weights of node i in the first hidden layer correspond to the coefficients of the respective ridge function inputs (βi). Ridge functions help approximate complex functions within the input features. The projection layer uses a linear activation function to ensure that each node in this layer learns a linear combination of the input features. The outputs of each node in the projection layer, where ridge functions are applied, can be used as inputs for a subnetwork.
Subnetwork
The subnetwork is primarily used to learn the ridge functions applied to the input features. Ridge functions are very important for the subnetwork as they approximate the complex functions of the projection layer. This makes it easier for the subnetwork to learn and provides interpretability of the learned ridge functions, enabling data analysts to understand the workings of the subnetwork and the process from input to output. The subnetwork only needs to have enough structure for each subnetwork to learn a large number of univariate functions. In simulations, researchers found that using a subnetwork consisting of two hidden layers with nonlinear activation functions was sufficient to learn sufficiently flexible ridge functions when fitting the model.
Combination Layer
The combination layer is the last hidden layer of the XNN, consisting of a single node. The inputs to the node include all outputs of the ridge functions and the weights learned and added in the subnetwork. A linear activation function is used in this layer, so the output of the entire network is a linear combination of the weighted sum of all ridge functions.
Visualization of XNN Components
The structures built into XNN (such as the projection layer and subnetworks) provide a mechanism to explain the functions learned by this network. This built-in mechanism describes the complex functions learned by the model in relatively simple terms of projections and univariate ridge functions to ensure the model is interpretable.
The diagram below shows how researchers explain and visualize the univariate transformations and ridge functions within the subnetwork.
Ridge functions (left) and corresponding projection indices of the trained XNN (right).
The first column explains the univariate functions, which contain the weights learned by the subnetwork to reach their outputs. The second column shows the values of βi, which are the projection coefficients. The projection coefficients indicate which combination of input features was used as input for each ridge function before passing through the subnetwork. This is very useful, as the above diagram shows the most relevant features of the network: scaled ridge functions and projection coefficients.
In the example above, from the structure of XNN we can see that Subnetwork 1 has learned a cubic Legendre function (f3(·)), Subnetwork 2 has learned a quadratic function (f2(·)), and only the coefficient of x2 is non-zero.
XNN as an Alternative Model
XNN can also serve as an alternative model to machine learning models such as Random Forest (RF) and Feedforward Neural Networks (FNN).
Something like this… but not exactly.
In this case, RF and FNN are considered the base models. Since XNN is designed as an interpretable model, we can train XNN using the corresponding output values predicted by the input features and the base models. Then, XNN can explain the relationships learned by the base models!
Using more interpretable alternative models to help explain complex machine learning models greatly increases the ability to integrate machine learning models across various industries.
Explainable Neural Networks (XNN) are a key machine learning model. Unlike other machine learning models, it can “open” the black box of neural networks. The structure and design of this model allow it to explain the learned features and the functions that lead to its outputs or predictions. These interpretable features are very attractive as they are inherently additive and can be directly explained by incorporating mechanisms within the neural network (such as subnetworks).
Whether used as a primary model or as an alternative to more complex models, XNN can directly explain how the model makes predictions using input features. This technology offers tremendous advantages for integrating machine learning models into various industries as it can transcend existing systems and clearly explain how it derives outputs.
-
Related Paper: Explainable Neural Networks based on Additive Index Models
-
Paper Address: https://arxiv.org/abs/1806.01933
Original Link: https://medium.com/@shagunm1210/the-explainable-neural-network-8f95256dcddb
This article is translated by Machine Heart, please contact this public account for authorization to reprint.
✄————————————————
Join Machine Heart (Full-time Reporter / Intern): [email protected]
Submit articles or seek coverage: content@jiqizhixin.com
Advertising & Business Cooperation: [email protected]