Activation Functions

Understanding the Mathematical Essence of Convolutional Networks

by
Understanding the Mathematical Essence of Convolutional Networks

Recently, researchers from Nanyang Technological University published a paper that describes the mathematical principles of convolutional networks. This paper explains the operations and propagation processes of convolutional networks from a mathematical perspective. It is very helpful for understanding the mathematical essence of convolutional networks and aids readers in implementing convolutional networks “from scratch” (without using … Read more

Implementing CNN From Scratch: Understanding the Mathematical Essence

by
Implementing CNN From Scratch: Understanding the Mathematical Essence

Selected from arXiv Translated by Machine Heart Contributors: Huang Xiaotian, Lu Xue, Jiang Siyuan Recently, researchers from Nanyang Technological University published a paper describing the mathematical principles of convolutional networks. This paper explains the entire operation and propagation process of convolutional networks from a mathematical perspective. It is very helpful for understanding the mathematical essence … Read more

Understanding Q, K, and V in Attention Mechanisms

by
Understanding Q, K, and V in Attention Mechanisms

Question: I have searched various materials and read the original papers, which detail how Q, K, and V are obtained through certain operations to derive output results. However, I have not found any explanation of where Q, K, and V come from. Isn’t the input to a layer just a tensor? Why do we have … Read more

Identifying Function Monotonicity Using Derivative Graphs

by
Identifying Function Monotonicity Using Derivative Graphs

To identify the monotonicity of a function based on the graph of its derivative, we need to understand the relationship between the derivative and the function’s monotonicity. The derivative describes the instantaneous rate of change of the function at a given point. If the derivative is positive, the function is increasing at that point; if … Read more

Stanford CS231N Deep Learning and Computer Vision Part 6: Neural Network Structure and Activation Functions

by
Stanford CS231N Deep Learning and Computer Vision Part 6: Neural Network Structure and Activation Functions

This article is a Chinese version of the Stanford University CS231N course notes, authorized for translation and publication by Professor Andrej Karpathy of the Stanford course. This is a work by Big Data Digest, and unauthorized reproduction is prohibited. For specific requirements for reproduction, please see the end of the article. Machine Learning Online Training … Read more