A Beginner’s Guide to Neural Networks

Produced by Big Data Digest

Compiled by: Li Lei, Da Jieqiong, Yun Zhou

If you have opened a browser in the past few years, you have definitely seen the term “neural network” hundreds of times.

In this short article, I will give you a preliminary introduction to this field and background information about neural networks themselves. The next 5 minutes may not make you a world-class expert in this field quickly, but it is very easy to take you through a meaningful introductory phase. Additionally, you will learn some popular terms (especially by following the list at the end for further reading), which can become topics for your casual conversations.

To understand neural networks, we first need to understand machine learning. And to understand machine learning, we must first talk about human learning, or “classical programming”.

In classical programming, I, as the developer, need to understand every aspect of the problem I am trying to solve and know exactly how to find the solution.

For example, suppose I want my program to know the difference between a square and a circle. One way to solve this problem is to write a program that can detect corners. If my program detects 4 corners, then the shape is a square; if it detects no corners, then the shape is a circle.

So what is machine learning? Generally speaking, machine learning = learning from examples.

In machine learning, when faced with the problem of distinguishing between circles and squares mentioned above, we will design a learning system that takes many examples of shapes and their categories (square or circle) as input, hoping that the machine can learn the features that can be used to distinguish the shapes on its own.

Then, once the machine has learned all the features, we can give it an image it has never seen before and hope it can classify it correctly.

In the context of neural networks, a neuron is a fascinating term that “smart” people are reluctant to call a function. Of course, in the context of mathematics and computer science, a function is also a fascinating name that takes input, implements processing logic, and outputs results.

More importantly, a neuron can be thought of as a learning unit.

Therefore, we need to understand what a learning unit is in the context of machine learning. After that, we can understand the most basic construction of neural networks, which is the neuron.

To illustrate this, suppose I am trying to understand the relationship between the number of words in a blog post and the number of words people actually read from that post. Note – we are doing this in the field of machine learning, so we are learning from examples.

So I collected many instances of word count statistics from blog posts, denoted as x, and how many words people actually read in those posts, denoted as y, assuming there is a certain relationship between them, denoted as f.

The magic of this is that I only need to tell the machine (program) what kind of relationship I expect to see (for example, a straight line), and the machine will understand the actual shape it needs to draw.

The next time I want to write a blog post with x words, the machine can use the relationship f it discovered to tell me how many words I can expect people to actually read, y.

Well, if a neuron is a function, then a neural network is a network of functions! This means we have many (many) such functions and learning units, and all their inputs and outputs are interwoven, meaning they are inputs and outputs of each other.

As the designer of this network, I need to answer the following questions:

• How to model the inputs and outputs? (For example, if the input is some text, can I model it with letters? What about numbers and vectors?…)

• What functions do each neuron have? (Are they linear? Or exponential?…)

• What is the architecture of the network? (That is, which function’s output is the input of which function?)

• What popular terms can I use to describe my network?

Once I have answered these questions, I can “show” my network many examples of correct input-output pairs, so that when I “show” it a new input it has never seen before, it will know the correct output.

You can also visit the super cool website, Neural Network Playground, to better understand what this process means.

Neural Network Playground:

https://playground.tensorflow.org/

Since this field is actually expanding continuously, the amount of new content appearing every minute is impossible for anyone to track. (I wonder if one day we will be able to build an AI that can track human progress in the field of AI)

When entering this field, the first thing to know is that no one knows everything. So you don’t have to worry about not being as good as others; just stay curious. 🙂

Therefore, I recommend some excellent resources:

Gal Yona, one of the fantastic bloggers in this field. Her posts include explanations of core technologies to semi-philosophical comments.

Explanation of core technologies:

https://towardsdatascience.com/do-gans-really-model-the-true-data-distribution-or-are-they-just-cleverly-fooling-us-d08df69f25eb

To semi-philosophical comments:

https://towardsdatascience.com/the-tale-of-1001-black-boxes-62d12b5886aa

Siraj Raval is a YouTuber with a lot of videos, from theoretical explanations to hands-on tutorials that are very interesting!

Theoretical explanations:

https://www.youtube.com/watch?v=xRJCOz3AfYY

Hands-on tutorials:

https://www.youtube.com/watch?v=pY9EwZ02sXU

Christopher Olah, a passionate and insightful researcher, has a super cool blog that includes posts from basic concepts to deep learning.

From basic concepts:

http://colah.github.io/posts/2015-09-Visual-Information/

Deep learning:

https://distill.pub/2017/feature-visualization/

Towards Data Science is the largest medium publication in this field, where you can start exploring everything from practical tools to deep algorithms, whether you have a few minutes or a few hours.

Towards Data Science:

https://towardsdatascience.com/

Related articles:

https://medium.freecodecamp.org/neural-networks-for-dummies-a-quick-intro-to-this-fascinating-field-795b1705104a