A Comprehensive Guide to TensorFlow 2.0

Source: Authorized by AI Technology Camp (ID: rgznai100)

This article is approximately 3000 words long and is recommended to be read in 7 minutes.

This article will introduce you to a comprehensive manual detailing TensorFlow 2.0.

[ Guide ]After the official release of TensorFlow 2.0, it immediately received widespread attention and praise from the academic and research communities. Previously, we invited experts to review the development history of TensorFlow 2.0 from its initial version to the latest version. Today, we will introduce a comprehensive manual detailing TensorFlow 2.0. This manual is part of the Google Summer of Code 2019 project, covering everything from basic installation and environment configuration, deployment, to large-scale training and acceleration, providing a comprehensive introduction to TensorFlow 2.0, along with relevant materials for readers’ reference.

This is a concise guide for getting started with TensorFlow 2.0, based on Keras and Eager Execution, aiming to help developers with a certain foundation in machine learning and Python quickly get started with TensorFlow 2.0.

All the code in this manual is based on the official version and Beta1 version of TensorFlow 2.0.

Chinese Guide:

https://tf.wiki/

GitHub Link:

https://github.com/snowkylin/tensorflow-handbook

All example codes in the article:

https://github.com/snowkylin/tensorflow-handbook/tree/master/source/_static/code/zh

All chapters of this manual are shown in the figure below, which not only includes an overview of TensorFlow but also covers basics, deployment, large-scale training and acceleration, expansion, and appendices.

Preface

Previously, the drawbacks of the traditional Graph Execution model on which TensorFlow is based, such as high entry barriers, debugging difficulties, poor flexibility, and the inability to use native Python control statements, have long been criticized by developers. Some new deep learning frameworks based on dynamic graphs (like PyTorch) have emerged and gained a foothold due to their ease of use and rapid development characteristics. Especially in fields like academic research that require rapid model iteration, emerging deep learning frameworks like PyTorch have become mainstream. In the machine learning lab of dozens of people where the author works, only one person is still “old-fashioned” in using TensorFlow.

However, until now, most Chinese technical books and materials related to TensorFlow are still based on the traditional Graph Execution model, making it difficult for many beginners (especially college students who have just learned machine learning courses) to get started. Therefore, at the time when TensorFlow officially supports Eager Execution, it is necessary to have a brand new technical manual to help beginners and researchers who need quick model iteration to get started with TensorFlow from a new perspective.

At the same time, this manual has a second task. Most Chinese technical books related to TensorFlow mainly focus on deep learning, treating TensorFlow as a means to implement these deep learning models. While this certainly has the advantage of being comprehensive, it may not be friendly enough for readers who already have some understanding of machine learning or deep learning theory and wish to focus on learning TensorFlow itself.

Therefore, the author hopes to write a manual that presents the main features of TensorFlow as a computational framework in a concise manner, compensating for the shortcomings of the official manual, aiming to enable readers with a certain level of machine learning/deep learning knowledge and programming ability to quickly get started with TensorFlow and refer to it at any time to solve practical problems during programming.

Target Audience

This book is suitable for the following readers:

• Students and researchers with a certain foundation in machine learning/deep learning who wish to implement the theoretical knowledge they have learned using TensorFlow;
• Developers who have used or are currently using TensorFlow 1.X or other deep learning frameworks (such as PyTorch) and wish to understand the new features of TensorFlow 2.0;
• Developers or engineers who wish to apply existing TensorFlow models in the industry.

Note

This book is not an introductory manual for the principles of machine learning/deep learning. If you wish to learn the theory of machine learning/deep learning, please refer to some introductory materials provided in the appendix.

Overview of TensorFlow

• Students and researchers: Model building and training
• Developers and engineers: Model invocation and deployment
• What can TensorFlow help us do?

Basics

1. TensorFlow Installation and Environment Configuration

The latest installation steps for TensorFlow can be found in the official website instructions (https://tensorflow.google.cn/install). TensorFlow supports various programming languages such as Python, Java, Go, C, and various operating systems such as Windows, OSX, Linux, with this section and subsequent sections using Python 3.7 as the standard.

Note

This chapter introduces the method to directly install TensorFlow 2.0 on a general personal computer or server. For methods to deploy TensorFlow in container environments (Docker), cloud platforms, or use TensorFlow in online environments, refer to the appendix on using Docker to deploy TensorFlow environments and using TensorFlow in the cloud. Software installation methods often have a time sensitivity, and the update date for this section is October 2019.

• General installation steps

• GPU version TensorFlow installation guide

• Preparation of GPU hardware

• Installation of NVIDIA drivers

• Installation of CUDA Toolkit and cuDNN

• The first program

• IDE setup

• Hardware configuration required for TensorFlow *

2. TensorFlow Basics

This chapter introduces the basic operations of TensorFlow.

• TensorFlow 1+1

• Automatic differentiation mechanism

• Basic example: Linear regression

• Linear regression under NumPy

• Linear regression under TensorFlow

3. Building and Training TensorFlow Models

This chapter introduces how to quickly build dynamic models using TensorFlow.

• Model (Model) and layer (Layer)

• Basic example: Multi-layer perceptron (MLP)

• Data acquisition and preprocessing: tf.keras.datasets

• Model construction: tf.keras.Model and tf.keras.layers

• Model training: tf.keras.losses and tf.keras.optimizer

• Model evaluation: tf.keras.metrics

• Convolutional Neural Networks (CNN)

• Implementing Convolutional Neural Networks using Keras

• Using predefined classic convolutional neural network structures in Keras

• Recurrent Neural Networks (RNN)

• Deep Reinforcement Learning (DRL)

• Keras Pipeline *

• Building models using Keras Sequential/Functional API modes

• Training and evaluating models using Keras Model’s compile, fit, and evaluate methods

• Custom layers, loss functions, and evaluation metrics *

• Custom layers

• Custom loss functions and evaluation metrics

4. Commonly Used TensorFlow Modules

• tf.train.Checkpoint: Saving and restoring variables
• TensorBoard: Visualization of the training process
• tf.data: Building and preprocessing datasets

• Building dataset objects
• Preprocessing dataset objects
• Accessing and using dataset elements
• Example: Cats_vs_dogs image classification

• @tf.function: Graph Execution mode *

• Basic usage of @tf.function

• Internal mechanism of @tf.function

• AutoGraph: Converting Python control flow into TensorFlow computation graphs

• Using traditional tf.Session

• tf.TensorArray: TensorFlow dynamic arrays *

• tf.config: Using and allocating GPU *

• Specifying the GPU used by the current program

• Setting memory usage policies

• Simulating a multi-GPU environment with a single GPU

Deployment

• Exporting TensorFlow models
• Using SavedModel for complete model export
• Keras Sequential save method (Jinpeng)
• TensorFlow Serving

• Installation of TensorFlow Serving

• Model deployment with TensorFlow Serving

• Deployment of models in Keras Sequential mode
• Deployment of custom Keras models
• Calling models deployed with TensorFlow Serving on the client side
• Python client example
• Node.js client example (Ziyang)
• TensorFlow Lite (Jinpeng)
• Model conversion
• Android deployment
• Quantization model conversion
• Summary
• TensorFlow in JavaScript (Huan)
• Introduction to TensorFlow.js
• Advantages of using TensorFlow.js in the browser
• Performance comparison of TensorFlow.js
• Environment configuration for TensorFlow.js
• Using TensorFlow.js in the browser
• Using TensorFlow.js in Node.js
• Using TensorFlow.js in WeChat Mini Programs
• Model deployment with TensorFlow.js
• Loading Python models with TensorFlow.js
• Using TensorFlow.js model library

• Model training with TensorFlow.js *

Large-Scale Training and Acceleration

1. Distributed Training with TensorFlow

When we have a lot of computing resources, we can fully utilize these resources through appropriate distributed strategies, significantly shortening the time for model training. Depending on different usage scenarios, TensorFlow provides several distributed strategies in tf.distribute.Strategy, allowing for more efficient model training.

• Single machine multi-GPU training: MirroredStrategy

• Multi-machine training: MultiWorkerMirroredStrategy

2. Training TensorFlow Models with TPU (Huan)

In May 2017, Alpha Go competed against the world’s number one player Ke Jie at the Go Summit in Wuzhen, China, achieving a clean sweep of three wins. The subsequent version, Alpha Zero, could reach the level of Alpha Go Master, which defeated Ke Jie, in just 21 days through self-learning.

The power behind Alpha Go was entirely provided by TPU, enabling it to “think” faster and see further between each step.

• Introduction to TPU
• What is TPU
• Why use TPU
• TPU performance
• TPU environment configuration
• Free TPU: Google Colab
• Cloud TPU
• Basic usage of TPU

Expansion

• TensorFlow Hub Model Reuse (Jinpeng)
• TensorFlow Datasets Loading Datasets
• Swift for TensorFlow (S4TF) (Huan)
• Introduction to S4TF
• Why use Swift for TensorFlow development
• S4TF environment configuration
• Locally installing Swift for TensorFlow
• Quickly experiencing Swift for TensorFlow in Colaboratory
• Quickly experiencing Swift for TensorFlow in Docker
• Basic usage of S4TF
• Using standard TensorFlow API in Swift
• Directly loading Python libraries in Swift
• Language-native support for automatic differentiation
• MNIST digit classification
• TensorFlow in Julia (Ziyang)
• Introduction to TensorFlow.jl
• Why use Julia?
• TensorFlow.jl environment configuration
• Quickly experiencing TensorFlow.jl in Docker
• Installing TensorFlow.jl in Julia package manager
• Basic usage of TensorFlow.jl
• MNIST digit classification

Appendix

• TensorFlow under Graph Models
• TensorFlow 1+1
• Basic example: Linear regression
• Using Docker to deploy TensorFlow environments
• Using TensorFlow in the cloud
• Using TensorFlow in Colab
• Using TensorFlow in Google Cloud Platform (GCP)
• Establishing a GPU instance in Compute Engine and deploying TensorFlow
• Using AI Platform’s Notebook to create an online JupyterLab environment with GPU
• Running TensorFlow on GPU instances in Alibaba Cloud (Ziyang)
• Deploying your interactive Python development environment JupyterLab
• TensorFlow performance optimization
• Reference materials and recommended reading
• Glossary of terms in Chinese and English

Q&A Section

• (Chinese) TensorFlow Chinese Community “Simple and Direct TensorFlow” Forum:https://www.tensorflowers.cn/b/48 (For questions and suggestions in Chinese, please come here, and discussions will be answered in Chinese. Developers using Chinese are welcome to come to the TensorFlow Chinese Community for exchange and discussion.)
• (English)https://github.com/snowkylin/tensorflow-handbook/releases (Questions or suggestions in English can be raised in GitHub issues, and will be answered in English.)

PDF Download (Old Version):

• Chinese Version:
  https://www.tensorflowers.cn/t/6230 (There is also an English version available for download.)
• English Version:
  https://github.com/snowkylin/tensorflow-handbook/releases

Search Page:

https://tf.wiki/search.html

The Author is a Top Student from Peking University

Finally, it is necessary to introduce the author of “Simple and Direct TensorFlow 2.0”, Li Xihan.

Image Source:LinkedIn

According to LinkedIn information, Li Xihan is a master’s student from the School of Information Science and Technology at Peking University, serving as an intern consultant and research assistant. He graduated in 2016 from Zhejiang University’s Zhuhai College and entered Jiaotong University as an exchange student in 2015.

Li Xihan interned in the machine learning group at Microsoft Research Asia, and was a Google Developers Expert (GDE) in machine learning at Google. His current research focus is on the application of multi-agent reinforcement learning in graph-based real-world scenarios.

Editor: Huang Jiyan

Proofreader: Gong Li