China Telecom Implements AI Using TensorFlow

The Telecom Business Hall APP, as the entry-level application for China Telecom’s online services, has allowed its development team to have close contact with TensorFlow and artificial intelligence (AI) technology. AI is an area that the Telecom Business Hall APP had never explored before, and even the engineers involved in the project transitioned from Android development. This article will discuss how the Telecom Business Hall APP uses TensorFlow to implement AI from a machine learning perspective and focus on the engineers’ journey.

To enhance the user experience and allow users to feel the so-called black technology in the most commonly used function of recharging, we launched the recharge card scanning project in 2016, hoping users could easily complete the recharge by pointing their camera at the recharge card password. “The core OCR recognition is considered a relatively old research area, and the recharge card password is a combination of numbers and spaces, so it shouldn’t be too difficult,” said an engineer.

The solution used by the engineers of the Telecom Business Hall APP at that time was roughly as follows:

Of course, the final effect was not ideal. The recharge card is different from a business card; the size of the scratched area for the password varies from person to person, noise is very irregular, and hand shaking during scanning increases the likelihood of errors. Additionally, lacking relevant experience in the OCR field, we ultimately had to shelve the project due to the inability to achieve a good user experience. However, in the days that followed, the engineers stated, “We have formed an inseparable bond with TensorFlow.”

As seen at the Google Developer Conference in 2017, Google’s underlying technology for artificial intelligence has been open-sourced. The release of TensorFlow aims to shift ordinary developers’ focus from ‘how to reinvent the wheel’ to ‘the patterns and styles of the wheel.’ During the conference, Google engineers introduced this framework entirely in Chinese, which deeply inspired our engineers.

They boldly overturned their previous approach and changed their thinking: to build a neural network that learns to recognize the telecom recharge card.

end2end learning:

The concept of neural networks brought a complete change to the original approach. The so-called end2end model aims to directly obtain the result number from the input image through this model. This method no longer requires manual processing of the image features; any errors in any part are treated as part of the whole, and can be corrected through the backpropagation of the neural network.

seq2seq learning:

Seq2seq is a network structure with an Encoder-Decoder architecture. Its input is a sequence, and its output is also a sequence. The Encoder transforms a variable-length signal sequence into a fixed-length vector representation, while the Decoder converts this fixed-length vector into a variable-length target signal sequence. This structure of neural networks is commonly used in speech recognition and can also be used for OCR recognition of variable-length text. For our project, the Encoder input is an image of size 200*30, and the Decoder is a variable-length combination of numbers and spaces.

Neural Network Architecture: CNN + LSTM + CTC

This is a mature and advanced neural network architecture. After analyzing and studying many scientific papers, we chose this solution. The CNN convolutional neural network is responsible for capturing image features, the LSTM long short-term memory network is responsible for character recognition, and the CTC algorithm optimizes the alignment of results. There are many excellent TF projects that can be referenced, and implementing its network structure with TF is also very convenient, which allows us to start development work more quickly and easily.

When we first got in touch with neural networks, we saw that a certain expert once said, “Deep learning frameworks are a cauldron for alchemy.” This description is very apt and helps us understand the machine learning process more intuitively. Today, we already have this cauldron and recipe (TensorFlow, network structure solution), and the next key step is data. The quality and quantity of data directly affect the model’s recognition performance. Due to insufficient samples of physical cards, we used a program to simulate the required images, aiming to make them as realistic as possible and include as many features as possible. We started with a simple recharge card, narrowing our focus to a 200*30 area:

By continuously training and comparing the model’s recognition results, we incorporated features that the model had not yet learned, resulting in the following data optimization route:

Basic Version:

Inclination, distortion, displacement:

Different fonts:

Simulated unclean effects, adding random noise:

Strong light:

Weak light:

Shaking blur:

The above features appear randomly combined, and the strength is randomly selected within a certain range, resulting in a training dataset of 300,000 and a test dataset of 10,000:

Next is the process of machine training, which is the stage that tests patience; while the machine learns, people also grow. Here are a few insights:

1. We can summarize this experience from hundreds of training records; keeping detailed notes of each training session is a good habit.

2. Only adjust network parameters or modify data features each time, which facilitates comparison of results.

3. Utilizing TensorBoard helps you view the network structure more intuitively and understand the training process. We kept all training logs of TB, and comparing the training curves of different hyperparameters is very convenient.

Finally, regarding the selection of the solution, we initially favored TF Lite for its compactness and flexibility. However, at that time, TF Lite did not support the LSTM operator. In the future, we will try using TF Lite again because edge-side artificial intelligence should be the mainstream in the future. Considering that our images are only 200*30 pixels, actual transmission does not consume much network, so TF Serving was chosen as the final solution, and it worked quite well.

The final effect: Backlight, shaking, and severe obstruction of numbers can still be easily recognized!

The model training success rate is 99.3%, with a recognition time of 0.05 seconds per instance, achieving such results has amazed and excited us! The user experience feedback has also been very positive! Achieving this is thanks to the excellent TensorFlow machine learning framework and advanced neural network solutions. It reminds me of my feelings when I first encountered the Android system in 2007: no longer needing to consider writing the underlying code significantly lowers the threshold for developers, just as TF offers us today, allowing entry-level players to use AI technology.

Finally, we pay tribute to the developers who have persisted in machine learning: your persistence and spirit of sharing enable us to master and use this technology more quickly. I believe that with our joint efforts, breakthroughs in the field of artificial intelligence will continue, just as this curve has experienced trials and has seen the moon through the clouds.