200,000, this is the total number of users New Intelligence has reached today. On the journey to an intelligent universe, we thank every friend who travels with New Intelligence. Your attention and support is the inexhaustible fuel for the “New Intelligence” starship.
200,000, every passenger is invaluable to us. We hope to deepen our understanding of each passenger and kindly ask for your valuable feedback on this brief survey that does not involve any privacy.
New Intelligence is holding a comment gift activity, click to read the original text and vote, and leave your suggestions for the New Intelligence public account to win a free copy of “Deep Learning”.
1Translated by New Intelligence
Source: IEEE Spectrum
Author: Evan Ackerman
Translator: Wen Qiang
[New Intelligence Guide]A previous study on adversarial attacks required complex data processing, but recently a group of researchers from the University of Washington, the University of Michigan, Stony Brook University, and the University of California, Berkeley published an article showing that slight changes in the physical world can also deceive visual classification algorithms. Completely eliminating human factors may be the way to ensure safety in autonomous driving.
It is very difficult for humans to understand how robots “see” the world. Machine cameras work like our eyes, but in the space between the images captured by the cameras and the information that can be processed from these images, there is a black box of machine learning algorithms. Training these algorithms typically involves showing the machine a set of different images (like stop signs) and then seeing if the machine can extract enough common features from these images to reliably identify stop signs that were not present in the training set.
This works well, but the features identified by machine learning algorithms for stop signs are often not “a red octagon with the letters STOP inside,” but rather features shared by all stop signs that humans cannot comprehend. If you find this hard to imagine, it is because this fact actually reflects a fundamental gap between our brains and artificial neural networks in interpreting/understanding the world.
The result is that slight modifications to an image can lead to machine learning algorithms recognizing something completely different (sometimes even nonsensical) from the original.
Generally speaking, these slight modifications are imperceptible to the human eye and usually require relatively complex analysis and image processing to achieve.
Below is a set of common “adversarial images” that have been “polluted”:
The original image of a giant panda (left), after adjustments invisible to the human eye (middle), results in the computer identifying it as a gibbon (right, confidence 99.3%).
Applied to road signs, it becomes this:
The top row shows valid signs, while the bottom row has been manipulated to cause the neural network classifier to produce incorrect identifications.
Clearly, while such modifications are effective (and dangerous), they are difficult to implement in practice because you generally cannot directly obtain the inputs of the neural network you want to confuse. Additionally, in the case of autonomous driving, neural networks can analyze images of a multitude of symbols from different distances and angles. Adversarial images often contain added modifications throughout the entire image (i.e., both the road signs and the background), making such “pollution” often ineffective in real life.
However, recently a group of researchers from the University of Washington, the University of Michigan, Stony Brook University, and the University of California, Berkeley published an article showing that slight changes in the physical world can also deceive visual classification algorithms. You only need to add a bit of spray paint or some stickers to a stop sign to trick a deep neural network classifier into seeing the stop sign as a speed limit sign.
Below are two examples where confusion is caused solely by stickers:
Because the area of the stickers is relatively small compared to the entire sign, the interference caused is even more severe. According to the researchers:
“According to our assessment method, 100% of the image classifiers misclassified the stop sign as a speed limit of 45. For the right turn sign… our attack achieved a misclassification success rate of 100%, with 66.67% of images classified as stop signs and 33.7% classified as added lane signs. [Graffiti] attack had a success rate of 73.33%. [Camouflaged abstract art attack] achieved a 100% misclassification rate.”
To implement these attacks, the researchers used publicly available road sign datasets and trained their road sign classifiers on TensorFlow. They believe that attackers will have “white-box” access to the classifier, meaning that attackers do not obfuscate or tamper with the data but instead add “clutter” to see what comes out. Thus, even if they cannot directly invade the classifier, attackers can still use this feedback to create a fairly accurate model to classify them. Finally, the researchers added images of the signs they wanted to attack to their classifier and included them in the attack algorithm, allowing the algorithm to output adversarial images.
Of course, the classifiers used by autonomous vehicles are more complex and robust than those that researchers successfully deceived (in the experiment, the researchers only used about 4,500 signs as training input). Nonetheless, it cannot be denied that attacks like this can be effective—even the most advanced deep neural network-based algorithms can make very silly judgments, and the reasons are not easily perceptible. Therefore, autonomous vehicles are best served by using multimodal systems for road sign recognition, just as they use multimodal systems for obstacle detection: relying solely on one type of sensor (whether radar, LiDAR, or camera) is very dangerous. Thus, multiple sensors should be used simultaneously to ensure they cover each other’s specific vulnerabilities.
Therefore, if a visual classifier is to be developed for autonomous vehicles, it should also include some GPS location signals. Alternatively, a dedicated red octagon detection system could be added. However, my suggestion is to completely remove all road signs (relying entirely on no road signs), completely eliminate human factors, and hand over all roads entirely to robots. This way, the problem is solved.
-
Related Paper: Robust Physical-World Attacks on Machine Learning Models (https://arxiv.org/abs/1707.08945)
Translated from: http://spectrum.ieee.org/cars-that-think/transportation/sensors/slight-street-sign-modifications-can-fool-machine-learning-algorithms
[Breaking News]New Intelligence is conducting a new round of recruitment, the most beautiful spaceship flying towards the intelligent universe still has N seats available.
Click to read the original text for job details, looking forward to your joining~
