XGBoost utilizes a second technique which is second-order optimization, expanding the loss function l(x,y) using a Taylor series expansion. To approximate it to the second order. This is relatively unique in the XGBoost algorithm, differing from the approach of optimizing using gradient descent in GBT, and also different from Adaboost which increases the weights of … Read more
XGBoost is an excellent algorithm that has been widely used in many competitions such as Kaggle, where we can see that many winning teams utilize XGBoost and achieve outstanding performance. Currently, most classification models in practical applications are based on XGBoost, making it a very practical and user-friendly algorithm. The main reference for this article … Read more
This article is excerpted from “Introduction to Machine Learning Basics (Micro Course Version)” 10.5 XGBoost Algorithm XGBoost is a machine learning algorithm based on the gradient boosting algorithm (GBDT) invented by PhD student Tianqi Chen from the University of Washington in February 2014. This algorithm not only has excellent learning performance but also trains efficiently, … Read more