Iterative Data Loading with XGBoost
Datawhale Insights Source: Coggle Data Science During the process of reading and training on large-scale datasets, iterative data loading is a very suitable choice, and Pytorch supports this method of iterative reading. Next, we will introduce the iterative loading method of XGBoost. Memory Data Loading class IterLoadForDMatrix(xgb.core.DataIter): def __init__(self, df=None, features=None, target=None, batch_size=256*1024): self.features = … Read more