Improving ETF Trading Strategies with XGBoost for 33.99% Annual Return

1. Introduction

This article mainly introduces the trading strategies for margin ETFs. As an index tracking tool, ETFs have good asset allocation value. At the same time, by combining price diffusion and Minsky moments, the advantages of ETFs can be fully utilized. Based on a “top-down” approach to construct ETF trading strategies, it can capture price momentum and reduce losses during the “Minsky” moment when momentum ends.

However, the price momentum indicators have subjective factors and limitations of non-intelligence. Therefore, this article uses machine learning methods to improve the ETF strategy, making the input variables more direct and richer, and the results have effectively improved compared to the original strategy. Backtesting on data since 2020 shows that the strategy has an annualized return of 33.99%, with a Sharpe ratio and Calmar ratio of 1.37 and 1.57, respectively.

2. Background Knowledge

Margin Trading

Margin trading (two-way margin) refers to the mechanism that allows investors to trade stocks using their own funds (margin) or borrowed funds (short selling) in the stock exchange. Under this mechanism, investors can use leverage within a certain ratio to trade stocks, thereby increasing their profit levels.

Specifically, the margin trading mechanism usually includes two parts: margin trading and short selling. Margin trading refers to investors obtaining loans by pledging the stocks they hold to securities firms, and then using these funds to purchase other stocks. Short selling refers to investors borrowing stocks, selling them within a certain period, and promising to return the same quantity of stocks within the agreed time.

In margin trading, investors need to pay certain interest and fees. In addition, the leverage ratio is also limited to control the risk for investors. Although margin trading can increase profit opportunities, it also carries higher risks, as leveraged trading can amplify investment losses, requiring investors to choose and manage risks carefully.

ETF

ETF stands for Exchange Traded Fund. It is an open-end index fund designed to provide investors with broad exposure to a specific market, industry, or investment strategy. ETFs trade similarly to individual stocks and can be listed on stock exchanges, allowing investors to buy and sell through regular brokerage accounts.

Compared to traditional index funds, ETFs have higher liquidity and trading flexibility, as they can be traded at any time during the trading day, with prices fluctuating in real-time. Additionally, ETFs typically offer diversified investment opportunities at low costs, making them one of the increasingly popular investment tools. Some well-known A-share ETFs include Huaxia SSE 50 ETF, Bosera CSI 300 ETF, and Harvest CSI 500 ETF. They can also track specific industries or investment strategies.

As of now, there are 249 margin trading ETFs in the Shanghai and Shenzhen stock markets, with a total asset management scale exceeding 1.2 trillion yuan, already establishing a good trading foundation. Moreover, there are significant differences in returns among different asset class ETFs, with the maximum and minimum monthly returns of stock ETFs differing by nearly 20% in the past year.

Classification of ETF Industries

Based on the correlation between stock ETF returns and the first-level industry index returns of Shenwan, margin stock ETFs (excluding broad-based ETFs and overseas ETFs) are classified into 22 industries. Due to the small number of ETFs in some industries, for the sake of scale and liquidity, appropriate mergers between industries are conducted.

Influenced by economic cycles, investor sentiment, and other factors, industry market rotation occurs, leading to significant differences in returns among different industries, with the monthly return difference between the best and worst-performing industries reaching as high as 25.86%, and the minimum reaching 8.83%. Due to the differences in ETF returns among different industries, industry timing has practical significance, and increasing the weight of high-return industry ETFs can enhance the strategy’s return.

Price Momentum

The price momentum strategy is an investment strategy based on predicting future stock prices from historical price trends. This strategy believes that stock price movements have inertia, meaning that an upward or downward trend tends to continue for a period. Based on this idea, the price momentum strategy selects stocks that have performed well over a past period and buys them while selling poorly performing stocks.

This strategy can use technical analysis tools such as relative strength index (RSI) and moving averages (MA) to determine buying and selling timing. The advantages of the price momentum strategy are its simplicity and flexibility, while its disadvantages include the need for a deeper understanding and analytical ability of the market, as well as issues like overfitting and stock selection risk. The price momentum strategy is widely used in practice, especially suitable for situations where the stock market is volatile or trends are hard to determine.

Crowding

Crowding refers to the concentration of investors on a specific asset or investment strategy. When a large number of investors flock to the same stock, investment strategy, or asset class, it leads to an increase in the asset or strategy’s crowding. Increased crowding can lead to mispricing in the market, as investors are often driven by emotions rather than rational analysis when chasing gains or cutting losses.

At the same time, high crowding can also increase liquidity risk and operational risk, as when investors want to exit, the market may not have enough buyers to take over these investors’ positions, leading to price declines. For investors, understanding crowding can help them manage risks better and avoid blindly following trends.

XGBoost Algorithm

XGBoost (eXtreme Gradient Boosting) is a commonly used ensemble learning model based on the gradient boosting tree algorithm. XGBoost has wide applications in data mining, machine learning, and statistical modeling. XGBoost constructs a strong classifier by serially training and weighting weak classifiers (decision trees) to gradually reduce errors. Specifically, it adopts a gradient boosting framework to train each tree by optimizing the loss function while using regularization methods to avoid overfitting. The advantages of this algorithm include:

• Good robustness, capable of handling various types of data, including categorical and continuous variables.
• Can be processed in parallel, with low memory consumption, demonstrating high efficiency.
• Excels in feature selection and handling missing values.
• Can automatically handle non-linear relationships and feature interactions.

XGBoost is a very powerful machine learning model that can be used for regression and classification problems and has shown excellent performance in practical applications across multiple fields.

3. Work of This Article

3.1 Momentum Timing Dynamic ETF Allocation Strategy (PMI Strategy)

Strategy Construction

The momentum timing dynamic ETF allocation strategy (PMI strategy) is a three-step approach based on a “sector-individual” top-down method to select ETFs.

First, calculate the scores of each sector based on the sector PMI (Purchasing Managers’ Index) and crowding, and select the top 5 sectors ranked by PMI and the top 5 sectors ranked by crowding score (historical percentile) as the sectors to buy.

Second, within the selected sectors, choose the ETF with the lowest PMI and the lowest crowding score as the investment target. If there are multiple ETFs that meet the criteria, choose the one with the highest trading volume to reduce trading costs.

Finally, calculate the weights of each ETF based on the historical percentiles of ETF fund shares, where higher historical percentiles lead to lower weights. This method helps investors to scientifically select ETFs and determine the weights of each ETF in their portfolio, reducing investment risks and improving investment returns.

Strategy Backtesting

The annualized return of the PMI strategy is 27.42%, with an annualized volatility of 21.26%, and a Sharpe ratio of 1.25, and a Calmar ratio of 1.65.

However, in comparison, the performance of the CSI 300 ETF allocation strategy is much worse, with an annualized return of only 1.13%, an annualized volatility of 19.56%, a Sharpe ratio of 0.16, and a Calmar ratio of 0.31. The 60/40 allocation strategy also performed worse than the PMI allocation strategy, with an annualized return of 1.88%, an annualized volatility of 11.71%, a Sharpe ratio of 0.22, and a Calmar ratio of 0.08. Therefore, constructing an ETF investment portfolio based on momentum and crowding can achieve more stable returns and show good returns in long-term investments.

3.2 Improving Trading Strategies Using XGBoost Algorithm

The PMI strategy uses a simple price momentum quantification method (such as MA moving averages), which is prone to lag and misjudgment, and is insensitive to complex market environments and multiple factors. Therefore, it is necessary to conduct multi-dimensional optimization and upgrading for momentum assessment to enhance the strategy’s judgment capability and investment returns.

Strategy Improvement The XGBoost algorithm includes six steps: defining the problem, data selection, feature engineering, model training, model evaluation, and decision-making:

1) Define the Problem: Predict the probability of ETF price increase. For stock ETFs: if the average return over the last 5 days is greater than 0.3% and the return of the current day is greater than zero, it is considered an increase, labeled as 1; otherwise, it is labeled as 0. For bond ETFs: if the average return over the last 5 days is greater than zero and the return of the current day is greater than zero, it is labeled as 1; otherwise, it is labeled as 0.

2) Data Selection: The sample includes margin stock ETFs and bond ETFs, with the sample period from 2016.01.01 to 2023.04.16.

3) Feature Engineering: Select feature variables that have the ability to predict ETF price increases, including 5, 10, 20, 30, 40, 50, 60, and 75-day moving averages, as well as the recent 10-day cumulative return (roc) and recent 10-day closing price cumulative growth (mom). Based on XGBoost feature importance ranking, the closing price has the greatest impact on the probability of ETF price increase, while the 30-day moving average MA60 has the least impact.

4) Model Training: The dataset is divided into training and test sets. The training set sample period for this report is from 2016.01.01 to 2019.12.31. Model training requires pre-setting parameters, i.e., hyperparameters, which can be optimized using grid search algorithms, random grid search, and manual tuning methods. In automated search methods, existing problems include excessive computational load and lengthy optimization processes. This report selects random grid search.

5) Model Evaluation: The goal of machine learning is to fit a “good” model with good predictive capability, so the model’s effectiveness needs to be evaluated. Evaluation metrics include accuracy, precision, recall, and F1 score. According to the model confusion matrix, the accuracy, precision, recall, and F1 score are 0.64, 0.85, 0.64, and 0.73, respectively.

6) Decision-Making: Output the probability of ETF price increase for the test set as a substitute variable for PMI.

Strategy Backtesting

In the improved strategy, we use the ETF price increase probability predicted by XGBoost as a substitute for the PMI indicator, no longer directly inputting the original moving average data into the model. The training period is from 2016 to 2019, with backtesting data from 2020 onwards used as the test set for the model.

The annualized return of the strategy increased to 33.99%, with an annualized volatility of 23.34%. The Sharpe ratio and Calmar ratio are 1.37 and 1.57, respectively, outperforming the original strategy. However, the strategy’s effectiveness has declined in the past two years; using a rolling optimization approach would yield better results. Therefore, in practice, it is necessary to pay attention to adjusting parameters and strategy cycles to ensure the stability and return level of the investment portfolio.

4. Conclusion and Outlook

The ETF industry rotation strategy based on momentum and crowding timing can effectively avoid the impending “Minsky” moment when momentum ends, while capturing momentum returns and reducing losses caused by price declines. By first selecting industries and then ETFs, a “top-down” approach to constructing ETF allocation strategies can achieve stable returns. However, since the price momentum PMI construction method relies on empirical rules, it may have certain subjectivity and uncertainty.

Therefore, we have improved the ETF allocation strategy using machine learning methods, resulting in a higher performance that can better adapt to market changes and risk management needs. In practice, investors need to combine their risk tolerance and investment goals to develop ETF allocation strategies scientifically, thereby improving the robustness and return level of their investment portfolios.

Note | This article is for knowledge sharing only and does not constitute any investment advice.

1. The Most Basic Monitoring Method for MYSQL

2. Using Python Algorithms to Predict Customer Behavior Case!

3. 10 Top Clustering Algorithms Python Implementation (with Complete Code)