Editor: We have a user who enjoys using machine learning to experiment with strategies. His descriptions of several models are quite vivid, and he has written a demo strategy using PonderLSTM, which we are sharing today~
The ACT model simulates the thinking process of complex problems by performing multiple computations at each time step (time series node). This algorithm is especially valuable when using RNN variants with external memory (such as DNC, NTM, etc.) to process long time series.
Next, let’s briefly introduce the model in this article (this is not a very accurate description, but it is vivid enough). We assume that the Vanilla RNN model is like a student taking an English listening exam. This student is required to “just listen,” meaning they cannot use paper and pen to take notes, and all the listening questions and answers are conducted orally. Of course, excluding exceptional talents, this student’s answers would be dismal.
Now let’s look at the second student, called LSTM. This student has a magnetic drawing board that can record a limited amount of information. However, during a listening exam lasting over 30 minutes, this student can only record a little information on the board, and if they want to write new information, they must clear previous notes. So, this student can only record limited information. Of course, for a long listening exam, this amount of notes is barely sufficient.
Next, we have the third student named DNC. This student uses a more advanced drawing board (E-person E-book), which can record more information and even all listening information (when the DNC’s external memory is large enough).
All three students are enhancing their memory capabilities to process information for the listening exam (long time series). They answer questions in the order of the listening playback. Although the third student (DNC) can record almost all listening information, they do not have extra time to review all recorded information and can only glance through their notes based on impressions in their brain (short-term memory) and the order in which they wrote.
Next, we introduce the fourth student, known as PonderDNC, who has a time machine cheat.
PonderDNC not only has the E-person E-book but also possesses a time-stopping device. This means that this student can pause time at any point they feel is appropriate during the listening exam, allowing them to leisurely review their notes on the E-person E-book to answer questions.
Without a doubt, PonderDNC is theoretically the most likely candidate to score high in the listening exam.
PonderDNC
By embedding the DNC computational unit into the ACT computational architecture, PonderDNC can perform multiple computations at each time step. This means that at time step t, after receiving external input at time t, the DNC does not need to output information immediately but can repeatedly think and compute at time t before making a decision output, then proceed to the next time step t+1. As shown in the figure below
The ACT (Adaptive Computation Time for Recurrent Neural Networks) computational architecture allows the RNN model to form multiple outputs after multiple computations (thinking) at time step t.
And the hidden state
, the output of the RNN at time t and the hidden state passed to the next time t+1 is obtained through the weighted coefficients accumulation.
This computational architecture simulates the thinking process by performing multiple computations at time t to obtain the weighted coefficients accumulation. This is particularly significant for using computational units like DNC with external memory to handle ultra-long sequences. This means that for a DNC with a massive external memory (such as 1K lines of memory), the DNC computational unit can read external memory matrix information multiple times based on external input information Xt to weigh and think before making a decision.
The original DNC model generally uses a 500-row external memory matrix, where at each time step t, it interacts with the external memory matrix through 1-3 write heads and 2-6 read heads. This means that the DNC model typically interacts with no more than 10 memory locations at time t. This works well for simple problems, such as memory reproduction, but performs poorly for those decision-making problems that involve complex interactions with previous information and have long time spans. By embedding the DNC computational unit into the ACT computational architecture, the PonderDNC computational unit can interact with the external memory matrix multiple times at each time step. For example, a PonderDNC with 2 write heads, 4 read heads, and 1000 rows of external memory can perform 50 computations at time t and interact with up to 300 memory locations in the external memory matrix, allowing it to make inferences based on 50 short-term memories and 200 read head memories at time t. This means that at time t, the PonderDNC computational unit can make inferences based on 20% of the total previous memory.
In simpler terms, if we consider these models as different traders, the LSTM trader makes judgments about future market trends based on the past two weeks of K-line charts, while the PonderDNC trader makes determinations based on various technical analysis indicators from the past quarter.
This PonderDNC model is suitable for handling long sequence tasks and complex backtracking tasks, such as processing high-frequency data in futures. The model requires a relatively large number of trainable weight parameters to achieve better results. When using it, be sure to increase the training data volume and minimize multicollinearity among input factors.
Note: The information passed from PonderDNC at each time step t to the next time step t+1 includes the accumulated controller state, the accumulated read head memory vector, and the external memory matrix after N(t) computations at time t (this is not the weight accumulation but the memory matrix directly passed after the last computation).
Updated on 2017-08-18 for futures daily frequency backtesting
In-sample backtesting
Out-of-sample backtesting from 2017-01-01 to 2018-08-01 shows decent generalization ability.
Read the original text to see the complete code