[Huatai Metalworking Lin Xiaoming Team] Inevitable by accident: Resampling technology to check overfitting-Huatai artificial intelligence series 19

movie 1079℃

Abstract

Bootstrap is a viable re-sampling method to build a "parallel A-share market"

Bootstrap is a feasible re-sampling method to build a "parallel A-share market", which can simulate the randomness of different links of machine learning, so as to verify the real A Whether the research conclusion drawn in the stock market is overfitting. We carried out Bootstrap resampling on the in-sample data, out-of-sample data, and backtest time respectively, and found that the model performance and single-factor backtest indicators of the grouped time series cross-validation method in the "parallel A-share market" are better than the other two methods. The statistical test result is significant. Real-world research conclusions can be reproduced in the parallel world, indicating that the conclusion is less likely to be over-fitting. We use the tools of "accidental" to explore the laws of "inevitability". The core idea of ​​

Bootstrap resampling is sampling with replacement.

Bootstrap is a statistical resampling method, also known as the bootstrap method, which is mainly used to study the statistical characteristics of statistics. The core idea of ​​this method is sampling with replacement. The original data set is sampled with replacement to obtain N sets of Bootstrap data sets. In each Bootstrap data set, some samples may be drawn repeatedly, and some samples may not be drawn. Calculate the statistics of each set of Bootstrap data sets, and obtain the distribution of the statistics of the N sets of Bootstrap data sets, and then obtain the statistics of the statistics. The

Bootstrap resampling method has guiding significance for the construction of machine learning quantitative research system

Bootstrap resampling has guiding significance for the construction of machine learning quantitative research system. The important difference between the development of machine learning quantitative strategy and the development of traditional quantitative strategy is that the complexity of machine learning research, the links involved, the number of hyperparameters, and the number of parameters far exceed those of traditional quantitative research. The introduction of randomness in any link will affect the final whole system. Both may cause effects similar to the butterfly effect. This paper uses Bootstrap to simulate the randomness of different links, and systematically evaluate the direction and degree of influence of randomness on the results of machine learning.

The randomness of different links of machine learning has different effects on model performance.

The three Bootstrap resampling schemes have different directions and degrees of influence on the same set of cross-validation methods. The in-sample data set of Bootstrap is equivalent to adding a small disturbance to the factor value of the training set, which may slightly weaken the performance of the model; the data set outside the Bootstrap sample is equivalent to adding a small disturbance to the factor value of the test set, which may partially enhance or weaken the model performance; Bootstrap returns The test time is to change the backtesting period of the model, which may greatly enhance or weaken the performance of the model. The enlightenment of the above results to researchers is that they need to pay close attention to the quality of training data during the development process, and at the same time avoid misjudgments caused by improper selection of the backtest time.

Bootstrap provides an idea of ​​describing randomness, enabling researchers to make decisions based on the distribution of indicators.

In the past quantitative model development process, historical backtest performance is usually regarded as a deterministic result, while the influence of randomness on the backtest results is ignored . When faced with the choice of different quantitative strategies, they are often simply based on evaluation indicators such as the annualized rate of return, the Sharpe ratio, and the rate of return drawdown based on the strategy. The Bootstrap resampling method provides a way of describing randomness, enabling researchers to make relatively objective judgments and decisions based on the statistical distribution of evaluation indicators, rather than individual statistics. From the perspective of methodology, this article reflects on the multi-factor stock selection framework combined with machine learning, and proposes innovative transformations for the model comparison link and model evaluation link, hoping to inspire investors in this field.

Risk reminder: The artificial intelligence stock selection method is an excavation of historical investment laws. If the future market investment environment changes, this method may fail. There are various sources of randomness in the machine learning stock selection model. This study only considers a limited number of three cases, and it is possible to ignore other more important sources of randomness. The Bootstrap resampling method is a simple simulation of randomness and may be oversimplified.

Introduction to this research

Almost the only thing that can be determined in the world is uncertainty. In the field of natural sciences, as small as the height of particles in the microcosmRapid movement, as large as the long evolution of creatures in the macroscopic world, is not at the mercy of the invisible hands of randomness. In the field of social sciences, countless individuals who follow randomness form a complex giant network through random connections and interactions, which derive a variety of colorful social phenomena. Generations of human beings have exhausted their minds and tried to understand and understand the world through the fog of randomness.

After hundreds of years of development, modern science has been recognized as one of the most powerful weapons for understanding the world. The essential feature of science is falsification. The proposition "all swans are white" itself is wrong, but the statement of the proposition is scientific, because the proposition can be falsified by "find out a black swan". However, due to the existence of randomness, most scientific propositions cannot be falsified simply by finding counterexamples. For example, it is not easy to falsify that "smokers live longer." The mechanism of smoking's health impact is too complicated. This proposition is only a vague and probabilistic statement. Simply finding a smoker with a shorter life span is not very convincing. People need to use statistical tools to get a glimpse of the "inevitable" truth through the cloak of "accidental".

The study of randomness statistics is one of the cornerstones of modern science. How does statistics solve the above problems? First define "smokers to live longer" as the null hypothesis, and "smokers to live longer" as the alternative hypothesis. Secondly, random sampling of smokers and non-smokers, the sample size as large as possible, and try to control other factors that affect life span. Then qualitatively compare and quantitatively test the life distribution of the two groups of people, and calculate the probability (p value) that the null hypothesis is established. Finally, reject or accept the null hypothesis at a certain level of significance, and make corresponding inferences. The above examples are only relatively simplified representations. In fact, more accurate judgments can be made through more sophisticated designs and more complex statistical models. But no matter how complicated the model is, the essence is still based on a large number of samples combined with statistical models to infer the probability of the null hypothesis. Although the use of p-value has been questioned in academia in recent years, the above-mentioned idea of ​​comparing the distribution of random variables and conducting statistical tests is still the most powerful tool that people find inevitable by accident. What is a little surprising is that in the field of quantitative investment, researchers seem to have forgotten the existence of randomness in the search for high-quality quantitative strategies and regard historical backtesting performance as a deterministic result. When faced with the choice of different quantitative strategies, they are often simply based on evaluation indicators such as the annualized rate of return, the Sharpe ratio, and the rate of return drawdown based on the strategy. For example, if the Sharpe ratio of strategy A is higher than strategy B, we abandon strategy B and choose strategy A. However, the financial market is an open and complex giant system. It only requires a little manipulation with random hands (such as slight changes in the data set inside and outside the sample), and the backtest results may be quite different. All our observations and conclusions are based on deterministic history and are directed to the real world we live in. If these conclusions are not valid in a "parallel world" that is similar to the real world, then we have reason to believe that these conclusions are just overfitting to the real world. What is more worrying about

is that the above problems will be amplified in the field of artificial intelligence quantitative research. The important difference between the development of machine learning quantitative strategy and the development of traditional quantitative strategy is that the complexity of machine learning research, the links involved, the number of hyperparameters, and the number of parameters far exceed those of traditional quantitative research. The introduction of randomness in any link will affect the final whole system. Both may cause effects similar to the butterfly effect. However, the degree of influence of randomness in different links on the results has not been systematically studied. For example, all the stock selection factor values ​​we observe are the superposition of signal and noise. If the training set factors are slightly disturbed, will it greatly affect the training results of the machine learning model? If the test set factor is slightly disturbed, will it greatly affect the backtest performance of the machine learning strategy? If the answers to the above questions are all "yes", then we also have reason to believe that the machine learning model has a greater risk of overfitting. Can

create a parallel world similar to the real world? Can the conclusions in the real world be verified in the parallel world? Can you learn from the idea of ​​statistical testing to measure the probability of overfitting in a quantitative strategy developed based on the real world? As the first exploration of the above-mentioned problems, this article uses the basic resampling method-Bootstrap to resample the monthly cross-sectional data of multi-factor stock selection to create several sets of "parallel A-share markets" similar to the real A-share market. And based on Huatai Metalworking's "Artificial Intelligence 16: Revisiting Time SequenceThe three sets of machine learning strategies in "Cross Verification Against Overfitting" are the objects of investigation, using qualitative comparison and quantitative statistical testing methods to examine the performance of each strategy in the parallel A-share market, and finally assessing the optimal strategy obtained by this research as over-simulation The possibility of cooperation. From a methodological perspective, this article reflects on the multi-factor stock selection framework combined with machine learning, and proposes innovative transformations for model comparison and model evaluation, hoping to inspire investors in this field.

uses Bootstrap re-sampling to construct a "parallel world"

The problem is raised: the dilemma of back-testing over-fitting

In the process of quantitative strategy development, when choosing between multiple sets of strategies, most of them are based on the performance of the back-test phase. For example, if the Sharpe ratio of strategy A is 2, the Sharpe ratio of strategy B is 1.5, and the Sharpe ratio of strategy C is 1, then we generally think that strategy A is better than strategies B and C. However, the good performance in the backtesting stage may be due to some accidental factors, which does not mean that the strategy correctly captures the laws of the market. The optimal strategy in the backtesting stage may perform flat in the real stage, as shown in the figure below. This phenomenon is called backtesting overfitting. Whether the difference in backtest performance of

[Huatai Metalworking Lin Xiaoming Team] Inevitable by accident: Resampling technology to check overfitting-Huatai artificial intelligence series 19 - Lujuba

is due to the difference in real effects or from some accidental factors, it is difficult for us to make precise attributions. It is not for nothing, but it is impossible for it. There is no "parallel world" in the real financial market, and it is impossible to confirm the authenticity of research conclusions through statistical tests through multiple measurements. The

Bootstrap method provides us with a window that can simulate a "parallel world" of numerous financial market data based on a single real financial market data. Assuming that 10,000 A-share data sets are simulated, then we can obtain the Sharpe ratios of 10,000 A, B, and C three groups of strategies, and perform statistical tests based on the distribution of the three, and then infer that "strategy A is better than strategy B, C" The conclusion of the study is whether it is because the model correctly captures the laws in the market or is it because the model is caught in backtesting and overfitting.

Bootstrap resampling method

Bootstrap is a statistical resampling method, also known as the bootstrap method, which is mainly used to study the statistical characteristics of statistics to test the stability of statistical results. The basic idea is shown in the figure below. The statistics of the original data set (such as the mean) can be easily obtained, so how to calculate the statistics of the statistics (such as the standard deviation of the mean)? The core idea of ​​the Bootstrap method is sampling with replacement. The original data set is sampled with replacement to obtain N sets of Bootstrap data sets, and N usually needs to be greater than 1000. Calculate the statistic (such as the mean value) of each group of Bootstrap data sets, and the distribution of the statistic of the N sets of Bootstrap data sets will be obtained, and then the statistic (such as the standard deviation) of the statistic will be obtained.

[Huatai Metalworking Lin Xiaoming Team] Inevitable by accident: Resampling technology to check overfitting-Huatai artificial intelligence series 19 - Lujuba

will be described with specific examples below. We would like to study the average rise and fall of A-share non-suspended stocks on April 1, 2019. Data set D is the rise and fall of 3,569 A-share non-suspended stocks on April 1, 2019. The average value is x_p=3.6132%. So how should the mean and standard deviation of the mean be calculated?

[Huatai Metalworking Lin Xiaoming Team] Inevitable by accident: Resampling technology to check overfitting-Huatai artificial intelligence series 19 - Lujuba

first sample the data set with replacement, randomly select a sample from the data set, then put it back, then take another sample, and then put it back,... Repeat this 3569 times to get a new stock containing 3569 stocks Data set D1. Note that some stocks in the original data set D may be repeatedly drawn, and some stocks may not be drawn. We call the new data set D1 a set of Bootstrap data sets, and get x_D1 by averaging. Repeating the above steps, we can get N sets of Bootstrap data sets D2, D3,..., DN, and the mean value of each set of data sets. Assuming the number of re-sampling, the average value of these 10,000 average fluctuations can be calculated:

[Huatai Metalworking Lin Xiaoming Team] Inevitable by accident: Resampling technology to check overfitting-Huatai artificial intelligence series 19 - Lujuba

Standard deviation:

[Huatai Metalworking Lin Xiaoming Team] Inevitable by accident: Resampling technology to check overfitting-Huatai artificial intelligence series 19 - Lujuba

With the Bootstrap method, we derived N new Bootstrap data sets from an original data set. Some readers of

may question: After Bootstrap resampling the original data set, some stocks in the obtained Bootstrap data set may be repeatedly drawn, and some stocks are not drawn, so calculate the average value for such a "fake data set" What's the point? In fact, a single BootstraThe statistics of the p data set do not have much meaning, but the distribution of the statistics of many Bootstrap data sets provides effective incremental information. Take a popular example to illustrate that a grain of sand may be insignificant. We are more concerned about the characteristics of the tower after the sand has been gathered.

The relationship between Bootstrap and machine learning

As a statistical method, what is the relationship between Bootstrap and machine learning? We try to answer from two levels.

First of all, from the "technical" level, Bootstrap can optimize machine learning algorithms. Bootstrap is widely used in the integration of weak learners, and this integration method is also called Bagging. Random forest is one of the classic integrated learning models based on Bagging idea, which integrates a large number of decision trees in parallel. When training each decision tree, first row sampling and column sampling are performed on the original data set. Column sampling means that only a part of the randomly selected features is selected during each step of splitting. Row sampling means that the original sample is replaced. Sampling in order to obtain a Bootstrap data set with the same sample size as the original data set. Finally, a large number of decision trees are integrated through the principle that the minority obeys the majority to realize the prediction of the random forest. The training of each tree here is only based on the incomplete Bootstrap data set, and the learning ability is relatively weak, and its judgment may be insignificant, but after the trees become forests, the random forest has a stronger learning generalization ability.

Secondly, from the "Tao" level, Bootstrap has guiding significance for the construction of a quantitative research system for machine learning. The important difference between the development of machine learning quantitative strategy and the development of traditional quantitative strategy is that the complexity of machine learning research, the links involved, the number of hyperparameters, and the number of parameters far exceed those of traditional quantitative research. The introduction of randomness in any link will affect the final whole system. Both may cause effects similar to the butterfly effect. However, the degree of influence of randomness in different links on the results has not been systematically studied. For example, all the stock selection factor values ​​we observe are the superposition of signal and noise. If the training set factors are slightly disturbed, will it greatly affect the training results of the machine learning model? If the test set factor is slightly disturbed, will it greatly affect the backtest performance of the machine learning strategy? We need to use tools to evaluate the impact of the randomness of different links on the results. This article will use the Bootstrap method to simulate these randomness, build a series of "parallel A-share markets", examine the performance of machine learning quantitative models in these parallel worlds, compare and choose different strategies, and test whether the strategies fall into backtesting and overfitting. . In the following chapters, we will introduce in detail the method of constructing a parallel A-share market.

Constructing a "parallel A-share market"

The source of randomness in the machine learning stock selection model is more diverse. Let's list three possible sources, as shown in the figure below.

1. The stock selection factor value we observed is the superposition of signal and noise. Assuming that the factor value of an individual stock is a random variable that obeys a certain distribution, and the observed value in the real world is a sampling of this random variable, then in some other "parallel A-share markets", whether it is in-sample or out-of-sample data sets, The same factor of the same stock may take different values. The random noise of the data set factor value in the sample may affect the model training process. We don't want the random perturbation of factors to make the trained machine learning model greatly change.

2. Similar to point 1, the random noise of the factor values ​​of the data set outside the sample may affect the model backtesting. We don't want the random disturbance of the factors to drastically change the backtest performance of the machine learning strategy.

3. The backtest performance of the model is closely related to the choice of backtest time. Whether the backtest interval is dominated by a bull market or a bear market, whether a small market value style or a value style dominates, the performance will vary greatly. We hope that the strategy can travel through different market environments, and we don't want the choice of backtest time to have a significant impact on the results.

[Huatai Metalworking Lin Xiaoming Team] Inevitable by accident: Resampling technology to check overfitting-Huatai artificial intelligence series 19 - Lujuba

Aiming at the above three sources of randomness, we tried the following three Bootstrap resampling schemes to build a "parallel A-share market".

1. Aiming at the randomness of the data set in the sample, we perform Bootstrap resampling on the data set in the sample, as shown in the figure below. For the cross-sectional period at the end of each month in the sample, assuming that the number of effective stocks in the all A stock pool (non-suspension, non-new stocks, non-daily limit at the beginning of next month) is 3600, thenWe sample the full A stock pool with replacement, repeat 3600 times, and obtain a Bootstrap stock pool containing 3600 stocks and its corresponding cross-sectional period factor data. Compared with the real all-A stock pool, some stocks in the Bootstrap stock pool may be drawn repeatedly, and some stocks may not be drawn. In the subsequent machine learning training session, the characteristics of the cross-sectional period at the end of the month are the factor values ​​of these 3,600 "new" stocks. It should be noted that the number of resampling of Bootstrap should usually be greater than 1000. However, considering the time cost of the research, this article sets the number of resampling N to 100. We found that N=100 has been able to observe the difference between the distributions, and obtained results with certain statistical power. For details, please refer to the results section of this article.

[Huatai Metalworking Lin Xiaoming Team] Inevitable by accident: Resampling technology to check overfitting-Huatai artificial intelligence series 19 - Lujuba

2. Aiming at the randomness of the out-of-sample data set, we perform Bootstrap resampling on the out-of-sample data set, as shown in the lower left figure. For the cross-sectional period at the end of each month outside the sample, sample the all A stock pool with replacement to obtain the Bootstrap stock pool and the corresponding cross-sectional period factor data. In the subsequent backtest session, the characteristics of the cross-sectional period at the end of the month are the factor values ​​of these "new" stocks.

[Huatai Metalworking Lin Xiaoming Team] Inevitable by accident: Resampling technology to check overfitting-Huatai artificial intelligence series 19 - Lujuba

3. In view of the randomness of the backtest time, we perform Bootstrap resampling on the backtest time. The specific process is shown in the upper right figure. Assuming that the original out-of-sample data set contains the back-test month corresponding to the cross-sectional period of 96 months from January 2011 to December 2018 (the back-test month corresponding to the last cross-sectional period of December 2018 is January 2019), We sample the 96 back-test months with replacement, and repeat it 96 times to obtain a Bootstrap back-test time containing the new 96 back-test months. Compared to the original 96 back-test months, some months in the Bootstrap back-test time may be repeatedly drawn, and some months may not be drawn. We then conducted a backtest of the machine learning model based on these 96 "new" backtest months. It should be pointed out that the 96 new backtest months do not have a strict time series relationship, that is, the months that are earlier in the Bootstrap backtest period are not necessarily higher in the real market.

method

Observation object: three sets of cross-validation tuning methods

We hope to use Bootstrap method to simulate the randomness in the financial market, build a series of "parallel A-share markets", and examine the machine learning quantitative stock selection model in these parallel worlds Performance, compare and select different strategies, and check whether the strategies fall into backtesting over-fitting. Specifically, the object of this research is the three sets of cross-validation tuning methods in Huatai Metalworking "Artificial Intelligence 16: Revisiting Time Series Cross-Validation Against Overfitting" (20190218), namely: K-fold cross-validation and disorder Group progressive cross validation, group timing cross validation. The three sets of parameter adjustment methods use the same base learner, all of which are XGBoost models, and the model construction details are also the same. The only difference is that the hyperparameters finally selected by the parameter adjustment are different. The research conclusion of "Artificial Intelligence 16: Revisiting Time Series Cross-Validation Against Overfitting" is that the artificial intelligence stock selection strategy obtained by based on the group time series cross-check parameter tuning method outperforms the other two methods . This study will test whether the above conclusions have backtesting overfitting.

The following briefly introduces the conclusion to be tested. For the three sets of cross-validation methods, we calculate model performance (such as AUC) and single factor test performance (such as Rank IC) monthly. In order to highlight the difference between the methods, subtract the K-fold AUC or Rank IC from the AUC or Rank IC of the disordered grouping progressive and grouping sequence, and then accumulate the difference. If the cumulative value of the difference rises steadily, then it can be considered that the method is more stable than the K-fold; if the cumulative value of the difference is maintained on the 0 axis, then it can be considered that there is no difference between the method and the K-fold. The results in the figure below show that the grouping sequence performance is better than the out-of-order grouping progressive type, indicating that the improvement brought by the timing cross-validation comes from the retention of timing information; at the same time, the out-of-order grouping progressive type is better than the K-fold, indicating that fewer samples Can partially improve the performance of the model.

[Huatai Metalworking Lin Xiaoming Team] Inevitable by accident: Resampling technology to check overfitting-Huatai artificial intelligence series 19 - Lujuba

The concept of the three cross-validation tuning methods will not be repeated in this article. For interested readers, please refer to Huatai Metalworking "Artificial Intelligence 14: Fighting Overfitting: From Time Series Cross-Validation" (20181128) and "Artificial Intelligence 16" : Revisiting temporal cross-validation against over-simulationTogether" (20190218). Simply put:

1. K-fold cross-validation is a classic cross-validation method, which is suitable for independent and identically distributed non-time series data. Time series data applied to the financial field has the risk of overfitting, which is the baseline model of this article.

2. The out-of-order grouping progressive cross-validation uses fewer samples than K-fold, but destroys the timing information, which is also the baseline model of this article.

3. Grouped time series cross-validation is an improvement proposed for time series data, which can reduce overfitting to a certain extent, and is the recommended model for this article.

[Huatai Metalworking Lin Xiaoming Team] Inevitable by accident: Resampling technology to check overfitting-Huatai artificial intelligence series 19 - Lujuba[Huatai Metalworking Lin Xiaoming Team] Inevitable by accident: Resampling technology to check overfitting-Huatai artificial intelligence series 19 - Lujuba

Artificial intelligence stock selection model test process

[Huatai Metalworking Lin Xiaoming Team] Inevitable by accident: Resampling technology to check overfitting-Huatai artificial intelligence series 19 - Lujuba

In this paper, XGBoost is selected as the base learner. The test process includes the following steps:

1. Data acquisition:

a) Stock pool: all A shares. Excluding ST stocks, excluding stocks suspended from trading on the next trading day of each cross-sectional period, and excluding stocks that have been listed within 3 months, each stock is treated as a sample.

b) Backtest interval: January 31, 2011 to January 31, 2019.

2. Feature and label extraction: On the last trading day of each natural month, calculate the exposure of the 70 factors in the previous report as the original features of the sample. The factor pool is shown in the following table. Calculate the excess return of individual stocks for the entire next natural month (based on the Shanghai and Shenzhen 300 Index). At the end of each month, select the top 30% of the stocks in the next month’s return as a positive example (y = 1), and the last 30% The stock is taken as a negative example (y = 0), as the label of the sample.

[Huatai Metalworking Lin Xiaoming Team] Inevitable by accident: Resampling technology to check overfitting-Huatai artificial intelligence series 19 - Lujuba

3. Feature preprocessing:

a) Median to extreme value: Let the exposure sequence of a factor in all stocks in period T be D_i, D_M is the median of the sequence, and D_M 1 is the median of the sequence |D_i -D_M| Reset all numbers in the sequence D_i that are greater than D_M+5D_M1 to D_M+5D_M1, and reset all the numbers in the sequence D_i that are less than D_M-5D_M1 to D_M-5D_M1;

b) Missing value treatment: get new factor exposure After the degree sequence, set the missing factor exposure to the average value of the same stocks in the CITIC Tier 1 industry;

c) Neutralize the industry market value: The factor exposure after filling the missing values ​​is compared to the industry dummy variable and the logarithm Perform linear regression on the market value, and take the residual as the new factor exposure;

d) Standardization: subtract the current mean value from the neutralized factor exposure sequence and divide by its standard deviation to obtain a new approximation subject to N (0, 1) Distribution sequence.

4. The division of rolling training set and validation set: This article adopts the annual rolling training method, and the internal and external data of all samples are divided into eight stages, as shown in the figure below. For example, when forecasting 2011, the data of 72 months from 2005 to 2010 are combined as the in-sample data set; when forecasting year T, the 72 months from T-6 to T-1 are combined as in-sample data. The training set and the validation set are divided according to different cross-validation methods, and the fold number of cross-validation is 12. For group time series cross-validation, the length of each training set is an integer multiple of 6 months, and the length of the validation set is equal to 6 months. For K-fold cross-validation, the number of verifications is 12; for out-of-order grouping progressive and grouping sequential cross-validation, the number of verifications is 11. For any cross-validation method involving scrambled data, the random number seed points are the same, thereby ensuring the same scrambled method.

[Huatai Metalworking Lin Xiaoming Team] Inevitable by accident: Resampling technology to check overfitting-Huatai artificial intelligence series 19 - Lujuba

5. Cross-validation tuning: Grid search for all hyper-parameter combinations; for each set of hyper-parameter combinations, use the XGBoost-based learner to train the training set for each verification, and record the performance of the model in the verification set; select all the verification sets The group of hyperparameters with the highest average AUC is regarded as the optimal hyperparameter of the model. Different cross-validation methods may get different optimal hyperparameters. The final optimal hyperparameter settings are shown in the table below.

[Huatai Metalworking Lin Xiaoming Team] Inevitable by accident: Resampling technology to check overfitting-Huatai artificial intelligence series 19 - Lujuba

6. In-sample training: Use the XGBoost-based learner under the optimal hyperparameter settings to train the complete original in-sample data set. When Bootstrap scheme 1 is used, 100 sets of Bootstrap sample data sets are trained separately to obtain 100 sets of XGBoost models.

7. Out-of-sample test: After the model training is completed, the preprocessed features of all samples in the cross-sectional period at the end of month T are used as the input of the model.Get the predicted value of each sample. Regarding the predicted value as a synthesized factor, the regression method, IC analysis method and hierarchical back-testing method are used for single factor testing. When Bootstrap scheme 2 is adopted, 100 sets of Bootstrap out-of-sample data sets are predicted and back-tested. When using Bootstrap solution 3, the original out-of-sample data set is reorganized and back-tested according to 100 sets of Bootstrap back-testing time.

8. Model evaluation:

a) (for Bootstrap scheme 1) the distribution of the accuracy of 100 test sets, AUC and other indicators to measure model performance;

b) (for Bootstrap schemes 1, 2 and 3) Statistical indicators and backtests obtained from single factor testing The distribution of performance.

single factor test

regression method and IC value analysis method

test model construction method is as follows:

1. Stock pool: all A shares, excluding ST stocks, excluding stocks suspended from trading on the next trading day of each cross-section period, excluding 3 months of listing Stocks within. When Bootstrap scheme 2 is adopted, the stock pool is an out-of-sample data set of Bootstrap, and it must meet the above-mentioned screening conditions. When Bootstrap scheme 3 is adopted, the stock pool is the stocks that correspond to the cross-sectional period at the end of the month determined by the Bootstrap backtest time and that also meet the above screening conditions.

2. Backtest interval: 2011-01-31 to 2019-01-31.

3. Cross-sectional period: At the end of each month, use the current cross-sectional period factor value and the return of individual stocks from the current cross-sectional period to the next cross-sectional period to regress, and calculate its Rank IC value.

4. Data processing method: For the classification model, the model's predicted value of the stock's next rise probability is regarded as a single factor. For regression models, the regression prediction value is treated as a single factor. Stocks with empty factor values ​​do not participate in the test.

5. Weighted least squares regression (WLS) is used in the regression test, and the square root of the market value of individual stocks is used as the weight. The industry market value is neutral for single factor during IC test. The

hierarchical backtesting method

scores stocks according to the factor value and constructs the portfolio backtest, which is the most intuitive way to measure the pros and cons of factors. The test model construction method is as follows:

1. The stock pool, back-test interval, and cross-sectional period are all the same as the regression method.

2. Swap: The factor value is calculated on the last trading day of each natural month, and the position is exchanged at the closing price of the day on the first trading day of the next natural month. The transaction fee is calculated at four thousandths of each side.

3. Hierarchical method: Factors first use the median method to remove extreme values, and then market value and industry neutralization (see the previous section for the methodology), and sort all stocks in the stock pool by factor from largest to smallest, etc. It is divided into N layers, and the individual stocks within each layer are allocated equal weight. When the total number of individual stocks cannot be divisible by N, any approximation method can be used. In fact, it has little effect on the backtest results of the stratified combination.

4. Long-short portfolio income calculation method: subtract the daily income of the Bottom group from the daily income of the Top group to get the daily long-short income sequence r_1,r_2,...,r_n, then the net value of the long-short combination on the nth day Equal to (1+r_1)(1+r_2)...(1+r_n).

5. Evaluation method: annual return rate of all N-tier portfolios (observe whether it changes monotonously), annual return rate of long-short portfolio, Sharpe ratio, maximum drawdown, etc.

Results

Scheme 1: Bootstrap resampling the in-sample data set

First, we show the results of Bootstrap resampling the in-sample data set. For each resampling, we build a "parallel A-share market" based on the real factor cross-sectional data; use the factor cross-sectional data in the parallel world as the in-sample data set, and use the optimal hyperparameters obtained by different cross-validation methods as the model Hyperparameters, train the XGBoost model; after the model training is completed, use the factor cross-section data of the real A-share market as the out-of-sample data set to perform predictions and single-factor backtests to calculate model performance and various backtest indicators. Finally, the model performance and single-factor backtest indicators of 100 re-sampling were counted, the distribution was drawn, and the indicator distributions of the three sets of cross-validation methods were compared. The results of the data set in the Bootstrap sampleAnswered the following question: Does the small change in the data in the sample affect the conclusion of the machine learning research ? The

model performance

three-group cross-validation method within the sample correct rate and the distribution of AUC as shown below. The distribution of the two indicators is relatively dense. For example, the accuracy of all grouped time series cross-validation samples is concentrated in the range of 61% to 62%, and the AUC in the sample is concentrated in the range of 0.66 to 0.67. From the in-sample performance of the three-group cross-validation method, K-fold is better than the random grouping progressive type, and the random grouping progressive type is better than the grouping sequence. The distribution of out-of-sample accuracy and AUC of the three-group cross-validation method

[Huatai Metalworking Lin Xiaoming Team] Inevitable by accident: Resampling technology to check overfitting-Huatai artificial intelligence series 19 - Lujuba

is shown in the figure below. Unlike the in-sample performance, the out-of-sample performance of the three-group cross-validation method has been reversed. The grouping sequence is far better than the out-of-order grouping progressive type, and the out-of-order grouping progressive type is better than K-fold. The research conclusions drawn in the parallel world constructed by the data set in the Bootstrap sample are consistent with the research conclusions drawn in the real world (see Figures 8 and 9 for the real world conclusions, the same below).

[Huatai Metalworking Lin Xiaoming Team] Inevitable by accident: Resampling technology to check overfitting-Huatai artificial intelligence series 19 - Lujuba

regression method and IC value analysis method

three-group cross-validation method Regression method and IC value analysis method The distribution of various indicators is as follows. The group time series cross-validated |t| mean, t mean, factor return mean, and Rank IC mean are far superior to the disordered progressive and K-fold. The research conclusions drawn in the parallel world constructed by the dataset in the Bootstrap sample are consistent with the research conclusions drawn in the real world.

[Huatai Metalworking Lin Xiaoming Team] Inevitable by accident: Resampling technology to check overfitting-Huatai artificial intelligence series 19 - Lujuba[Huatai Metalworking Lin Xiaoming Team] Inevitable by accident: Resampling technology to check overfitting-Huatai artificial intelligence series 19 - Lujuba

hierarchical back-testing method

three-group cross-validation method The distribution of each index of the hierarchical back-testing method is as follows. The group time series cross-validation of the long-short portfolio annualized return, the top portfolio annualized return, and the top portfolio Sharpe ratio are better than the disordered progressive and K-fold; for the long-short portfolio Sharpe ratio, the group time series cross-validation is weak Advantage. The research conclusions drawn in the parallel world constructed by the dataset in the Bootstrap sample are basically the same as those drawn in the real world.

[Huatai Metalworking Lin Xiaoming Team] Inevitable by accident: Resampling technology to check overfitting-Huatai artificial intelligence series 19 - Lujuba[Huatai Metalworking Lin Xiaoming Team] Inevitable by accident: Resampling technology to check overfitting-Huatai artificial intelligence series 19 - Lujuba

The above figure shows the net value of the two long-short portfolios with the highest returns and the lowest returns under 100 Bootstrap re-sampling of the three sets of cross-validation methods, and the net value of the long-short portfolios in the real A-share market. First of all, you can still observe the sorting relationship of grouping sequence, disordered grouping progressive, and K-fold income from high to low. Secondly, we found that the true net worth (solid line) is close to the best net worth (dotted line), and there is a big difference from the worst net worth (dashed line). In other words, the parallel-world training models in most Bootstrap samples do not perform as well as the real-world training models. This shows that a small disturbance of the data set in the sample may weaken the performance of the model in most cases.

Scheme 2: Bootstrap resampling the out-of-sample data set

The following shows the results of Bootstrap resampling the out-of-sample data set. For each re-sampling, we build a "parallel A-share market" based on the real factor cross-sectional data; use the real-world factor cross-sectional data as the in-sample data set, and use the optimal hyperparameters obtained by different cross-validation methods as the model Hyperparameters, train the XGBoost model; after the model training is completed, take the factor cross-section data of the parallel A-share market as the out-of-sample data set, perform a single-factor backtest, and calculate various backtest indicators. Finally, count the single-factor back-test indicators of 100 resamples, draw the distribution, and compare the indicator distributions of the three sets of cross-validation methods. The results of the Bootstrap out-of-sample data set answer the following question: Does the small change in the out-of-sample data of affect the machine learning research conclusions?

regression method and IC value analysis method

three-group cross-validation method Regression method and IC value analysis method The distribution of various indicators is as follows. The group time series cross-validated |t| mean, t mean, factor return mean, and Rank IC mean are far superior to the disordered progressive and K-fold. The research conclusions drawn in the parallel world constructed by the Bootstrap out-of-sample data set are consistent with the research conclusions drawn in the real world and the conclusions drawn in the Bootstrap sample data set.

[Huatai Metalworking Lin Xiaoming Team] Inevitable by accident: Resampling technology to check overfitting-Huatai artificial intelligence series 19 - Lujuba

hierarchical back-testing

three-group cross-validation method The distribution of each index of the hierarchical back-testing method is as follows. Long-short portfolio annualized return based on group time series cross-validationThe rate is better than the disordered progressive and K-fold; for the long-short portfolio Sharpe ratio, Top portfolio annualized return rate, Top portfolio Sharpe ratio, group time series cross-validation has a weak advantage. The research conclusions obtained in the parallel world constructed by the Bootstrap sample data set are basically consistent with the research conclusions obtained in the real world, and the conclusions drawn from the Bootstrap sample data set are basically the same.

[Huatai Metalworking Lin Xiaoming Team] Inevitable by accident: Resampling technology to check overfitting-Huatai artificial intelligence series 19 - Lujuba

The following figure shows the net worth of the two long-short portfolios with the highest and lowest returns under 100 Bootstrap re-sampling of three sets of cross-validation methods and the net worth of the long-short portfolios in the real A-share market. First of all, it can still be observed that the grouping sequence is better than the other two methods, but there is no significant difference between the out-of-order grouping progressive and K-fold. Second, we found that the true net worth (solid line) is located in the middle of the best net worth (dotted line) and the worst net worth (dashed line). This shows that the small disturbance of the data set outside the sample has a neutral effect on the performance of the model.

[Huatai Metalworking Lin Xiaoming Team] Inevitable by accident: Resampling technology to check overfitting-Huatai artificial intelligence series 19 - Lujuba

Scheme 3: Bootstrap resampling the backtest time

Below we show the results of Bootstrap resampling the backtest time. For each re-sampling, we build a new time series containing 96 back-test months based on the original 96 back-test months; use the factor cross-sectional data in the real world as the in-sample data set, and use different cross-validation The optimal hyper-parameters obtained by the method are used as the hyper-parameters of the model to train the XGBoost model; after the model training is completed, the factor cross-sectional data of the real A-share market in the new back-test month sequence is used as the out-of-sample data set to perform single-factor back-testing. Calculate various backtest indicators. Finally, count the single-factor back-test indicators of 100 resamples, draw the distribution, and compare the indicator distributions of the three sets of cross-validation methods. The results of Bootstrap backtest time answer the following questions: Does the change of backtest time affect the conclusions of machine learning research?

regression method and IC value analysis method

three-group cross-validation method Regression method and IC value analysis method The distribution of various indicators is as follows. The group time series cross-validated |t| mean, t mean, factor return mean, and Rank IC mean are slightly better than the disordered progressive and K-fold, but the advantages are not as obvious as the first two Bootstrap schemes. The research conclusions drawn in the parallel world constructed by Bootstrap backtesting time are consistent with the research conclusions drawn in the real world.

[Huatai Metalworking Lin Xiaoming Team] Inevitable by accident: Resampling technology to check overfitting-Huatai artificial intelligence series 19 - Lujuba[Huatai Metalworking Lin Xiaoming Team] Inevitable by accident: Resampling technology to check overfitting-Huatai artificial intelligence series 19 - Lujuba

hierarchical back-testing method

three-group cross-validation method The distribution of each index of the hierarchical back-testing method is shown in the figure below. The long-short portfolio annualized return rate of grouping time series cross-validation is better than the out-of-order progressive and K-fold annualized return rate; for long-short portfolio Sharpe ratio, Top portfolio annualized return rate, Top portfolio Sharpe ratio, if only From the perspective of distribution, there is no advantage in group time series cross-validation. The research conclusions obtained in the parallel world constructed by Bootstrap backtesting time are partially consistent with the research conclusions obtained in the real world.

[Huatai Metalworking Lin Xiaoming Team] Inevitable by accident: Resampling technology to check overfitting-Huatai artificial intelligence series 19 - Lujuba

Horizontal comparison of the three Bootstrap resampling schemes

In addition to comparing the three sets of cross-validation methods under the same re-sampling scheme, you can also compare the index distributions obtained by the three re-sampling schemes under the same set of cross-validation methods . Let's take the group time sequence cross-validation as an example to show the horizontal comparison results. The horizontal comparison results answer the following question: What is the difference in the degree of influence of the randomness of the different links of machine learning stock selection on the research conclusion?

regression method and IC value analysis method

The distribution of various indicators of the regression method and IC value analysis method under the three resampling schemes is shown in the figure below. First of all, from the perspective of the width of the distribution, the distribution of the data set within the Bootstrap sample is narrow, that is, the degree of variation is low; the data set outside the Bootstrap sample is second; the distribution of Bootstrap backtest time is wider, that is, the degree of variation is large. This shows that small changes in the data set in the sample have little impact on the research conclusions; changes in the back-test time have a relatively large impact on the research conclusions, so in the development of quantitative strategies, it is necessary to avoid the choice of back-test time Misjudgment caused by impropriety.

Secondly, from the perspective of the location of the distribution, the data set in the Bootstrap sample is located to the left of the true value, and the data set outside the Bootstrap sample and BootstrapThe center position of the backtest time is close to the true value. This shows that small changes in the data set in the sample may weaken the performance of the model. Researchers need to pay close attention to the quality of the training data during the development process; and when the model is trained, the out-of-sample data set and changes in the backtest time affect the single factor test The overall influence of various statistical indicators is neutral.

[Huatai Metalworking Lin Xiaoming Team] Inevitable by accident: Resampling technology to check overfitting-Huatai artificial intelligence series 19 - Lujuba

hierarchical back-testing

The distribution of the annualized return and Sharpe ratio of the long-short portfolio under the three resampling schemes is shown in the figure below. First of all, from the perspective of the width of the distribution, the Bootstrap sample still has a low degree of influence on the results, and the Bootstrap sample is second. The Bootstrap backtest time has a higher degree of influence on the results. Secondly, from the perspective of the location of the distribution, the distribution of the Sharpe ratio of the long-short combination in the Bootstrap sample and outside the Bootstrap sample is on the left of the true value as a whole, indicating that small changes in the data set in or out of the sample may reduce the Sharpe ratio of the long-short combination . Compared with the results in the previous section, it can be seen that statistical indicators such as t-value, factor return rate, and Rank IC are relatively insensitive to changes in out-of-sample data sets, but indicators such as long-short portfolio Sharpe ratios that are closely related to transactions have an effect on out-of-sample data. Set changes are more sensitive.

[Huatai Metalworking Lin Xiaoming Team] Inevitable by accident: Resampling technology to check overfitting-Huatai artificial intelligence series 19 - Lujuba

The distribution of the top portfolio annualized return and Sharpe ratio of the hierarchical backtesting method under the three resampling schemes is shown in the figure below. The distribution width of Bootstrap backtest time is much larger than the other two Bootstrap solutions, indicating that the performance of the Top combination is more sensitive to the backtest time. The center position of the Top portfolio indicator distribution is on the right side of the true value. The reason may be that the monthly return of the Top portfolio is a positive skew distribution, and the long tail is in the positive return part on the right. When Bootstrap resampling, the high return month has a greater probability of being The draw improves the performance of the Top combination in the parallel world. The summary of the results of

[Huatai Metalworking Lin Xiaoming Team] Inevitable by accident: Resampling technology to check overfitting-Huatai artificial intelligence series 19 - Lujuba

and the quantitative characterization of the risk of back-testing over-fitting

summarizes the back-test performance of the three sets of cross-validation methods under the three Bootstrap schemes, and the results are shown in the following table. In general, the research conclusions drawn under the three Bootstrap schemes are consistent with the research conclusions drawn in the real A-share market, that is, the grouping sequence is relatively good, the out-of-order grouping is the next step, and the K-fold is relatively poor. The parallel world and the real world can confirm each other, indicating that the research conclusion based on real data that "group time series cross-validation is better than the other two methods" has a lower risk of over-fitting in backtesting.

[Huatai Metalworking Lin Xiaoming Team] Inevitable by accident: Resampling technology to check overfitting-Huatai artificial intelligence series 19 - Lujuba

Through the comparison of the different performance of the model in the parallel world and the real world, we are able to qualitatively describe the risk of backtesting and overfitting based on real data to draw research conclusions. Can the backtesting overfitting risk be quantitatively described? Let's try two simple methods.

1. Perform one-way Repeated Measures ANOVA (One-way Repeated Measures ANOVA) statistical test for the back-test index distribution of the three cross-validation methods under the same Bootstrap scheme. ANOVA's statistical indicators F value and P value are used to measure the difference of the three groups of distributions. The larger the F value or the smaller the corresponding P value, the greater the difference between the three groups of distributions. Then "group time series cross-validation is better than the other two The “group method” study concluded that the risk of over-fitting in the backtest may be smaller. Observing the following table shows that, with the exception of the long-short combination Sharpe ratio and the Top combination Sharpe ratio, most of the back-test indicators have significant differences between the three sets of cross-validation methods, which further validates the previous conclusions.

2. For each Bootstrap re-sampling, determine whether the backtest index of the grouping time sequence cross-validation is better than the other two groups. Count the probability of optimal performance of grouping time sequence cross-validation under all 100 resamples. The closer the probability value is to 1, the better the performance of grouped time series cross-validation in the parallel world, and the smaller the risk of overfitting based on real data. Observing the following table shows that the probability value of most indicators is equal to or close to 1, which also confirms the previous conclusion.

[Huatai Metalworking Lin Xiaoming Team] Inevitable by accident: Resampling technology to check overfitting-Huatai artificial intelligence series 19 - Lujuba

summary

Through the comparison of three sets of cross-validation methods under each Bootstrap scheme and the horizontal comparison of the same set of cross-validation methods under the three Bootstrap schemes, we have the following conclusions:

1. Bootstrap is a feasible way to build a "parallel A-share market" "The resampling method can simulate the machineLearn the randomness of different links to test whether the research conclusions drawn in the real A-share market are over-fitting. We carried out Bootstrap resampling on the in-sample data, out-of-sample data, and backtest time respectively, and found that the model performance and single-factor backtest indicators of the grouped time series cross-validation method in the "parallel A-share market" are better than the other two methods. The statistical test result is significant. Real-world research conclusions can be reproduced in the parallel world, indicating that the conclusion is less likely to be over-fitting. We use the tools of "accidental" to explore the laws of "inevitability".

2. The three Bootstrap schemes have different directions and degrees of influence on the same set of cross-validation methods. The in-sample data set of Bootstrap is equivalent to adding a small disturbance to the training set factor value, which may slightly weaken the model performance; the out-of-sample data set of Bootstrap is equivalent to adding a small disturbance to the test set factor value, which may partially enhance or weaken the model performance; Bootstrap backtest time That is, changing the backtest period of the model may greatly enhance or weaken the performance of the model. The enlightenment of the above results to researchers is that they need to pay close attention to the quality of training data during the development process, and at the same time avoid misjudgments caused by improper selection of the backtest time.

3. In the past quantitative model development process, historical backtest performance is usually regarded as a deterministic result, and the influence of randomness on the backtest results is ignored. When faced with the choice of different quantitative strategies, they are often simply based on evaluation indicators such as the annualized rate of return, the Sharpe ratio, and the rate of return drawdown based on the strategy. The Bootstrap resampling method provides a way of describing randomness, enabling researchers to make relatively objective judgments and decisions based on the statistical distribution of evaluation indicators, rather than individual statistics. From a methodological perspective, this article reflects on the multi-factor stock selection framework combined with machine learning, and proposes innovative transformations for model comparison and model evaluation, hoping to inspire investors in this field.

risk reminder

artificial intelligence stock selection method is the mining of historical investment rules, if the future market investment environment changes, this method may fail. There are various sources of randomness in the machine learning stock selection model. This study only considers a limited number of three cases, and it is possible to ignore other more important sources of randomness. The Bootstrap resampling method is a simple simulation of randomness, and it may be oversimplified.

Disclaimer

This public platform is not the official subscription platform of Huatai Securities Research Institute. For relevant opinions or information, please refer to the official public platform of Huatai Securities. In accordance with the relevant requirements of the "Measures for the Appropriateness Management of Securities and Futures Investors", the content of this official account is only for professional investors among Huatai Securities clients, and please do not forward the content of this official account in any form. If you are not a professional investor among Huatai Securities clients, please unfollow this official account and no longer subscribe, receive or use the content in this official account. Because it is difficult to set access permissions for this official account, if it causes you inconvenience, please forgive me! The purpose of this official account is to communicate research information and exchange research experience. Huatai Securities does not regard subscribers as customers of Huatai Securities due to any subscription to this official account.

The relevant content of this public number research report is extracted from the published research report. If there is any ambiguity due to the abstract of the report, the complete content of the report shall prevail. For details, please refer to the full report issued by Huatai Securities.

The content of this official account is compiled based on the information that the author believes is reliable and has been published, but the author does not guarantee the accuracy and completeness of such information, nor does it make definite judgments on the rise or fall of securities prices or market trends. The opinions, evaluations and forecasts contained in this official account only reflect the opinions and judgments on the day of publication. At different times, Huatai Securities may issue research reports that are inconsistent with the opinions, assessments and forecasts contained in this official account.

In any case, the information or opinions expressed in this official account do not constitute private investment advice to clients. Subscribers should not rely solely on the information in this subscription account to replace their own independent judgments, and should independently make investment decisions and bear investment risks. If ordinary investors use this information, they may have ambiguities in their understanding of the content due to lack of interpretation services, which may result in investment losses. Huatai Securities and the author shall not bear any legal responsibility for all consequences arising from the basis or use of the contents of this official account.

The copyright of this public account is exclusively owned by Huatai Securities Co., Ltd.Without the written permission of the company, any organization or individual shall not infringe the copyright of all contents published by this official account in any form such as copying, copying, publishing, quoting or redistributing to others. If Huatai Securities causes any direct or indirect losses due to infringements, Huatai Securities reserves the right to pursue all legal liabilities. The company has the business qualification of "Securities Investment Consulting" approved by the China Securities Regulatory Commission, and the business license number is: 91320000704041011J.

Lin Xiaoming

Practicing Certificate Number: S0570516010001

Huatai Metalworking In-depth Reports at a glance

Financial Cycle Series Research (Asset Allocation)

[Huatai Metalworking Lin Xiaoming Team] Twenty years of reappearance, three to four-year cycle cycle-2019 China and global market Annual Views on Asset Allocation (Part 2)

[Huatai Metalworking Lin Xiaoming Team] Twenty years of reappearance, a cycle of three to four years-2019 China and Global Market Quantitative Asset Allocation Annual Views (Part 1)

[Huatai Metalworking Lin Xiaoming Team] Cycle BL asset allocation strategy under rotation

[Huatai Metalworking Lin Xiaoming Team] Cycle Theory and Machine Learning Asset Return Forecast——Huatai Metalworking Market Cycle and Asset Allocation Research

[Huatai Metalworking Lin Xiaoming Team] Judging the Market Turning Point

[Huatai Metalworking Lin Xiaoming Team] 2018 China and Global Market Opportunities and Risks · Annual Strategy Report (Part 1)

[Huatai Metalworking Lin Xiaoming Team] Kitchin Cycle Quantitative Measurement and Historical Laws · Huatai Metalworking Cycle Series Research

[Huatai Metalworking Lin Xiaoming Team] Three Cycle Factors Pricing and Asset Allocation Model (4)-Huatai Metalworking Cycle Series Research

[Huatai Metalworking Lin Xiaoming Team] Cycle Three-Factor Pricing and Asset Allocation Model (3)-Huatai Metalworking Cycle Series Research

[Huatai Metalworking Lin Xiaoming Team] Cycle Three Factors Pricing and Asset Allocation Model (2)-Huatai Metalworking Cycle Series Research

[Huatai Metalworking Lin Xiaoming Team] Cycle Three-Factor Pricing and Asset Allocation Model (1)-Huatai Metalworking Cycle Series Research

[Huatai Metalworking Lin Xiaoming Team] Huatai Metalworking Cycle Research series · Exploration of plate rotation based on DDM model

[Huatai Metalworking Lin Xiaoming Team] Quantitative Decomposition of Market Cycles

[Huatai Metalworking Lin Xiaoming Team] Cycle Research on Forecasting Viewpoints of Large Categories of Assets

[Huatai Metalworking Lin Xiaoming Team] Financial Economic System Cycle Determination of Huatai Metalworking Cycle Series (Part 2)

[Huatai Metalworking Lin Xiaoming Team] Determination of Financial Economic System Cycles (Part 1)-Huatai Metalworking Cycle Series Research

[Huatai Metalworking Lin Xiaoming Team] A Preliminary Study of Global Multi-market Timing Configuration — —Huatai Cycle Timing Research Series

Industry Index Spectrum Analysis and Configuration Model: Market Cycle Analysis Series 3

[Huatai Metalworking Lin Xiaoming Team] Market Frequency-Market Rebirth, Cycle Rebirth

[Huatai Metalworking Lin Xiaoming Team] Market Rebirth ——Preliminary study on the relationship between financial market cycle and business cycle

FOF and financial innovation product

[Huatai Metalworking] Life Cycle Fund Glide Path Development Example——Huatai FOF and Financial Innovation Product Series Research Report One

Factor Week Period (Factor Timing)

[Huatai Metalworking Lin Xiaoming Team] The Relationship between Market Value Factor Income and Economic Structure——Huatai Factor Cycle Research Series 3

[Huatai Metalworking Lin Xiaoming Team] Factor Investment Clock from a Cycle Perspective--Huatai Factor Cycle Research Series 2

[Huatai Metalworking Lin Xiaoming Team] A Preliminary Study on the Periodicity of Factor Returns

Timing

[Huatai Metalworking Lin Xiaoming Team] Huatai Risk-Return Consistency Timing Model

[Huatai Metalworking Lin Xiaoming Team] Technical Indicators and Periodic Price Selection Combination of time model

[Huatai Metalworking Lin Xiaoming Team] Huatai Price and Volume Timing Model-Application of Market Cycle in the Field of Time Selection

Industry Wheeling

[Huatai Metalworking Lin Xiaoming Team] Industry Wheeling Series Six: "Huatai Cycle Wheeling "Fund Portfolio Construction 20190312

[Huatai Metalworking Lin Xiaoming Team] Application of Valuation Factors in Industry Configuration-Huatai IndustryThe fifth series of reports on the rotation

[Huatai Metalworking Lin Xiaoming's team] The application of momentum enhancement factors in the industry configuration--The fourth series of reports on the Huatai industry rotation series

[Huatai Metalworking Lin Xiaoming team] The application of financial quality factors in industry configuration-- Huatai Industry Rotation Series Report III

[Huatai Metalworking Lin Xiaoming Team] An Empirical Analysis of Industry Rotation from the Perspective of Cycle · Huatai Industry Rotation Series II

[Huatai Metal Engineering Lin Xiaoming Team] Industry Rotation Strategy Based on General Regression Model Huatai One of the industry wheeling series:

Smartbeta

[Huatai Metalworking Lin Xiaoming Team] Smart Beta: Riding the Wind and Waves to Take Advantage of the Time-One of the Huatai Smart Beta Series

[Huatai Metalworking Lin Xiaoming Team] Smartbeta's advantages in asset allocation-Huatai Metalworking Smartbeta Special Research One

multi-factor stock selection

[Huatai Metalworking Lin Xiaoming team] Empirical analysis of factor synthesis method-Huatai multi-factor series ten

[Huatai Metalworking Lin Xiaoming team] Huatai single factor test consensus expected factor-Huatai multi-factor series nine

[Huatai Metalworking Lin Xiaoming team] Huatai single factor test financial quality factor-Huatai multi-factor series eight

[Huatai Metalworking Lin Xiaoming team] Huatai single factor test fund flow factor-Huatai multi-factor series seven

[Huatai Metalworking Lin Xiaoming Team] Huatai single factor test volatility factor-Huatai multi-factor series 6

[Huatai Metalworking Lin Xiaoming team] Huatai single factor test volatility factor-Huatai multi-factor series 5

[Huatai Metalworking Lin Xiaoming team ] Momentum factor of Huatai single factor test-Huatai multi-factor series 4

[Huatai Metalworking Lin Xiaoming team] Huatai single factor test growth factor-Huatai multi-factor series 3

[Huatai Metalworking Lin Xiaoming team] Huatai single factor Valuation factors for testing-Huatai multi-factor series 2

[Huatai Metalworking Lin Xiaoming team] Huatai multi-factor model system preliminary research-one of Huatai multi-factor series

[Huatai Metalworking Lin Xiaoming team] Five-factor model A-share empirical research

[Huatai Metalworking Lin Xiaoming Team] A Study on the Effectiveness of Dividend Factors——Huatai Dividend Index and Dividend Factor Series Research Report 2

Artificial Intelligence

[Huatai Metalworking Lin Xiaoming Team] Empirical Research on the Frequency of Machine Learning Stock Selection Model——Huatai Artificial Intelligence Series 18

[Huatai Metalworking Lin Xiaoming Team] Empirical Research on Data Annotation Method for Artificial Intelligence Stock Selection-Huatai Artificial Intelligence Series 17

[Huatai Metalworking Lin Xiaoming Team] Discussing Time Series Cross Validation Again Against Overfitting-Huatai Artificial Intelligence Series 16

[Huatai Metalworking Lin Xiaoming Team] Convolutional Neural Network for Artificial Intelligence Stock Selection Network——Huatai Artificial Intelligence Series 15

[Huatai Metalworking Lin Xiaoming Team] Confronting Overfitting: Talking from Time Series Cross Validation

[Huatai Metalworking Lin Xiaoming Team] Artificial Intelligence Stock Selection Loss Function Improvement——Huatai Artificial Intelligence Series No. 13

[Huatai Metalworking Lin Xiaoming Team] Feature Selection of Artificial Intelligence Stock Selection-Huatai Artificial Intelligence Series 12

[Huatai Metalworking Lin Xiaoming Team] Stacking Integrated Learning of Artificial Intelligence Stock Selection-Huatai Artificial Intelligence Series 11

[Huatai Metalworking Lin Xiaoming Team] Macro Cycle Indicators Applied in Random Forest Stock Selection-Huatai Artificial Intelligence Series 10

[Huatai Metalworking Lin Xiaoming Team] Artificial Intelligence Stock Selection Cyclic Neural Network-Huatai Artificial Intelligence Series 9

[Huatai Jingong Lin Xiaoming’s team] Fully connected neural network for artificial intelligence stock selection-Huatai artificial intelligence series 8

[Huatai Jingong Lin Xiaoming’s team] Artificial intelligence stock selection Python actual combat-Huatai artificial intelligence series 7

[Huatai Jingong Lin Xiaoming team] Boosting model of artificial intelligence stock selection-Huatai artificial intelligence series 6

[Huatai Metalworking Lin Xiaoming team] Artificial intelligence stock selection random forest model-Huatai artificial intelligence series 5

[Huatai Metalworking Lin Xiaoming team] Artificial intelligence stock selection Naive Bayesian Model-Huatai Artificial Intelligence Series 4

[Huatai Metalworking Lin Xiaoming Team] Support vector machine model of artificial intelligence stock selection-Huatai artificial intelligence series 3

[Huatai Metalworking Lin Xiaoming team] Generalized linear model of artificial intelligence stock selection-Huatai artificial intelligence series 2

index enhancement fund analysis

[Huatai Metalworking Lin Xiaoming's team] Re-exploring the regression method to measure fund holding positions——Huatai Fund Position Analysis Special Report

[Huatai Metalworking Lin Xiaoming Team] Juegu Yujin: Index Enhancement Fund Return Analysis

[Huatai Metalworking Lin Xiaoming Team] Fund holdings based on regression method Stock position measurement

[Huatai Metalworking Lin Xiaoming Team] Index Enhancement Methods Summary and Examples-Empirical Empirical Quantification of Multi-factor Index Enhancement Strategies

Fundamental Stock Selection

[Huatai Metalworking Lin Xiaoming Team] Huatai Value Stock Selection's Relative P/E Ratio Hong Kong Stock Model-Relative P/E Ratio An Empirical Study on Southbound Stock Connect Models

[Huatai Metalworking Lin Xiaoming Team] Huatai Value Stock Selection FFScore Model

[Huatai Metalworking Lin Xiaoming Team] Relative P/E Ratio Stock Selection Model A-Share Market Empirical Research

[Huatai Metalworking Lin Xiaoming Team] Huatai Value Stock Selection Cash Flow Factor Research——Empirical Research on Cash Flow Factor Stock Selection Strategy

[Huatai Metalworking Lin Xiaoming Team] Huatai Fundamental Stock Selection Low Price Yield Model-Tip Fisher Stock Selection Method A-Share Empirical Research

[Huatai Metalworking Lin Xiaoming Team] Huatai Basic An empirical study of the A-share selection method of the high dividend rate model of face-to-face stock selection by Ohenness stock selection method

Fund fixed investment

[Huatai Metalworking Lin Xiaoming team] Dacheng's fund 2018 fixed investment strategy research

[Huatai Metalworking Lin Xiaoming team] Bollinger bands and dividend rate timing Fixed investment model-Fund fixed investment series of special research report 4

[Huatai Metalworking Lin Xiaoming team] Fund fixed investment 3-Markowitz validity test

[Huatai Metalworking Lin Xiaoming team] Fund fixed investment 2-Investment target and timing selection method

[ Huatai Metal Industry Lin Xiaoming Team] Fund Fixed Investment 1-Analysis Method and Theoretical Basis

Others

[Huatai Metal Engineering Lin Xiaoming Team] A-share market and industry lunar month effect-month effect 2

A-share market and industry month effect-detailed history The hidden law in the data

Tags: movie