GARCH
Moderator: hocus2004
I have added easy to copy listings to my tables with HDBR50 and HDBR80. I have included the percentage earnings yield 100E10/P and P/E10 listings as well.
New HDBR Tables dated Sun Jan 11, 2004.
http://nofeeboards.com/boards/viewtopic.php?t=1962
This should make life easier for gummy if I have posted these tables soon enough. I apologize for the delay.
Have fun.
John R.
New HDBR Tables dated Sun Jan 11, 2004.
http://nofeeboards.com/boards/viewtopic.php?t=1962
This should make life easier for gummy if I have posted these tables soon enough. I apologize for the delay.
Have fun.
John R.
Here are the equations for the 30-year Historical Surviving Withdrawal Rates of HDBR50 when using only twenty start years.
HDBR50 Curve Fitting Results:
y=Historical Surviving Withdrawal Rate
x=percentage earnings yield 100E10/P (using Professor Shiller's P/E10).
Look at the values of R-squared (listed as R2).
Retirement start years of 1891-1910 show very little correlation between the percentage earnings yield and the Historical Surviving Withdrawal Rates.
Since we have found that the first eleven years are the most important for predicting portfolio success, the anomalous results are associated primarily with the years of 1891-1921.
Have fun.
John R.
HDBR50 Curve Fitting Results:
y=Historical Surviving Withdrawal Rate
x=percentage earnings yield 100E10/P (using Professor Shiller's P/E10).
Code: Select all
1871-1890
y = 0.4194x + 4.7548
R2 = 0.2591
1891-1910
y = 0.0241x + 5.0012
R2 = 0.0024
1911-1930
y = 0.3072x + 3.1385
R2 = 0.7875
1931-1950
y = 0.3971x + 2.3409
R2 = 0.4373
1951-1970
y = 0.658x + 1.3092
R2 = 0.9561
Retirement start years of 1891-1910 show very little correlation between the percentage earnings yield and the Historical Surviving Withdrawal Rates.
Since we have found that the first eleven years are the most important for predicting portfolio success, the anomalous results are associated primarily with the years of 1891-1921.
Have fun.
John R.
Here are the values for CPI that are actually used by the Retire Early Safe Withdrawal Calculator, version 1.61, dated November 7, 2002 (when CPI is selected). The same values are in all of my calculators, which are modified versions.
The original source is Professor Robert Shiller's database.
This is in an easy to copy form.
1871-1920
1921-1980
Have fun.
John R.
The original source is Professor Robert Shiller's database.
This is in an easy to copy form.
1871-1920
Code: Select all
12.46
12.65
12.94
12.37
11.51
10.85
10.94
9.23
8.28
9.99
9.42
10.18
9.99
9.23
8.28
7.99
7.99
8.37
7.99
7.61
7.80
7.33
7.90
6.85
6.57
6.66
6.47
6.66
6.76
7.90
7.71
7.90
8.66
8.28
8.47
8.47
8.85
8.66
8.94
9.90
9.23
9.13
9.80
10.00
10.10
10.40
11.70
14.00
16.50
19.30
Code: Select all
19.00
16.90
16.80
17.30
17.30
17.90
17.50
17.30
17.10
17.10
15.90
14.30
12.90
13.20
13.60
13.80
14.10
14.20
14.00
13.90
14.10
15.70
16.90
17.40
17.80
18.20
21.50
23.70
24.00
23.50
25.40
26.50
26.60
26.90
26.70
26.80
27.60
28.60
29.00
29.30
29.80
30.00
30.40
30.90
31.20
31.80
32.90
34.10
35.60
37.80
39.80
41.10
42.60
46.60
52.10
55.60
58.50
62.50
68.30
77.80
John R.
Here are the values for CPI that are actually used by the Retire Early Safe Withdrawal Calculator, version 1.61, dated November 7, 2002 (when CPI is selected). The same values are in all of my calculators, which are modified versions.
The original source is Professor Robert Shiller's database.
This is in an easy to read form.
Notice the years of declining prices in the late 1800s. They bottomed out in 1897. Even though the late 1800s were a time of great prosperity, the index fell much more than they did in the Great Depression.
Year CPI
Year CPI
Have fun.
John R.
The original source is Professor Robert Shiller's database.
This is in an easy to read form.
Notice the years of declining prices in the late 1800s. They bottomed out in 1897. Even though the late 1800s were a time of great prosperity, the index fell much more than they did in the Great Depression.
Year CPI
Code: Select all
1871 12.46
1872 12.65
1873 12.94
1874 12.37
1875 11.51
1876 10.85
1877 10.94
1878 9.23
1879 8.28
1880 9.99
1881 9.42
1882 10.18
1883 9.99
1884 9.23
1885 8.28
1886 7.99
1887 7.99
1888 8.37
1889 7.99
1890 7.61
1891 7.80
1892 7.33
1893 7.90
1894 6.85
1895 6.57
1896 6.66
1897 6.47
1898 6.66
1899 6.76
1900 7.90
1901 7.71
1902 7.90
1903 8.66
1904 8.28
1905 8.47
1906 8.47
1907 8.85
1908 8.66
1909 8.94
1910 9.90
1911 9.23
1912 9.13
1913 9.80
1914 10.00
1915 10.10
1916 10.40
1917 11.70
1918 14.00
1919 16.50
1920 19.30
Code: Select all
1921 19.00
1922 16.90
1923 16.80
1924 17.30
1925 17.30
1926 17.90
1927 17.50
1928 17.30
1929 17.10
1930 17.10
1931 15.90
1932 14.30
1933 12.90
1934 13.20
1935 13.60
1936 13.80
1937 14.10
1938 14.20
1939 14.00
1940 13.90
1941 14.10
1942 15.70
1943 16.90
1944 17.40
1945 17.80
1946 18.20
1947 21.50
1948 23.70
1949 24.00
1950 23.50
1951 25.40
1952 26.50
1953 26.60
1954 26.90
1955 26.70
1956 26.80
1957 27.60
1958 28.60
1959 29.00
1960 29.30
1961 29.80
1962 30.00
1963 30.40
1964 30.90
1965 31.20
1966 31.80
1967 32.90
1968 34.10
1969 35.60
1970 37.80
1971 39.80
1972 41.10
1973 42.60
1974 46.60
1975 52.10
1976 55.60
1977 58.50
1978 62.50
1979 68.30
1980 77.80
John R.
Here are the equations for the 30-year Historical Surviving Withdrawal Rates of HDBR80 when using only twenty start years.
HDBR80 consists of 80% stocks and 20% commercial paper.
HDBR80 Curve Fitting Results:
y = Historical Surviving Withdrawal Rate
x = percentage earnings yield 100E10/P
Pay special attention to R-squared. It points to an anomaly associated with 1891-1910.
Retirement start years of 1891-1910 show very little correlation between the percentage earnings yield and the Historical Surviving Withdrawal Rates.
Variations around 1891-1910 for HDBR80
y = Historical Surviving Withdrawal Rate
x = percentage earnings yield 100E10/P
Looking at these results and Professor Shiller's CPI numbers (that I extracted from my calculator), this anomaly seems to be associated with the reversal from declining prices that bottomed out in 1897. Later, prices increased.
I do not know the extent to which this reversal was a result of changing economic conditions. I do know that the late 1800s saw major advances (especially those associated with transportation) that reduced prices sharply without cutting into profitability. In addition, these were the days of the Gold standard and before the creation of the Federal Reserve Banking System.
A side effect of these generally declining prices was that commercial paper showed huge profits simply because interest rates were always above zero.
Have fun.
John R.
HDBR80 consists of 80% stocks and 20% commercial paper.
HDBR80 Curve Fitting Results:
y = Historical Surviving Withdrawal Rate
x = percentage earnings yield 100E10/P
Code: Select all
1871-1890
y = 0.6721x + 3.2028
R2 = 0.5259
1891-1910
y = 0.1114x + 4.9226
R2 = 0.0245
1911-1930
y = 0.4194x + 2.7638
R2 = 0.8089
1931-1950
y = 0.7189x + 1.5714
R2 = 0.5634
1951-1970
y = 1.1936x - 1.2958
R2 = 0.9495
Retirement start years of 1891-1910 show very little correlation between the percentage earnings yield and the Historical Surviving Withdrawal Rates.
Variations around 1891-1910 for HDBR80
y = Historical Surviving Withdrawal Rate
x = percentage earnings yield 100E10/P
Code: Select all
1881-1900
y = 0.5113x + 3.445
R2 = 0.3858
1891-1910
y = 0.1114x + 4.9226
R2 = 0.0245
1901-1920
y = 0.4215x + 2.3582
R2 = 0.8776
I do not know the extent to which this reversal was a result of changing economic conditions. I do know that the late 1800s saw major advances (especially those associated with transportation) that reduced prices sharply without cutting into profitability. In addition, these were the days of the Gold standard and before the creation of the Federal Reserve Banking System.
A side effect of these generally declining prices was that commercial paper showed huge profits simply because interest rates were always above zero.
Have fun.
John R.
Why not keep them both?gummy wrote:When I get a minute I'll change the HFWR charts to HSWR charts....
I think that your HFWR charts are interesting.
I know that many people, especially younger retirees, are more interested in HFWR than in HSWR. They don't want financial problems to take away from their happiness if they live longer than 30 years.
Have fun.
John R.
They should if the original hypothesis that prices are closely related to withdrawal rates is true.gummy wrote:In the meantime, peek at this chart:
After spending a couple of weeks on GARCH (and volatility clustering) it looks so familiar!!
If so, we would expect HFWR charts and price charts to share some of their properties.
Have fun.
John R.
Here's something to think about:
1: We'd like to predict next year's SWR.
2: We note the intimate relationship between SWR's and E10/P ... in the past.
3: We also note that GARCH (Nobel prize, 2003) is a mechanism for predicting tomorrow's daily volatility, based upon recent daily volatility data ... and that GARCH generates "clustering" (as observed in historical daily volatility).
4: We also note that historical yearly SWR exhibits similar clustering (as per the pretty chart above)
5: So we use GARCH to predict next year's SWR ... but (of course) with years instead of days.
5: Then we win the 2006 Nobel prize
P.S.
I have a scheme much simpler than GARCH that does a similar thing.
Mine is much like the riskmetrics model.
1: We'd like to predict next year's SWR.
2: We note the intimate relationship between SWR's and E10/P ... in the past.
3: We also note that GARCH (Nobel prize, 2003) is a mechanism for predicting tomorrow's daily volatility, based upon recent daily volatility data ... and that GARCH generates "clustering" (as observed in historical daily volatility).
4: We also note that historical yearly SWR exhibits similar clustering (as per the pretty chart above)
5: So we use GARCH to predict next year's SWR ... but (of course) with years instead of days.
5: Then we win the 2006 Nobel prize
P.S.
I have a scheme much simpler than GARCH that does a similar thing.
Mine is much like the riskmetrics model.
Gummy wrote:I still don't understand those chart labels
Are there two-count-em-two HDBR50 Fits ?
Sorry about the typo in the plot labels. I ran those plots off pretty quickly, and never even noticed the mistake! Matching the colors instead of letting gnuplot pick them automatically would have been a good idea, too.John R. wrote:PS#1: Yes, that is a glitch. The bottom line (magenta or reddish purple) should be labeled HDBR80. [It would have been better if the data and the lines had shared the same colors.]
Your plots are much nicer, Gummy.
Bpp
To Bpp:
If you would like to update your plots, I am sure that ES will be glad to incorporate them. Your plots have been exceedingly helpful. Just one glance and a viewer knows that Historical Surviving Withdrawal Rates (also known as Historical Database Rates) and P/E10 are closely related.
Have fun.
John R.
If you would like to update your plots, I am sure that ES will be glad to incorporate them. Your plots have been exceedingly helpful. Just one glance and a viewer knows that Historical Surviving Withdrawal Rates (also known as Historical Database Rates) and P/E10 are closely related.
Have fun.
John R.
While implementing your own approach, consider taking advantage of the following incomplete set of software and instructions. I think that it comes very close to describing a very useful, commercially valuable stock market price generator. My impression is that they fail this only because they are perfectionists. They do not realize the value of what they have already done.gummy wrote:Here's something to think about:
1: We'd like to predict next year's SWR.
2: We note the intimate relationship between SWR's and E10/P ... in the past.
3: We also note that GARCH (Nobel prize, 2003) is a mechanism for predicting tomorrow's daily volatility, based upon recent daily volatility data ... and that GARCH generates "clustering" (as observed in historical daily volatility).
4: We also note that historical yearly SWR exhibits similar clustering (as per the pretty chart above)
5: So we use GARCH to predict next year's SWR ... but (of course) with years instead of days.
5: Then we win the 2006 Nobel prize
P.S.
I have a scheme much simpler than GARCH that does a similar thing. Mine is much like the riskmetrics model.
I went to this site after following links in Benoit Mandelbrot's book.
http://classes.yale.edu/fractals/
I think that there is enough information to build reasonable surrogates for stocks and the stock market (which includes the S&P500 index).
They come very close to providing everything that we would need. They have downloadable software. (I think that it is sufficient.)
They even provide pictures. They provide comparisons of real data and generated data using various inputs to what they call Iterated Function Systems (IFS). Most of those pictures look realistic to me.
They provide sufficient detail to generate Trading Time from (the actual) Clock Time and then to drive a conventional (Brownian) probability distribution of price differences. [Making this composite function appears to be the best approach. Brownian motion corresponds to taking random steps from a standard Gaussian, normal, bell shaped distribution without any memory from previous steps.]
They do not provide a simple, standard piece of software to simulate price changes. My impression is that they are demanding too much realism. There is money to be made by packaging input software for financial planning Monte Carlo models. Few, if any, require a high degree of fidelity. All, or almost all, of them would benefit greatly from simulating price clusters.
Have fun.
John R.
Actually, I have no intention of doing more than providing space for your tutorial ... and maybe a few chartsWhile implementing your own approach, consider taking advantage of the following incomplete set of software and instructions.
However, when I read what's there now, I say to myself:
It's January, 2005.
I have all the historical data anyone could want, going back a hundred years and ending last month
I see all the relationships, the charts and the numbers that can be generated with this info.
Now I ask: What should my withdrawal rate be this year"
Professor Robert Shiller's database does not have today's value of P/E10. It is necessary to extrapolate from an earlier, but recent, value.gummy wrote:However, when I read what's there now, I say to myself:
It's January, 2005.
...
Now I ask: What should my withdrawal rate be this year"
From Professor Robert Shiller's database: in January 2004, the S&P500 price (or index value) was 1132.52 and P/E10 was 27.65.
From cbs.marketwatch.com using the symbol sp500: today's index value (or price) of the S&P500 is 1185.
Assuming that E10 has not changed very much (since it is the average of a whole decade's worth of earnings), today's value of P/E10 is approximately equal to [today's S&P500 price/last year's S&P500 price]*[last year's value of P/E10] = [1185 / 1132.52]*[27.65] = 28.93.
The formulas require the percentage earnings yield 100% / [P/E10]. Today's percentage earnings yield is very close to 3.46%.
Therefore: use x = 3.46 or 3.5 in the formulas.
Have fun.
John R.
Look here:
http://www.gummy-stuff.org/JWR.htm#HSWR
and see if'n I got the procedure right.
In my explanation, what's N?
http://www.gummy-stuff.org/JWR.htm#HSWR
and see if'n I got the procedure right.
In my explanation, what's N?
I realize it's a judgment call, but if the stuff y'all have been working on for years doesn't give some guidance then I don't understand its usefullness.That's always going to be a personal judgment call, isn't it?
Here's something which occurs to me:
Suppose that, each January (starting in Jan, 1970), you estimated (predicted?) the SWR for the coming year (using the procedure you've been working on) and actually withdrew that amount.
What would your porfolio be, 30 years later (in Jan, 2000)?
Has anybuddy looked at that?
For gummy:
Change: "Determine E10/P over the past N years. (Each value involves ten years of data.)" to something with this information: "Look up E10/P from Professor Shiller's database at the start each year of the historical sequences that you are going to use. Typically, this is 1921-1980 or 1923-1980. Sometimes, it is necessary to limit the historical sequences to completed sequences. In such cases, the final year of each sequence must be 2002 or earlier. [The Retire Early Safe Withdrawal Calculator and its modified versions available from the NoFeeBoards.com website use dummy data with heavy losses for 2003-2010.] Sometimes, you will be interested in starting from 1871, typically using start years of 1871-1980 (or 1881-1980 because of uncertainty regarding how the 1871-1880 values of P/E10 were determined)."
Professor Shiller's web address is:
http://www.econ.yale.edu/~shiller/
[P/E10 data are used by the calculators. This provides a convenient alternative source of P/E10 for those willing to look into the details of spreadsheet calculations.]
Change: "the Historical Survival Withdrawal Rates (where the final portfolio is greater than $0)... to "the Historical Survival Withdrawal Rates (where the final portfolio is an amount slightly greater than $0)...
Your procedure correctly describes how to determine the Calculated Rate. To determine the Safe Withdrawal Rate and the High Risk Rate, we must provide confidence limits. The lower confidence limit is the Safe Withdrawal Rate and the higher confidence limit is the High Risk Rate.
Eyeball estimates are sufficient in most cases. Draw lines parallel to the line given by equation that capture almost all of the individual data points. The lower line is the Safe Withdrawal Rate and the upper line is the High Risk Rate.
For more precise estimates, use standard procedures to calculate variances and standard deviations. That is, calculate the differences between the data points and the calculated rates, square them, take the sum and divide by the number of the degrees of freedom. [See the section that follows regarding the number of the degrees of freedom to use.] Take the square root to determine the standard deviation. Set the confidence limits at plus and minus 1.64 standard deviations (corresponding roughly to a 90% confidence level).
The withdrawal rates that people actually choose are called Personal Withdrawal Rates. Typically, they are a little bit higher than the Safe Withdrawal Rate but below the Calculated Rate. Some people prefer to withdraw even less than the Safe Withdrawal Rate.
Have fun.
John R.
Change: "Determine E10/P over the past N years. (Each value involves ten years of data.)" to something with this information: "Look up E10/P from Professor Shiller's database at the start each year of the historical sequences that you are going to use. Typically, this is 1921-1980 or 1923-1980. Sometimes, it is necessary to limit the historical sequences to completed sequences. In such cases, the final year of each sequence must be 2002 or earlier. [The Retire Early Safe Withdrawal Calculator and its modified versions available from the NoFeeBoards.com website use dummy data with heavy losses for 2003-2010.] Sometimes, you will be interested in starting from 1871, typically using start years of 1871-1980 (or 1881-1980 because of uncertainty regarding how the 1871-1880 values of P/E10 were determined)."
Professor Shiller's web address is:
http://www.econ.yale.edu/~shiller/
[P/E10 data are used by the calculators. This provides a convenient alternative source of P/E10 for those willing to look into the details of spreadsheet calculations.]
Change: "the Historical Survival Withdrawal Rates (where the final portfolio is greater than $0)... to "the Historical Survival Withdrawal Rates (where the final portfolio is an amount slightly greater than $0)...
Your procedure correctly describes how to determine the Calculated Rate. To determine the Safe Withdrawal Rate and the High Risk Rate, we must provide confidence limits. The lower confidence limit is the Safe Withdrawal Rate and the higher confidence limit is the High Risk Rate.
Eyeball estimates are sufficient in most cases. Draw lines parallel to the line given by equation that capture almost all of the individual data points. The lower line is the Safe Withdrawal Rate and the upper line is the High Risk Rate.
For more precise estimates, use standard procedures to calculate variances and standard deviations. That is, calculate the differences between the data points and the calculated rates, square them, take the sum and divide by the number of the degrees of freedom. [See the section that follows regarding the number of the degrees of freedom to use.] Take the square root to determine the standard deviation. Set the confidence limits at plus and minus 1.64 standard deviations (corresponding roughly to a 90% confidence level).
The withdrawal rates that people actually choose are called Personal Withdrawal Rates. Typically, they are a little bit higher than the Safe Withdrawal Rate but below the Calculated Rate. Some people prefer to withdraw even less than the Safe Withdrawal Rate.
Have fun.
John R.
No.Suppose that, each January (starting in Jan, 1970), you estimated (predicted?) the SWR for the coming year (using the procedure you've been working on) and actually withdrew that amount.
What would your portfolio be, 30 years later (in Jan, 2000)?
Has anybuddy looked at that?
The procedure would require that the SWR calculation for the second year be based upon a time frame of 29 years. For the third year, it would have to be from a time frame of 28 years. And so on.
That's a lot of equations to calculate. We also have to determine the lower confidence limits for all of them.
Have fun.
John R.