Home   Workshops    Products    Contact Us

View this newsletter on-line, or read back issues

   

Tharp's Thoughts Weekly Newsletter

October 31, 2007 — Issue #345
  
New 20% OFF Sale Expires Next Week on How to Develop a Winning Trading System Home Study 
Article

Historical Testing of Efficient Markets Part III by Van K. Tharp Ph.D.

Workshops E-mini Workshop Coming Next Weekend
Trading Tip

Markets Drive on as Fed Sticks to Script – What Next? by D.R. Barton, Jr. 

Melita's Corner

Psychology 101 by Melita Hunt

New

Revised 2nd Edition

20% Off Discount Expires Next Week

How to Develop a Winning Trading  System that Fits You Home Study 

•  11 audio CDs of new material! 

•   9 CDs from the classic home study course

•   Updated 340 page manual

Learn More...

20% off for a limited time only

Feature

Historical Testing of Efficient Markets

Part III

by

Van K. Tharp, Ph.D.

In part II, we did historical testing of an efficiency signal on today’s S&P 500 data going back to the year 1980.  Basically we bought highly efficient stocks on the first of the month and held them with a 25% trailing stop.  Once the portfolio had 25 stocks (i.e., with 1% risk we were fully invested), it only bought more stocks when we were stopped out of a loser.  The net result was a compounded annual ROI of 38%.  It took 674 trades and rejected (i.e., we were fully invested) 701.  56.7% of our trades made money and the average win was 3.87 times bigger than the average loss.  We also spent 378 days in a winning trade versus 80 days in a losing trade.  And we spent nine million four hundred dollars in trading costs, which amounted to 1% going in and 1% going out, so you can’t say that low cost influenced our results.  The Sharpe ratio of this system is nearly two and the System Quality Number™ is close to Holy Grail range.  Also some of our early position sizes were ridiculous because a split-adjusted stock like Microsoft might have been 10 cents at its onset.

However, remember that this was today’s S&P 500 going back to 1980.  Tell me what the S&P 500 will be 25 years from now, and I can probably just buy and hold those stocks that are NOT ON THE LIST and produce tremendous results.  Also we were not buying stocks that were subsequently dropped from the list.  So how much of our results are due to the survivorship bias?  Potentially, a lot of the gain could have been due to that.  Furthermore, we have not yet determined if we could get similar results with any trend following entry, including a simple 180 -day channel breakout (i.e., the stock makes a new 180-day high).

Remember that this research is based upon my feeling of confidence that if you buy stocks that show efficient uptrends (i.e., they are fairly straight lines going up) with a 25% trailing stop and risking 1%, you will make nice profits under most market conditions.  I showed that in a 2001 issue of Market Mastery and in the recent evaluation of our portfolio.  Except for two potential data errors 1) the survivorship bias or worse yet, 2) the possibility of buying great stocks before they are recognized as being the industry leaders, the last study might have proved my point.  After all we made a compounded ROI of 38% while the S&P 500 during that time only made a compounded return of 10-12%.   

1)     Is it possible to automate this form of trading?  How do you take something that’s fairly discretionary and turn it into something that’s objective?  Right now, it’s my subjective judgment that determines whether or not something is a good efficient stock.  And this question, by the way, is probably the most difficult question for most trading systems. 

2)      If the method can be automated, what is the formula that will totally define an efficient stock for us?  At this point we seem to have answered the first two questions, but have we really?

3)     How can we overcome some of the data problems that are present in historical stock data? 

4)     What market conditions are favorable for this method and what market conditions should be avoided?  Most of our testing included the entire secular bull market from 1982 through 2000 and we didn’t lose money in any year until 2002 and 2003 (which perhaps suggests that getting a list of future big stocks did influence the data).

5)      What can we expect from this method long term?

      I don’t expect to answer all of those questions in these studies.  But if you begin to understand some of the problems involved in backtesting,  then I’ve met my objective for writing these articles.  And I think I’ve already illustrated many of them.  We’ve shown great results through backtesting with a method I’m already confident in.  But the problems still remain.  And there is always the possibility that some coding errors were involved in some of our great results.

When I published the first study a couple of weeks ago, I asked my readers for comments on what might be wrong with it from your perspective.  Thank you for your answers which included 1) the survivorship bias, 2) inflationary impact of the last 30 years, and 3) the impact of trade size.  Trade size was considered major because as our equity got huge, we were taking on huge positions that could have moved the market. 

Initially, our sizes were pretty big because of the split- adjusted S&P 500.  For example, if you buy a stock that was introduced at $40 per share, but because of splits is data adjusted to be $1 per share at its entry, then you are going to be buying huge position at the onset.  However, those positions are only huge because of the split/dividend adjustment.  Second, we did gravitate toward huge positions as our equity grew, but many of the S&P 500 stocks can take huge positions without moving the market that much.  In addition, to compensate for some of that we took a 1% hit both going in and going out.  That’s probably quite high for early positions and low for big positions at the end of the study.

Probably the biggest adjustment that we’d have to make to the data is the impact of withdrawals for taxes, etc.  However, you would have that problem with any data set.

Incidentally, what most of you probably don’t realize is that I have made no position sizing adjustment to really push the results.  With the kind of System Quality Numbers we’ve been getting in this research, it would be pretty easy to make position sizing adjustments that would give us triple digit returns.  But we haven’t done that.  Instead, we have used a simple 1% risk.  I could increase my starting equity to one million and then do 0.1% risk.  This would allow me to take $4000 positions on 250 different stocks (so we could be in as many as half of the S&P 500 stocks at any one time).  However, before I make position sizing adjustments, I’m looking to find the algorithms that I like.

All of this research, by the way, is being done with Mechanica Professional software with the assistance of its developer, Bob Spear.  Thank you, Bob.  And while I’m using the Professional  version, everything we’re doing can be done with the standard version.

Overcoming Bugs in Our Coding

Between the time I published the first study and this writing, we have found several errors in our coding.  So remember, you can get great results despite coding errors and sometimes because of them.  The first coding error was in the way that a variable called TOTALCASH (that was used to determine position sizing) was calculated.  That error was fixed.  It affected position sizing and reduced the total compounded return by about 5% -- the rest of the variables remained the same.  But that illustrates how much a variable that affects position sizing can affect the results.

Table 1 shows the major changes in the results with correction in the TOTALCASH variable.

Table 1:  Impact of the TOTALCASH Adjustment
  With TotalCash Error Without TotalCash Error
Compounded ROI 38.65% 33.33%
Trades Taken 674 596
Trades Rejected 701 779
Percent Win 56.22% 58.22%
W/L Ratio 3.87 3.07
Days Winning Trade 378 376
Days Losing Trade 8 80

The second error was a bug in how we did the ranking.  However, that error actually improved the results slightly and that problem is fixed in all subsequent studies.  The third coding problem was in the way the smoothing function was calculated.  When you calculate the difference in the price and find the standard deviation (which is what we did for the smoothing function), it’s going to give a very low standard deviation with low-priced stocks (over high-priced stocks) and favor them in the ranking.  However, at the same time, the efficiency algorithm actually moves away from low priced stocks (i.e., a stock that moves from $0.25 to $0.75 is very unlikely to get an efficiency rating of 8).  Thus, while very few low-priced stocks will have an efficiency rating above 8, when we have “close minus close” involved in our smoothing ranking, we’ll tend favor those when they do achieve an efficiency rating above 8.  Table 3 shows a sample of the first 25 stocks picked.  The average position size was 8000 shares, which suggests that at $1000 risk (i.e., 1% of the initial $100,000) the average risk was 0.125 and the average price of those shares was 4 times that or 50 cents.  Table 4 shows a sample of the first 25 stocks picked with “close divided by close” used.  The average price of those shares, based upon the same calculations, was about $2.82.  As a result, we started using “close divided by close” in our smoothing function. 

Unfortunately, this change produced a very unusual result.  It kept us in both winning and losing trades much longer.  And as a result, we had fewer trades and much poorer results.  The compounded return on investments dropped significantly to 14.58%.

Table 2 shows the impact of using “close divided by close” in our smoothing function.  Notice that 1) win percentage doesn’t change much and 2) the win/loss ratio actually goes up.  Thus, the only reason for the major decrease in the performance is the dramatic increase of number of days in both winning and losing trades increases dramatically.

I have no clue why we stayed in both winning and losing trades so much longer and this aspect of the study is not over (in my opinion) until I understand the reason.  It is particularly surprising to me that we stayed in losing trades for almost a year (versus 80 days).  Why?  We still have the same 25% trailing stop.  However, this is typical of the kinds of issues that you get into with back testing.

Table 2: Impact of the Change in the Smoothing Adjustment
  Using Close/Close Without Total Cash Error
Compounded ROI 14.58 33.33%
Trades Taken 207 596
Trades Rejected 211 779
Percent Win 55.56% 58.22%
W/L Ratio 3.43 3.07
Days Winning Trade 714 376
Days Losing Trade 212 80

This data also shows why backtesting is not THE answer – at least, not by itself.  I could have stopped testing when I found a compounded ROI of 38.65% with a method that I was already confident in.  Instead, this sort of testing is just the means to helping you determine how and why your method works (or doesn’t work).

But why does the change in the smoothing function dramatically alter the average days in both the winning and losing trades?  One possibility could be that the trades generated by the two systems were totally different as shown in Tables 3 and 4. 

Table 3:  Close Minus Close Smoothing
Symbol Entry Date Exit Date $ P/L Position Size
SHW 10/1/1980 9/1/1981 ($216.00) 3448
CAG 10/2/1980 7/22/1981 ($110.00) 3125
LUV 10/3/1980 7/9/1981 $2,484.00 10810
MDP 11/3/1980 10/1/1985 $11,630.00 2139
SYY 12/1/1980 4/14/1983 $5,619.00 9760
GE 12/1/1980 9/18/1981 ($646.00) 3177
MTB 1/2/1981 10/20/1987 $18,497.00 5214
PGR 1/2/1981 1/12/1982 $1,684.00 9932
PFE 1/2/1981 9/18/1981 ($1,051.00) 3758
CTB 1/2/1981 9/3/1981 $104.00 4850
SYK 1/2/1981 2/3/1981 ($834.00) 9481
LEG 3/2/1981 9/25/1981 $936.00 7383
LOW 3/2/1981 8/28/1981 ($781.00) 7020
SYK 3/2/1981 8/17/1981 ($86.00) 11896
FRX 3/2/1981 7/6/1981 ($351.00) 13383
WAG 4/1/1981 5/25/1984 $6,085.00 11555
BMS 4/1/1981 11/6/1981 ($865.00) 4898
IPG 5/1/1981 10/26/1981 ($1,120.00) 4709
SLE 6/1/1981 10/19/1987 $16,031.00 4601
DDS 6/1/1981 2/7/1984 $16,806.00 5654
STT 7/1/1981 8/9/1983 $6,489.00 10393
FDO 7/1/1981 6/9/1982 $607.00 10171
FRX 8/3/1981 8/25/1981 ($1,239.00) 14503
FITB 9/1/1981 10/8/1986 $13,979.00 6517
LUV 9/1/1981 8/24/1983 $5,561.00 8779
SYK 9/1/1981 12/15/1982 $2,492.00 13035
Average # Shares 8007.64
Average Risk Amount = $0.12
Average Price of Stock= $0.50

Note that the first 25 trades were all taken in a major BEAR market, but some of them still turned into huge winners with the 25% trailing stop.  In Table 3, two stocks were held until the 1987 crash.  In Table 4, three stocks were held that long.

 

Table 4: Close Divided by Close Smoothing
Symbol Entry Date Exit Date $ P/L Position Size
LEG   10/1/1980 12/28/1981 ($626) 645
ITT   10/1/1980 2/2/1981 ($207) 167
UNH   10/1/1980 1/12/1981 ($755) 76
PGR   11/3/1980 10/1/1985 $11,679 2148
STA   11/3/1980 8/2/1983 $2,211 471
DOV   12/1/1980 9/18/1981 ($643) 3159
EMN   12/1/1980 5/12/1981 ($1,072) 742
FHN   1/2/1981 7/25/1984 $5,376 1626
EQR   1/2/1981 5/28/1982 ($323) 765
KEY   1/2/1981 3/8/1982 $52 1560
TAP   1/2/1981 9/4/1981 ($621) 1947
BMS   1/2/1981 9/3/1981 ($825) 1017
RRD   2/2/1981 10/14/1981 ($968) 85
DUK   3/2/1981 10/19/1987 $8,003 1035
BMS   3/2/1981 10/16/1987 $14,108 1445
DIS   3/2/1981 2/1/1984 $1,343 2285
PEP   3/2/1981 9/23/1981 ($675) 254
GDW   4/1/1981 12/14/1982 $118 1192
RRD   4/1/1981 6/9/1982 ($650) 853
HNZ   4/1/1981 11/6/1981 ($808) 4575
AYE   5/1/1981 10/20/1987 $12,699 907
DTE   5/1/1981 6/9/1982 ($599) 762
WB    5/1/1981 10/26/1981 ($1,041) 4379
UST   6/1/1981 9/11/1986 $7,186 1590
D     6/1/1981 6/15/1984 $356 784
EIX   7/1/1981 2/25/1982 ($925) 977
Average # Shares 1417.84
Average Risk Amount = $0.71
Average Price of Stock= $2.82

I looked at the standard deviation of the days in trades for both samples.  For the "close minus close" sample, the mean was 274.74 days and the standard deviation was 311.72 days.  For the "close divided by close", the mean was 491.71 days and the standard deviation was 492.7 days.  If you did a t-test comparing these two samples, you could not reject the hypothesis that they were from the sample population.

For the next few articles, I plan to look at the following:

1.      What happens when we simply buy and hold a position from each stock for the entire 25 years?  One of you actually did this study, but it did not correspond to the same years so it is difficult for us to do a meaningful comparison.

2.      We’ll also determine what happens when we allow ourselves to take as many as 250 trades (i.e., half the S&P 500 database) at any one time with the two smoothing functions.  With 1% risk and a 25% trailing stop we are limited to 25 trades.  With a 0.1% risk and a 25% trailing stop, we are limited to 250 trades.  We’ll simply increase our starting equity to $1M so that we’ll be investing the same amount ($4000) with each trade.

3.      Is there a better algorithm to find what I’m looking for?  I’m not convinced, given these results, that I’m really buying the stocks I’d normally buy when looking at a chart.  One way would be to look at charts of the 100 trades from both smoothing algorithms to determine how many of them look like efficient stocks.  This will give us a good idea of whether or not we are looking at efficient stocks.  If any of you would like to do that and save me some time, I’d appreciate it.  Please let us know and we’ll send you the data.  And if there are a number of you, we’ll simply split them up. 

4.      We’ll also try both the 180 day channel breakout and linear regression to pick our trades.

5.      And, lastly, when I feel I have some of the answers I’m looking for, we’ll move to the real S&P 500 database that we have.

Notice at this point I still have not yet done the following: 1) Looked at the effect of any trend following algorithm and compared it with efficiency. 2) Looked at the data on a S&P 500 database that added and subtracted stocks as the index did. 3) Made position sizing adjustments to see what’s really possible with this sort of trading.  All of that is still to come in subsequent articles and it looks like this series might continue for some time.

About Van Tharp: Trading coach, and author, Dr. Van K. Tharp is widely recognized for his best-selling book Trade Your Way to Financial Freedom and his outstanding Peak Performance Home Study program - a highly regarded classic that is suitable for all levels of traders and investors. You can learn more about Van Tharp at www.iitm.com.

Workshops

 


Professional E-Mini Futures Trading Tactics

with D.R. Barton and Christopher Castroviejo

E-mini index futures are among the most popular trading instruments in the world. Low margin rates and low commissions combine to make e-mini futures trading one of the most highly leveraged and profitable areas in all of trading. 

Nov 10-11-12

Cary, NC

 

Trading Tip

Markets Drive on as Fed Sticks to Script – What Next?

by D.R. Barton, Jr.

Wednesday the Fed dropped the discount rate and the overnight Fed funds rate by 25 basis points to the surprise of absolutely no one. 

Everyone with any political agenda will continue to apply salve to the credit markets as they try to heal the retail housing woes.  And since the housing woes are far from over, it’s a good guess that the rate cuts aren’t done yet either.

And as our good friend Dr. Steve Sjuggerud says, it’s hard to bet against the markets when the Fed is adding fuel to the fire.

So the intermediate-to-long-term outlook remains bullish for many analysts, and it’s difficult to make a case against a market that just won’t go down!  But — there are some factors that point to short- term stalling signs.

There is little doubt that the Nasdaq has been the driving force in the market for this whole year (Nasdaq 100 is up more than 20% while the S&P 500 is up only around 8%).  Here’s a comparison chart for the major indexes since the August lows:

Here we see that the Nasdaq 100 has outperformed the Dow Jones 30, the S&P 500 and the Russell 2000 by two-to-one.  So this is clearly the fuel that’s feeding the market's engine.  But all is not fair and bright in technology land.  It seems that only a few of the big players, Google, Microsoft, and Apple chief among them, have been powering the latest drive.  One way that we can plainly see this phenomenon is that while the Nasdaq has moved to new yearly highs, the percentage of stocks on the Nasdq 100 that are above their 50- day moving average has dropped from a whopping 85% in early October, down to about 55% now.  So there are clearly a bunch of stocks not participating. This can be seen graphically in the chart below:

What can we do with this information?  Longer term traders who are looking to get in or add to positions would do well to wait for an opportunistic pullback.  Short term traders can just enjoy the volatility that has crept back into the markets. This is just a divergence that has shorter term implications, but it should be given due consideration in your analysis.

Great Trading!

D. R

About D.R. Barton: D.R. (along with Christopher Castroviejo) will be presenting the upcoming “Professional E-Mini Futures Tactics” workshop, November 10-12.

A passion for the systematic approach to the markets and lifelong love of teaching and learning have propelled D.R. Barton, Jr. to the top of the investment and trading arena where he is one of the most widely read and followed traders and analysts in the world.

He is a regularly featured guest analyst on both Report on Business TV,  and WTOP News Radio in Washington, D. C., and has been a guest analyst on Bloomberg Radio.  His articles have appeared on SmartMoney.com and Financial Advisor magazine. You may contact D.R. at drbarton@iitm.com.

 

Melita's Inspirational Corner

Psychology 101

by Melita Hunt

I just wrote an entire article about Psychology 101 and the issues of transference, projection, displacement and rationalization, which are just some of the things that have been showing up for me over the last few weeks. My life has become a whirlwind of emotions, decision making and new lessons. I have been calm at times and the next minute I am upset. It’s a rollercoaster.

I’ve been reading a lot, observing myself and talking to therapists, doctors and counselors about a variety of things. So I decided to share some of this information in my column. However, when I finished the article, I read back through it and went “Wow!  How depressing!". Did I really write those words? It didn’t seem like it came from me, and what on earth was I trying to convey?

If people want to learn about why they inadvertently kick the dog after a bad day at the office; or shout at their wives for no reason after they have a losing day in the markets, then they can search the web for the subjects that describe these very primitive and common issues and learn Psychology 101 for themselves. My job is to write an inspirational column! I’ll leave the rest to the psychologists, psychiatrists and therapists. Life is too short to be writing miserable stuff.

So what has been inspirational in my life this week? It’s been another week of dealing with cancer issues, which some may think doesn’t translate readily to inspirational. But as I said last week, I can’t ignore the elephant in the room. So let’s look for five blessings about my cancer instead.

1. I feel very loved. I have had the chance to feel love and re-connect with many people in my life. The words “I love you” are floating around much more than I have ever noticed before.

2. Doing years of self- work is well worth it. The majority of the time I feel calm, accepting and ready for the lessons that are yet to come. I am looking for the true purpose of this journey and why am I experiencing it? Living in the moment, day by day, without knowing the outcome is actually quite exciting.

3. I started a blog, and I get to write and share my thoughts and feelings with others.

4. I let my body guide me as to what it needs rather than pushing it to its limits. Life doesn’t need to be lived in the fast lane and believe it or not, things still get done.

5. I can choose to be happy, regardless of the circumstances.   

So what is something that supposedly sucks in your life right now? Is it your trading? Your relationship? Your job? Your health?

Take a moment to think about whether it really is as bad as you think. Perhaps there is a profound lesson waiting to be learned. Can you find five blessings or lessons about your current circumstance and see it from a new perspective?

If not, then maybe you have some decisions and changes to make about how you are choosing to live and view your life.

I wish you the best of luck.

Melita Hunt is the CEO of the Van Tharp Institute. If you would like to keep up with Melita’s progress regarding her recently diagnosed lung cancer. Please feel free to read her blog at www.myleftlung.com.

Feedback

Feedback to Dr. Tharp and the Van Tharp Institute

Everything that we do here at the Van Tharp Institute is to help you improve as a trader and investor. Therefore, we love to get your feedback, both positive and negative!

Feel free to click below to leave us any comments so that we can serve you better.

Click Here for Feedback Form

 

Do Not Reply to this email using the reply button as the email address is not monitored, your email will not be seen. Please click this link to contact us: suggestions@iitm.com

The Van Tharp Institute does not support spamming in any way, shape or form. This is a subscription based newsletter.

If you no longer wish to subscribe, Unsubscribe Here

Or, paste this address in your browser: http://www.iitm.com/privacy_policy.htm

 

The Van Tharp Institute
102-A Commonwealth Court, Cary, NC 27511 USA
800-385-4486 * 919-466-0043 *  Fax 919-466-0408

Back to top

Copyright 2007 the International Institute of Trading Mastery, Inc.

.

.

.

...

.

Quote:

"People may doubt what you say, but they will believe what you do. ~Lewis Cass

.

.

.

.

.

.

..


Trade Your Way to Financial Freedom

.

.

.

.

.

..

.

Back to top

.

 

 

 

 

 

Learn More...

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

Back to top

.

.

.

.

.

.

.

.

.

.

.

.

.

.

~

~~~~~

Click here to see our
Workshop Schedule

 

.Back to top

~

~

~

~

 

Free Downloads

Handbook for Traders and Investors

Workshop Syllabus

 

~

~

~~

~

Free Trading Simulation Game

A computerized version of Van's famous "marble game."

It is designed to teach you the important principles of proper position sizing.

Download the 1st three levels of the game for free. Register now.

~

.

.

.

.

.

Check out Dr. Tharp's Blog:

smarttraderblog.com  

~

.

.

~

Tharp Concepts Explained...

 

- Psychology of Trading

- System Development

- Risk and R-Multiples

- Position Sizing

- Expectancy

- Business Planning

Learn the concepts...

~.

.

.

.

.

.

.

~

Share this newsletter with a friend!

~

~

~

~

~~