CAN STOCK MARKET FORECASTERS FORECAST? a paper read before a joint meeting of the Econometric Society and the American Statistical Association, Cincinnati, Ohio, December 31, 1932 INTRODUCTION THIS paper presents results of analyses of the forecasting efforts of 45 professional agencies which have attempted, either to select specific common stocks which should prove superior in investment merit to the general run of equities, or to predict the future movements of the stock market itself. The paper falls into two main parts. The first deals with the attempts of two groups, 20 fire insurance companies and 16 finan al services, to foretell which specific securities would prove most profitable. The second part deals with the efforts of 25 financial publi- cations to foretell the future course of the stock market. various sta- tistical tests of these results are given These investigations were instituted five years ago as a means of testing the success of applied economies in the investment field. It seemed a plausible assumption that if we could demonstrate the exist- ence in individuals or organizations of the ability to foretell the elusive fluctuations, either of particular stocks, or of stocks in general, this might lead to the identification of economic theories or statistical prac- tices whose soundness had been established by successful prediction The forecasters include well-known organizations in the diffe fields represented, many of which are large and well financed, employ ing economists and statisticians of unquestioned ability. The names of these organizations are omitted, since their publication would be likely to invite wholesale controversy over the interpretation of their records Some of the forecasters seem to have taken a page from the book of the Delphic Oracle expressing their prophecies in terms susceptible of more than one construction. It would frequently be possible, therefore, for an editor, after the event, to present a plausible challenge of our interpretation. Most of the forecasts appear through the medium of weekly publications and each of these has been read and recorded on the day it became available to us, which in practically every case was before the event. In this way certain possible elements of bias have been eliminated. It was impossible that hindsight could infuence our judgment, either in the selection of publications for analysis or in the interpretations placed on their forecasts. In the case of the fire insur ance companies, however, the analyses were made annually, based the transactions reported in Kimber' s Record of Insurance Company Security Purchases. The companies were selected as fairly representa 309
310 ECONOMETRICA tive of their class. The analysis of the 26-year forecasting record of William Peter Hamilton, former editor of the Wall Street Journal, also falls in a different category, in that it was undertaken because of the reputation for successful forecasting which he had established over a long period of years. FORECASTING THE COURSE OF INDIVIDUAL STOCK PRICES We turn first to the records of two groups the financial services and the fire insurance companies, which have attempted to select individual stocks that would prove more profitable for in vestment than the aver- age issue. The first part of this section deals with the records, over the 43 years ending july, 1932, of 16 leading financial services which have made a practice of regularly submitting to their subscribers selected lists of common stocks for investment. Our analysis includes about 7,500 separate recommendations, requiring approximately 75, 000 en- tries. The first step was to record each week the name and price of each stock recommended for purchase or sale by each service. Next came the tabulation of the advice to sell or cover the commitment previously advised Reiterated advice was not considered, action being assumed to have been taken as of the date when the recommendation was first published. The percentage gain or loss on each such transac tion was recorded and, in a parallel column, the gain or loss of the stock market for the identical period. a balance was struck every six months which summarized the total results secured by each service as com pared with the action of the stock market. Proper corrections were of course, made to offset the effect of changes in capital structure resulting from the issue of rights, stock dividends, etc. Since a tendency existed among some services to emphasize their conspicuously successful stock recommendations and ignore more unfortunate commitments adopted a practice of automatically dropping a stock from the list six months after it had been last recommended, when specific advice to sell was not given a redistribution of funds in equal amounts among all stocks recom- mended has been assumed for each service at the beginning of every si months'period analy zed. It could be maintained of course, that this equalizing process should take place as often as once a week but this would increase the labor of computation to overwhelming proportions Provisional experiments demonstrated that it would yield conclusions practically identical with those secured by the shorter method. Com pounding the successive six months'records gives the percentage by which each service s recommendations have exceeded, or fallen behind the stock market as shown in table i Only six of the 16 services achieved any success. To arrive at an
ALFRED COWLES BRD 311 average performance, the record of each service was reduced to an ef. fective annual rate which was then weighted in accordance with the length of the period represented. The average annual effective rate of all the services, thus arrived at, is.43 per cent TABLE I SULTS OF COMMITMENTB IN STOCKS RECOMMENDED BY 16 FINANCIAL SERVICES(RELATED TO MARKET AVERAGES) Service Weel Per cent 234 123456 234 17.2 34 234 234 462 860 104 0.5 234 52 o123456 020 2284 04 33.0 PROBABILITY TESTS In an attempt to determine whether the service having the best rec- ord achieved its result through skill or chance, we resorted to the the- ories of compound and inverse probability. Our conclusion is thus rendered consistent by obtaining approximately the same answer in two different ways. With the aid of various checks, involving 1250 computations of the action of individual stocks selected at random we derived a formula A, D.(0)=5.42+1.5t(A D,=average deviation, t, in units of 4 weeks 21), representing the deviation, for all periods from one month up to one year, of the a verage individual stock from the average of all stocks ce Number 1, for the 9 six months' periods from January 1 July 1, 1932, was successful 7 times and unsuccessful 2 times with the aid of the table referred to, the averages of "chances in 1000 to do worse''for the 7 periods in which it was successful and the 2 pe- riods in which it was unsuccessful were found to be 842 and 66 re spectively. By the theory of direct probabilities, the probability of a single service being right at least 7 times in 9 is equal to the sum of the first3 terms of the binomial(}+号)° p=1/2+9/29+36/20=46/512=090 The probability that a single service could in 9 predictions be 7 times
312 ECONOMETRICA on the positive side and in these 7 forecasts equal the achievement of ervice Number P=090×(1-842)=014 However, the record of the best service is marred by its failure in the two negative cases. The average of the two chances to do worse in these ases is 066. We then have Q=(7/9)×842+2/9×066=.670 as the probability of a single random service having a record worse than that of Service Number 1. We therefore conclude that the proba bility that a random service can, first, be on the right side of the market 7 times out of 9, and second, equal in performance the record of Service Number l, is P=090×(1-670)=.030 This means that in 16 services we should expect to find 16X.030=48 services which will equal the record of Service Number 1. That is to say, the chance is even that we should get at least one service as good Because of the assumptions implied in this computation, we shall ar gue this another way We shall assume that the probability that a serv- ice for its total forecast shall be on the positive side of the market is 1/2. Then the estimate of its success must be made by a different evaluation of Q. For this purpose we shall adopt a formula suggested by Bayes'rule in inverse probability in which the weights. 910 and. 090 instead of 7/9 and 2/9 are used. We get 910(842) 910(842)+(090)(934) Hence, if a service was on the right side of the market, the proba Thus the compound probability would be Ce Number 1 would be 1-Q bility of its achieving the success of Servic P=1/2(1-901)=050 Among the 16 services the probability of the most successful one equal- ling the record of Service Number l would be P=16 X.050=80, that is to say, we should expect to get among 16 random services about one service which would equal Number 1. Since this answer is quite con- sistent with our previous answer, our analysis suggests the conclusion that the record of Service Number l could not be definitely attributed to skill
ALFRED COWLES 3RD 313 TWENTY FIRE INSURANCE COMPANIES The second analysis deals with the common stock investments, from 1928 to 1931 inclusive, of 20 of our leading fire insurance companies Its significance lies in the fact that these companies are representative of a class of common stock investor which has had long years of experi ence and large amounts of capital at its disposal. Fire insurance dates from the great London Fire of 1666, and active investment in stocks developed during the nineteenth century. The fire insurance companies are much older hands at the business of investment than either the financial services, which are a twentieth century product, or American investment trusts, which are largely a development of the last few years. The investment policies of these companies are based on the ccumulated knowledge of successive boards of directors whose judg ment might be presumed, over the years, to have been well above that of the average investor. The 20 companies which were selected for analysis hold assets totalling several hundred million dollars, and seem a fair sample of their kind Fire insurance companies carry between 20 and 30 per cent of their total investments in common stocks Their average turnover amounts to only some 5 per cent a year For this reason it was thought best to confine our analysis to the record of the actual purchases and sales made during the period under examination, rather than to compute the record of the entire common stock portfolio. To simplify the labor, allitems of stock purchased were given equal weights, regardless of the amounts involved. While the conclusion does not exactly refect the actual investment results secured by these companies, it should, how ever, provide a satisfactory test of the success of these organizations in selecting stocks which performed better than the average The method employed in the analysis is essentially the same as that ed in the case of the investment services. a second purchase of an item was omitted from the record unless a sale of this item intervened A record of the sale of an item, of course determined the date as of which it was dropped from the list. Also, any item of which there had been no purchase recorded for 12 months was automatically considered The compounded records of the 20 companies for the 4-year are shown in Table It Six of the companies show evidence of success, and the average of the 20 is -4.72 per cent. The average record of the companies in the stocks which they selected for investment fell below the average of the stock market at the effective annual rate of 1. 20 per cent. a comparable result could have been achieved through a purely random selection of