Quantitative Trading Systems

Our goal in system development is to combine a set of logic with a data stream in such a way that profitable trading opportunities can be identified. Short of knowledge of the future, the best estimates of future performance are obtained when two conditions are both true.

• Tests of the system are run using data that has not been used in the development of the system. That is, out-of-sample data and out-of-sample results.
• The data used is representative of the data that will be analyzed and traded.

Those conditions are discussed in detail in other postings. This article focuses on analysis of the test results.

Emanuel Derman, in his book Models.Behaving.Badly., states that models are simplifications; and simplifications can be dangerous. The point I hope to make in this article is that systems developers should avoid simplification of data representation. In short – whenever possible use distributions rather than a limited number of scalar values.

The information content that describes a trading system over a given period of time can be described in many ways. The following list is in decreasing order of information.

• Reality. Trades, in sequence, that actually will result from applying the system.
• List of trades, in time sequence.
• List of trades.
• Distribution of trades.
• Four moments describing the distribution.
• Mean and standard deviation.
• Mean.
• Direction.

Probability and statistics distinguish between population and sample. The population is all items of the type being analyzed — the sample is a subset of the population that has been observed. The purpose of developing trading systems is to learn as much as possible about the population of trades that will occur in the future and make estimates of future performance. The results of testing trading systems form the sample that will be used to make those estimates.
 

Reality

Reality cannot be known in advance. Estimating reality, the population, is the purpose of system validation. Reality is the logic of the system processing the future data series.

List of trades, in time sequence

The list of trades, in time sequence, that results from processing a set of that data that is similar to the future data is the best estimate we can obtain of reality. There is one of these sequences for each unique set of test data and each set of logic and parameter values. Using these results to estimate future profitability and risk depends on the degree of similarity between the test data and the future data.

List of trades

The list of trades, ignoring time sequence, relaxes the assumption of the trades occurring in a particular sequence. It provides a set of data with, hopefully, the same characteristics as the future data, such as amount won or lost per trade, holding period, intra-trade drawdown, and frequency of trading. Selecting trades from this list in random order gives opportunity to evaluate the effects of similar conditions, but in different time sequence.

Distributions

A distribution can be formed using any of the metrics of the individual trades. The distribution is a further simplification since there are fewer (or at most the same number of) categories for the distribution than for the data used to form it. For example, a distribution of percentage gain per trade is formed by sorting the individual trades according to gain per trade, establishing ranges and bins, assigning each trade to a bin, and counting the number of trades in each bin. A plot of the count per bin versus gain per bin gives a plot of the probability mass function (often called the probability density function, pdf).

Four moments

Distributions can be described by their moments. The four moments most commonly used are named mean, variance, skewness, and kurtosis. Depending on the distribution, some or all of the moments may be undefined.

• Mean. The first moment. The arithmetic average of the data points.
• Variance. Second moment. A measure of the deviation of data points from the mean. Standard deviation is the positive square root of variance.
• Skewness. Third moment. A measure of the lopsidedness of the distribution.
• Kurtosis. Fourth moment. A measure of the peakedness and tail weight of the distribution.

Mean and standard deviation

Mean and standard deviation are commonly computed and used to describe trade results. They can be used in the definition of metrics such Bollinger Bands, z-score, Sharpe ratio, mean-variance portfolio, etc.

Mean

The mean gives the average of the values. Mean can be computed in several ways, such as arithmetic mean and geometric mean.

Direction

Direction of a trade describes whether it was a winning trade or a losing trade. Direction is meant to represent any way of describing the trades in a binary fashion. Other ways might be whether the result was large or small in absolute value, or whether the maximum favorable excursion met some criterion, etc.

Stay high on the list

With each step down this list, a larger number of data points are consolidated into a smaller number of categories, and information is irretrievably lost. Knowing only the information available at one level makes it impossible to know anything definite about the population that could be determined at a higher level. Working with only the mean tells us nothing about variability. Working with only mean and standard deviation tells us nothing about the heaviness of the tails. Using the four values of the first four moments enables us to calculate some information about the shape of the population, but nothing about the lumpiness or gaps that may exist.

Application to Modeling Trading Systems

The modeling and simulation technique I recommend repeatedly builds and analyzes a sequence of trades. Each sequence represents a fixed time horizon, say four years. The number of trades in that sequence is the number of trades that can be expected to occur in four years. Each trade is selected from either a trade list or a distribution that represents a trade list. When enough trades to cover four years have been selected, the equity curve, win-to-loss ratio, maximum drawdown, and other metrics of interest can be calculated – just as if this sequence was a four-year trade history.

After generating and analyzing many, usually thousands, of four-year sequence, form and analyze the distributions of the results. Two of the most important metrics of results are account growth and maximum closed trade drawdown. Others that might be of interest include win-to-loss ratio, maximum intra-trade drawdown, etc.

The important points are:

• Use the best data available. Out-of-sample data that best represents the anticipated future data that will be traded.
• Use the highest representation of the data possible. That is, either a list of trades or a detailed distribution of trades.
• Using simulation techniques, generate many possible trade sequences.
• Analyze the distributions of the results of the simulation.
• Calculate and use metrics based on the distributions to judge the usefulness of the system.

Further Reading

If you can read just one:

• Savage, Sam, The Flaw of Averages: Why We Underestimate Risk in the Face of Uncertainty, Wiley, 2009

Add these as you have time:

• Bernstein, Peter, Against the Gods: The Remarkable Story of Risk, Wiley, 1996
• Haigh, John Taking Chances: Winning with Probability, Oxford, 2003
• Leach, Patrick, Why Can’t You Just Give Me the Number?: An Executive’s Guide to Using Probabilistic Thinking to Manage Risk and to Make Better Decisions, Probabilistic Publishing, 2006
• Myerson, Roger, Probability Models for Economic Decisions, Thomeson, 2005
• Nahin, Paul, Digital Dice: Computational Solutions to Practical Probability Problems, Princeton, 2008
• Plous, Scott, The Psychology of Judgement and Decision Making, McGraw-Hill, 1993
• Taleb, Nassim, Fooled by Randomness: The Hidden Role of Chance in Life and in the Markets, Random House, 2005
• Vose, David, Risk Analysis: A Quantitative Guide, Third Edition, Wiley, 2008