Part One. This article is about building my own PHP backtesting software and the lessons I learned along the way.
I have been trading Stocks and Shares on etoro for some time now. The results have been great so far. However, I am bright enough to know that this success is mostly down to a bullish stock market rather than any skill on my part.
I have of course been learning about stocks and shares, watching plenty of videos and reading lots of articles. Sorting the wheat from the chaff can be an effort.Many of them are scams where people pretend to be millionaires and then try to sell you a course. Keep your scam alert set to high and don’t pay good money to fraudsters. I recommend watching coffeezilla videos which expose the scams. If you spend time learning to spot a scam you will find that is time well worth spending.
Like me, you have probably read somewhere that 76% or 80% or “90% of people who buy and sell shares will loose money.” Whatever the exact percentage, the statement is broadly true. However this is not the same as saying “people who approach trading sensibly will lose their money”.
The ‘90%’ includes the hundreds of thousands of people who approach shares like they approach casually betting on a horse. They put some money on a stock because they like the name or got a tip from a friend. Just like with horse racing, those people will mostly go home sadder and poorer and will hopefully vow never to gamble again.
The stock market has a constant stream of new small investors who enter the market, make a couple of plays, lose a bit of money, get frightened and go away again, never to return.
The stock market is rigged against us… but that’s ok!
One of the key things I have come to understand is that that the game is effectively rigged against small investors like us. It is done through a number of techniques that also appear in similar forms in casino gambling, there are other people (large corporations) manipulating the system to their betterment and our detriment. However the good news is that are things we can do to mitigate the risk and actually engineer a risk to reward profile that is similar to the big investors.
I have read about lots of trading strategies based on statistics that should (and do, so far) give me a consistent modest return. I accept there will be losing trades and winning trades. As long as the profit I make on winning trades more than offsets losses on losing trades, we win in the end. I didn’t invent these strategies. They are not secret. Pretty much everyone uses them. I will talk more about the specifics later on.
I am not here to convince you to invest in stocks. I am simply letting you know what I am doing.
Why automate trading?
You can trade manually, clicking buttons to make trades, or you can set up a computer to effectively click the buttons for you based on the rules you have designated. The reason this is a truly exceptional idea is that it keeps emotions out of the process. Emotions lead to irrational decisions and these are often the key difference between success and failure.
For example, we may have a ‘rule’ that says never buy until a particular stock reaches ‘x percent above the previous day’s close’. However, after an afternoon that has seen a run of trades that make a loss it is easy to tweak the rule in order to try and claw back some extra cash. This is where the emotion leads to poor choices. We KNOW that we will have runs of losses from time to time. We have factored them in to our profit calculations, but right there and then, when the screen is full of red, we long to see a bit of green and our plans fly out of the window. At that point we are no longer acting as a reasoning investor but have turned in to a gambler… and we know what happens to them.
Well a good way to keep things very unemotional is to automate your trading. A computer simply follows the rules without emotion. If the rules say buy, the computer will buy. If the rules say sell, the computer will sell. The computer doesn’t worry about whether it is the right thing to do, or whether the previous five trades were positive or negative. The computer doesn’t fret and dither. This has the added benefit of working a lot faster than manually trading. This is also a good thing because alongside emotionless trading, faster trading is another way of levelling up the stacked game.
How to automate
Now there are plenty of people trying to sell info about how to automate trading but there seem to be so many sharks out there that I don’t really know who is legitimate and who is a hucksters out for my cash. In these situations it’s a good strategy to assume that everyone who tries to sell me something is a crook, so I decided to build my own trading system from scratch (dont worry, I wont try and sell it to you. I suspect PHP backtesting is a bit niche, so you and I are in a small group of enthusiasts. Sensible people are probably searching ‘C++ backtesting’ or ‘python backtesting‘).
What is backtesting?
Backtesting is the process of checking that your automated rules will actually work and give you a consistent profitable result. Without backtesting you will not know whether your automated trading system will make money or loose money, so backtesting is essential to the whole operation.
Why PHP backtesting?
I have years of programming experience so although it is a daunting challenge, it is one I am confident I can finish. I will use the PHP programming language because that is the one I am most familiar with so for me it will be fastest to do it that way. Other people advocate python and it probably is a better choice for a number of good reasons, but I am sticking with PHP backtesting because I want this to be fun.
I searched online for PHP backtesting but found no useful results but I read lots of articles on building backtesting engines in other programming languages, so I know I a not doing anything particularly novel or radical… which is good news for both of us. I am just translating to the PHP language, backtesting strategies that have been done in other languages already. Hopefully I can write clearly to make the process as easy as possible.
And if you want to come along with me on this journey you can, because I won’t hide anything or try to sell you anything. I will be honest about what I do know and what I don’t know. I will also be commenting all the code I show. This is something that really helps both you and me understand exactly what is happening at each point of the program. When we have many thousands of lines of code it can get confusing unless everything is well annotated with clear comments.
There is one thing that has been bugging me while I have been thinking this through. It is that eToro don’t currently have an API for placing trades. So at the moment on this platform my trades have to be placed manually. This sounds like it kind of defeats the purpose of the exercise, but there is a way around it. I will look at browser macros. I have written them in the past and they are a wonderful automation tool, although I will need the macro to collect data from somewhere outside of the macro… but hey, If my backtesting engine can write to a website (and it can), then the macro can always go and find that data. If all that fails I may have to switch from etoro to a different trading platform. Regardless of that the automated backtesting is still going to save me a ton of time and effort, so here we go.
How to start?
The first step is to think about what we need. I find that drawing out a diagram (or series of diagrams) is always a good start. My first diagram was pretty useless (as you can see below)but that doesn’t matter. The point is to get ideas out of your head and on to a more concrete form.
So let’s start building the first part. Look at your diagram. If (like me) you can’t see where the start might be, draw it again and make a change here or there. After 4 or 5 diagrams you will almost certainly worked out a far more understandable map of where you are headed.
Building a PHP backtesting engine
We will call the first variable we need $symbol. This will hold the stock exchange code for whichever specific stock we are looking at. Google’s code is GOOG, Amazon’s code is AMZN and Facebooks code is FB.
We can use the Coca-cola company as an example.
<?php //backtesting program //set initial variables $symbol="KO"; ?>
There, we have started. Now I won’t tell you how to run a program. If you don’t know how to do that then this article is not the right one for you. You could find a prebuilt backtesting tool and skip the whole build it yourself approach, but you will probably have to pay for the pleasure.
For those who are still with me, the next task in PHP backtesting is to go and get some historic share data for our chosen stock. Thankfully many providers offer this info on the web. Both Google and Yahoo have API’s available that are free to use (within limits) I am going to use Google Finance who offer a clean, simplefree api for stocks and shares data but please be aware it is unsupported so if it breaks at some point in the future it is unlikely that Google themselves will fix it for us. It is a basic api that takes a symbol as a get variable and returns a JSON object
To keep things neat I will create a sub-folder called ‘includes’and inside that create a file called getdata.php. This is where we will put the code to go grab a dataset from Google Finance. Trust me on this. Sub folders are a great way to keep things organised and will make life a lot easier later on when we don’t have to track through reams of code to find the missing semicolon that has broken the entire program.
In fact, mapping out a basic plan of the folders and files of your PHP backtesting machine is another good thing to do at this point.
Now we need to connect our two files together and send the symbol to the getdata function. This should return an array we will call $dataset.
<?php //backtesting program //set initial variables $symbol="KO"; $dataset=""; //connect to the getdata file include ('/includes/getdata.php'); //run the getdata function getdata($symbol); //for testing output the results - we will comment this out later print_r($dataset); ?>
Part two coming September 2021