Skip to content

16 Comments

  1. JEI
    August 2, 2019 @ 9:47 am

    I’m not sure why I haven’t mentioned this, but since starting to read this series I purchased access to the OptionAlpha backtester. It’s hard to compare apples to apples: OA doesn’t have the same strike selection options, the same time period (they start in 2007), they don’t include commissions, etc. One thing about the results I see over there is that they vary wildly: changing one variable, for example frequency from daily to weekly, or avg days to expiration from 30 to 40, can have massive effects. In one test changing the avg days till expiration changed the CAGR from 13% to -5%.

    Mostly, this makes me doubt their methodology. Maybe their’s some sequence risk the results step into that changes the outcome, who knows.

    I’m also not really seeing results that line up with yours. For example:
    Trade Frequency:Daily
    Days to Expiration:40
    Min IV Rank:None
    Portfolio Size:250,000
    Overall Allocation:50%
    Profit Exit:None
    Stop Loss Exit:None
    Short Strike Delta:0.15
    Spread Width:5 Strikes

    Output:
    Total Profit/Loss: 431,112
    Total Return: 172.44%
    Annual CAGR: 10.46%
    Sharpe Ratio: 0.38

    It looks like a similar setup in your tests produce a CAGR of around 1%. So I dunno. If I can find an email address I’ll send you a couple of PDFs.

    Reply

    • spintwig.com
      August 6, 2019 @ 1:24 am

      Not too long ago I was poking around some “precomputed” backtests that had yielded a similar experience – change a variable and the results change significantly. The tool / table was far too opaque for my tastes and had unusual delta selections such as 15D.

      Yeah, I have many methodology questions too. For example, what does “Spread Width: 5 strikes” mean?

      Even the tool I use to backtest produces many numbers that are nonsense. I throw most out and compute my own stats from the trade logs and other summary info.

      I’ll send you a PM; would be happy to take a look at the PDFs

      Reply

      • JEI
        August 6, 2019 @ 12:57 pm

        Spread width is how far out the long leg of these spreads are. The tool has options for 10 and 15 as well, but I don’t trust the results because it skips trades that don’t look like would fill. And there are no logs, so can’t recompute.

        Reply

        • spintwig.com
          August 6, 2019 @ 2:08 pm

          Let me rephrase: how wide is “5 strikes?” Is that 5 delta, 5 dollars, 5 specific increments of standard deviation, 5 something else? I’ve never heard strike width being defined as a strike.

          Some platforms offer more granularity between strikes than others. For example, Schwab’s StreetSmart Edge doesn’t have nearly as many strikes as Interactive Brokers. In particular, many of the SPY strike values ending with 50 cents are omitted in Schwab’s option chains.

          Depending on moneyness of the strikes being considered, the width between one strike vs the next is not always the same. For example, SPY may have have a $1 difference between strikes then jump to $2.5 difference between strikes, then jump again to $5 difference between strikes.

          Bummer no trade logs are available. Just curious, what happens if you were to ask for them? Assuming they crunched the numbers themselves and didn’t purchase someone else’s work and republish, there should be a trade log somewhere.

          Reply

          • JEI
            August 7, 2019 @ 11:20 pm

            Good question. I’m not sure, beyond just guessing that it’s literally whatever a strike is in the data, whether it’s $0.50, $1.00, or $5.00, depending on the underlying. The mystery deepens 🙂

            Reply

  2. Pushpaw
    September 5, 2019 @ 9:27 pm

    Question: maybe I missed it, but what is the portfolio allocation for the ICs? I’m assuming 100% of the portfolio would be invested in SPY. Does the test have 100% of the portfolio allocated in the daily IC strategy as well? How much is allocated per trade? Allocation could affect results in a test like this.

    That said, I’m with you on the finding that SPY ICs underperforms SPY as I’ve backtested it too on OptionStack, albeit using a shorter backtest period (2011-2019) and not as comprehensively as you have done here. I’m also very glad you have demonstrated the percentage of profits that gets eaten by commissions on a strategy like this. I have a spreadsheet to show myself the percentage of my profits I’m going to keep on each trade, and it’s a sobering thing to track. It makes me very careful about how much credit I take in.

    I also have access to the Option Alpha backtester (as JEI does) and it is indeed a pre-computed tool that has no trade logs. However, they are supposed to be coming out with a new more powerful backtester at OA sometime soon, one that supposedly will be able to backtest and then create a bot that can trade in real time. I would hope that it will have real trade logs, for one, and be much more versatile and sophisticated than the current backtester. It will have to be if it’s going to produce bots.

    Reply

    • spintwig.com
      September 6, 2019 @ 12:53 am

      The portfolio is allocated 100% (no leverage) to the IC strategy. There is no portfolio per se; a position is opened and is measured against the underlying each day. As the days roll on accumulation of positions occurs. These accumulated positions are factored into the daily return calculation. Take a peek at the “download sample” at the end of the post and check out the “StrategyReturns.csv” file. You’ll see the positions accumulating and the daily return calculation.

      All this to say, the mechanics and methodology of the backtest are designed to answer the question: if multiple SPY short IC ETFs were to exist, one for each of the strategies in this study, what would their curves and stats look like? One can choose to invest a single dollar or thousands of dollars into the ETF; the performance characteristics would be the same. It’s portfolio-size agnostic.

      It took some adjusting when I made the switch from the very empirical OptionStack tool to a tool that enables separation of option strategy performance from the effects of leverage. “Daily average returns” wasn’t an intuitive concept for me.

      One of the tool outputs is “StrategyTrades.csv”. With that file I can run in a spreadsheet what’s essentially an OptionStack backtest using leverage, margin, etc. and calculate portfolio allocation stats. It’s with this file one can wear the hat of an ETF sponsor and reverse engineer minimum starting capital for portfolio survival and factor in margin of safety, etc.

      Let me know if this helps.

      With regard for the OA backtester, hopefully the new solution generates trade logs. Without logs there is no validity, no peer review, no way to verify whether the code is accurate.

      Is there an ETA on the new tool’s arrival and/or a document that lists the new backtester’s features / requirements? Curious how expectations are being managed.

      Reply

  3. Pushpaw
    September 6, 2019 @ 11:25 pm

    Ok, I see. Yes, that is a different way of backtesting from what I am used to.

    As for the OA backtester, they keep having “announcements” about what’s coming, but as far as I know there’s no date yet. Here’s the link to the place they’re posting updates (apparently will be updated soon): https://optionalpha.com/option-alphas-new-auto-trading-platform-updates-162692.html

    I guess whoever was behind Alta5 is teaming up with OA for the bots/backtester. I don’t know anything about Alta5 though.

    Reply

  4. Pushpaw
    September 17, 2019 @ 11:27 pm

    Hey, have you ever tried the IC strategy (on any of the ETFs) at 100-150 DTE?

    The idea comes from this guy’s trading plan: https://firebyarthur.com/2018/12/17/the-j-arthur-squiers-trading-plan-cheat-sheet/

    He sells 10 delta strangles at 100-150 DTE. The strategy backtests well on multiple ETFs on OptionStack. I also backtested it as an IC and IC 20 delta at 100-150 DTE performed very well. And I’ve tested the 40DTE ICs before and was underwhelmed. The 100-150 DTE was significantly better.

    Would be interested to see what your testing finds on this strategy!

    Reply

    • spintwig.com
      September 21, 2019 @ 5:54 pm

      Not yet, but exploring varying DTE lengths is on the roadmap.

      In that example commission drag is likely suppressed since the higher DTE strategies have more meat on the bone. Since the IC is one of the most commission-intensive option strategies, it serves as a best-case scenario to mitigate commission drag.

      Reply

      • Pushpaw
        September 22, 2019 @ 11:49 pm

        Would be interesting to see.

        A general question about the backtesting on Spintwig: while it is certainly interesting to see how ICs and many other strategies underperform the benchmark, aside from EEM and VIX, there aren’t strategies posted here that DO beat SPY.

        What I would love to see is Spintwig post option backtests that beat the market. Whatever these strategies might be. Otherwise, the purpose of the site seems to be to prove that option trading isn’t worth the effort. Am I missing something?

        Reply

        • spintwig.com
          September 23, 2019 @ 12:56 am

          Not sure if we want to add USO to the mix. Options on USO outperformed the underlying, though all the strategies were unprofitable, even buy and hold.

          All the research I do gets published. There’s no withholding of information. I’ve been looking high and low for a systematic strategy that outperforms SPY, defined as either having matching returns with lower volatility or higher returns with matching volatility. I’m still empty handed.

          The purpose of the site is to report the facts. If we’re talking about options on SPY, I would agree; you’re not missing anything. Other than a handful of trades I placed in August when VIX was >20, I stopped trading at the end of June based on what the data was suggesting. The lower volatility (and returns) that options on SPY generate on a portfolio of cash can be replicated with a simple shift in asset allocation.

          All this being said, if my portfolio were to have an emerging markets allocation I would consider selling said position and implementing an option strategy on EEM using that slice of capital.

          Meanwhile, in the coming weeks I’ll be researching option strategies on AAPL, AMZN, C, GE and T. I hold no individual positions but those that do may find value in this data.

          Reply

          • Pushpaw
            September 23, 2019 @ 8:47 am

            Wasn’t suggesting you withhold information. Was suggesting simply that maybe some different strategies need to be looked into. For example strategies based on technical entry?

            Clearly, indiscriminate IC and credit spread entries don’t do the trick. Maybe they will work better on the stocks but I doubt it.

            Reply

            • spintwig.com
              September 24, 2019 @ 11:01 pm

              Works for me. There are scores of indicators. Perhaps I’ll start with IV percentile above/below 50% and benchmark against holding a delta-comparable amount of SPY for those durations. I’ll add this to the list.

              Reply

  5. Nick
    May 13, 2020 @ 12:50 am

    I’ve been looking over many backtests recently. It seems like there are significantly different results. Your backtest is very thorough in defining terms. In other backtests, they place stop losses, enter one trade a month, have spread widths of $1, and exit earlier at 8 DTE. It seems like those factors greatly improve the results. Have you tried adding these variables? Also, a trade every day seems very risky when a sudden move or black swan event occurs. I appreciate going through these.

    Thanks!

    Reply

    • spintwig.com
      May 13, 2020 @ 2:01 am

      Hi Nick!

      Can you point me in the direction of some of those other backtest results please? I’d like to take a look to see what their methodology entails. As for those variables, I’ve explored three of the four:

      Stop losses were performed on the SPY Short Put 45 DTE Cash-Secured study with results suggesting a profit-taker exit outperformed. As markets ebb and flow, implementing a profit taker attempts to exit at a high point while stop losses exit at a low point.

      As for a once-a-month trade, there are actually ~30 different return profiles for such a strategy. Namely, opening a position on the 1st of every month, the 2nd of every month, and so on. The technical name for this phenomena is called timing luck and portfolio tranching (see page 7) (here’s a visual of the associated concept in action) and it allows someone to find the tranche that performs most in line with their narrative then publish only that scenario. Opening a position every day eliminates this potential bias in strategy performance.

      Using spread widths in terms of fixed dollar amounts is unique in that it represents different spread widths and risks at different times. For example, a $1 spread width when SPY is $150 represents perhaps a 10-delta difference between strikes (this of course depends on whether the short position is 50D or 16D). Meanwhile, when SPY is $300 that same $1 width may only be a 5-delta difference between strikes. Using a fixed-dollar-width strategy on an underlying that appreciates in value will overtime get more conservative as the distance between short and long strikes gets smaller from a % standpoint. I haven’t yet studied such strategies but can certainly look into them.

      The SPY Short Straddle 45 DTE study explored exiting at 28, 21, 14, and 7 DTE as well as holding till expiation. 21 DTE had the highest CAGR and Sharpe Ratio among the profit target exits so I continued to stick with 21 DTE as the default early-management duration exit.

      Opening a position every day does expose a portfolio to downside risks. This and the other studies are mostly a tool to provide ballpark performance figures for a given strategy. The daily position is more about ensuring solid methodology and results and less bout suggesting someone implement a given strategy as their daily trade (though if they do, the results should be similar).

      For what it’s worth, I recall reviewing a few of the OptionAlpha backtests. The methodology was unclear and left several unanswered questions such as commission cost assumptions, idle cash calculations, which tranche was used when positions were opened once a month, etc. Also worth noting: the benchmark used was SPY price return as opposed to total return, making the option strategy appear more profitable than it actually is vs the buy/hold alternative.

      My backtesting tool is public so others can peer review the code, report any bugs and run their own backtests.

      Hopefully this helps!

      Reply

Leave a Reply

Your email address will not be published. Required fields are marked *


This site uses Akismet to reduce spam. Learn how your comment data is processed.