Option Return Predictability with Machine Learning and Big Data

Turan G. Bali Georgetown University, USA

Heiner Beckmeyer University of Münster, Germany

Mathis Mörke

University of St.Gallen, Switzerland

Florian Weigert

University of Neuchâtel, Switzerland

Drawing upon more than 12 million observations over the period from 1996 to 2020, we find that allowing for nonlinearities significantly increases the out-of-sample performance of option and stock characteristics in predicting future option returns. The nonlinear machine learning models generate statistically and economically sizable profits in the long-short portfolios of equity options even after accounting for transaction costs. Although optionbased characteristics are the most important standalone predictors, stock-based measures offer substantial incremental predictive power when considered alongside option-based characteristics. Finally, we provide compelling evidence that option return predictability is driven by informational frictions and option mispricing. (JEL G10, G12, G13, G14)

Received November 8, 2021; editorial decision January 19, 2023 by Editor Stefano Giglio. Authors have furnished an Internet Appendix, which is available on the Oxford University Press Web site next to the link to the final published paper online.

群内每日免费分享5份+最新资料

300T网盘资源+40万份行业报告

为您的创业、职场、商业、投资、亲子、网赚、艺术、健身、心理、个人成长 …… 全面赋能！

关注公众号获取更多资料

添加微信，备
注“入群”
立刻免费领取
200套知识地图+最新研报

致终身学习者社群

行业报告/思维导图/电子书/资讯情报

收钱文案、增长黑客、产品运营、品牌企划、营销战略、办公软件、会计财务、广告设计、摄影修图、视频剪辑、直播带货、电商运营、投资理财、汽车房产、餐饮烹饪、职场经验、演讲口才、风水命理、心理思维、恋爱情趣、美妆护肤、健身瘦身、格斗搏击、漫画手绘、声乐训练、自媒体打造、效率软件工具、游戏影音……

扫码先加好友，以备不时之需

The importance of option markets has gained momentum over the past decade. According to data from the Futures Industry Association’s (FIA) annual statistical review, options trading on exchanges worldwide has increased from \$9.42 billion contracts in 2013 to \$21.22 billion contracts in 2020, a growth rate of more than 125%. Approximately 60% of these contracts are written on individual stocks and stock indices, making equity the most popular underlying asset of financial market participants. Given the high popularity of options trading by investors, the question arises whether individual option returns are predictable and, if yes, which characteristics can give rise to such predictability. Our paper is devoted to answering these questions.

While classical option pricing models assume that options are redundant assets (Black and Scholes 1973), more recent research rejects this idea and shows that option prices depend on risks other than the underlying’s exposure (Buraschi and Jackwerth 2001; Garleanu, Pedersen, and Poteshman 2009). As an example, Goyal and Saretto (2009) document that the cross section of option returns reflects a premium for variance risk, computed as the difference between historical realized volatility and at-the-money implied volatility. In this paper, we follow the idea of characteristic-based asset pricing and link future delta-hedged option returns to ex-ante characteristics drawn from both options and stocks. As we eliminate the directional impact of stock prices through our hedging procedure, we focus on risks that are inherently nonlinear and are likely to interact with each other in complex ways. Hence, the described setup is ideally suited for the application of machine learning models that are not only able to capture the impact of nonlinearities and interactions between a large set of option and stock characteristics, but also mitigate the risk of in-sample model overfitting.

We study the cross section of individual U.S. equity option returns using data from OptionMetrics IvyDB over the period from January 1996 to December 2020. To abstract from the directional exposure to the underlying, we follow Bakshi and Kapadia (2003) and perform daily delta-hedges for each option as the market closes. Our main variable of interest is the monthly excess deltahedged option return. After accounting for different filtering techniques, our data set consists of more than 12 million option-month return observations of calls and puts, all written on individual U.S. stocks.

To predict future option returns, we use a total of 273 variables composed of 80 option-based characteristics (e.g., option illiquidity, time-tomaturity, and the implied shorting fee) and 193 stock-based characteristics.1 The stock characteristics include the 94 predictor variables proposed by

Green, Hand, and Zhang (2017) to predict the cross section of stock returns, 90 industry dummies, and additional characteristics that have been shown to be significantly associated with future stock returns (such as the bear beta proposed by Lu and Murray [2019], default risk of Vasquez and Xiao [2021], and the underlying’s close price following Eisdorfer, Goyal, and Zhdanov [2022]). In the same fashion as Gu, Kelly, and Xiu (2020), we apply different linear and nonlinear machine learning models to form optimal predictions based on these option- and stock-based characteristics. Linear models included are penalized regression models (Ridge, Lasso, and Elastic-Net) and dimensionality reduction regressions (principal component and partial least squares). Nonlinear models comprise gradient-boosted regression trees with and without dropout, random forests, and fully-connected feed-forward neural networks. We also compute equal-weighted ensembles of all linear and all nonlinear models to combine the informational content of the individual models.

To assess the predictive power of the different models for individual option returns, we follow Gu, Kelly, and Xiu (2020) and use the out-of-sample R2- statistic, which benchmarks the R2 against a forecast of zero excess returns.2 To make pairwise comparisons of the forecast accuracy of different machine learning models, we utilize the model-free Diebold and Mariano (1995) test statistic.

Our empirical results advance the knowledge on predictability of the cross section of individual option returns in various dimensions: First, we show that complexity of the prediction model matters. While none of the linear models manages to produce positive out-of-sample R2s for the entire testing sample, all nonlinear models do. Our results reveal that the best-performing models are gradient-boosted regression trees with and without dropout (GBR and Dart) producing out-of-sample R2s of 2.26% and 1.96%.3 Moreover, the equal-weighted ensemble of all nonlinear models (denoted N-En) outperforms the ensemble of all linear models (denoted L-En) by more than 1.7% in outof-sample R2 prediction power. Our results are confirmed when we compare pairwise forecast accuracy using Diebold and Mariano (1995) tests: The ensemble of all nonlinear models beats all other models and most other models with statistical significance at the 5% level (the only exceptions to this finding are GBR, Dart, and feed-forward neural networks, which all produce forecasts highly correlated with the nonlinear ensemble model (correlations amount to

0.95, 0.93, and 0.77, respectively). The outperformance of nonlinear models compared to linear models is stable over time with a higher predictability for future option returns in 69.8% of the months in our sample (86.0% when considering the cross-sectional out-of-sample R2). Notably, we also find better predictions for the nonlinear models during the December 2019–December 2020 period, in which the COVID-19 pandemic shook financial markets worldwide.4 The higher predictability of nonlinear models not only holds for the complete set of options investigated in our sample, but also for different option buckets, such as options sorted by maturity (i.e., short-term and long-term options) and moneyness (i.e., out-of-the money, at-the-money, and in-the-money options).

Second, we inspect whether predictability of option returns through machine learning models can be exploited in an economically profitable trading strategy. Our results indicate that the long-short portfolios based on L-En’s and N-En’s forecasts of expected returns generate economically significant return spreads of 1.30% and 2.04% per month, respectively, both statistically significant at the 1% level.5 The long-short return spread of the nonlinear ensemble outperforms the return spread of the linear ensemble by a statistically significant 0.74% per month, stressing the importance of nonlinearities. This result holds also for the subset of call and put options separately, does not depend on earnings announcements, and persists over time. Moreover, the profitability of the long-short return spread of the nonlinear ensemble exceeds existing and newly proposed measures of expected return benchmarks and is robust to risk adjustments of established asset pricing models, accounting for time-varying leverage, as well as changes in the length of the training window, return frequency, and different samples of big and liquid stocks on which options can be traded. The results also remain significant across different states of the economy.

Digging deeper into the compositions of the different spread portfolios, we find that the short leg contains more puts and short-term options than the long leg. Interestingly, the short leg of the spread portfolio also displays strong differences from the other portfolios in terms of complexity of the characteristics that determine the allocation of options into portfolios. In this sense, options that are selected in the short leg are determined by the least number of characteristics, but have the highest number of nonlinearities and interaction effects among these characteristics.

Ofek, Richardson, and Whitelaw (2004) show that transaction costs in the options market are high and that these costs can substantially reduce economic profits of option-based trading strategies. Hence, to understand how far the machine learning trading strategy based on the nonlinear ensemble is implementable, we examine its profitability after accounting for transaction costs. Since actual transaction costs of trades are not observable in the OptionMetrics IvyDB database, we assume that investors have to pay 25%–100% of the quoted bid and ask spread, which we denote as the effective spread (Eisdorfer, Goyal, and Zhdanov 2022). In addition, we incorporate the costs of trading with the delta-hedging procedure by accounting for a similar percentage of the underlying’s quoted spread.6 Our results show that the returns of the nonlinear machine learning trading strategy remain sizable (0.67% per month) even if investors have to pay the full effective spread for transactions and delta-hedging on all options.7 Margins are an important consideration when investing in the options market. On top of the transaction costs arising from bid-ask spreads, we also include different margin requirements for setting up hedged long and short option positions. Realized returns and Sharpe ratios of the predictions made by the nonlinear ensemble decrease, but only turn insignificant if the investor had to pay 100% of the quoted spread for each option and delta-hedge. Importantly, the predictions by the nonlinear ensemble significantly outperform those by its linear counterpart in all cases.

As our third main empirical result, we quantify the relative importance of different characteristics for the prediction of option returns. We follow recent advances in computer science and estimate SHAP values (Lundberg and Lee 2017), which approximate changes in the model predictions had we excluded certain characteristics in its estimation. To do so, we classify our 273 option and stock predictor variables into 12 subgroups: Accruals, Industry, Investment, Profitability, Quality, Value, Contract, Frictions, Illiquidity, Informed Trading, Past Prices, and Risk. Our results reveal that the contract group contains the most important predictors, which include information about the option’s location on the underlying’s implied volatility surface. Illiquidity and risk measures follow as the second and third most important variable group, respectively. With respect to the relative importance of single characteristics, we find that implied volatility plays by far the most important role, followed by the bid-ask spread of the underlying stock and industry momentum. If we assess the functional form of the impact of the three most important single characteristics on the model-predicted delta-hedged return, our results reveal that higher implied volatility negatively affects returns, whereas higher bid-ask spreads and industry momentum predict returns positively.8

Our empirical setting enables us to answer whether option or stock characteristics are more important to accurately predict future option returns. Hence, we reestimate the machine learning models using only (i) option-based characteristics, (ii) stock-based characteristics, as well as (iii) option-based characteristics that operate on the bucket or contract level, and compare the out-of-sample forecasting results with the full information set of all option and stock characteristics.9 We observe that the models only based on a subset of information show severely lower out-of-sample R2s compared to those that incorporate all option and stock characteristics. When comparing different subsets of information, our results indicate that restricting information to only option-based characteristics yields substantially higher predictive R2s than information on only stock-based characteristics. However, the benefit of adding stock-based to option-based characteristics is substantial and helps to obtain more accurate forecasts for future option returns.

Finally, we determine possible sources of option return predictability. We hypothesize that option return predictability originates partly as a result of informational frictions, such that the information implied from stock- and option-based characteristics is not directly incorporated into option prices. To test this conjecture, we create different indices of information frictions based on stock- and option-level information. In line with our prediction, we find that the predictability of option returns increases with higher informational frictions. Our results reveal that the out-of-sample R2 for the nonlinear ensemble model equals 5.32% (0%) for options whose underlyings fall within the highest (lowest) quintile of stock-level information frictions. Options exhibiting the highest (lowest) level of information frictions show an R2 of 4.00% (1.36%). We also estimate the level of mispricing per option contract, again using a composite mispricing score. Consistent with the intuition that machine learning models manage to identify misvaluation in the options market, overall predictability is increasing in the mispricing score.

1. Related Literature

Our paper contributes to the literature on predicting and explaining the cross section of individual option returns. Dennis and Mayhew (2002) document the importance of various factors, such as beta, size, and trading volume, in explaining the risk-neutral volatility skew observed in stock option prices, whereas Bollen and Whaley (2004) investigate the relation between net buying pressure and the shape of the implied volatility function of stock options. Garleanu, Pedersen, and Poteshman (2009) theoretically model and empirically investigate demand pressure effects on option prices. By examining volatility risk in the options market, Goyal and Saretto (2009) find that options with a large positive difference between realized and implied volatility have low future delta-hedged returns. Roll, Schwartz, and Subrahmanyam (2010) examine trading volume in option markets relative to the volume in underlying stocks and relate it to contemporaneous returns. Cao and Han (2013) show that delta-hedged option returns decrease monotonically with an increase in the idiosyncratic volatility of the underlying stock. Bali and Murray (2013) find a strong negative relation between risk-neutral skewness and delta- and veganeutral equity option returns, consistent with a positive skewness preference. An et al. (2014) show that the cross section of stock returns predicts future changes in option implied volatilities. Byun and Kim (2016) find that call options written on the most lottery-like stocks underperform otherwise similar call options on the least lottery-like stocks. Christoffersen, Fournier, and Jacobs (2018) show that equity options display a strong factor structure, which is highly correlated to the volatility, skew, and term structure of S&P500 index options. Christoffersen et al. (2018) include illiquidity premia in option valuation models and Kanne, Korn, and Uhrig-Homburg (2022) find that these premia are negative (positive) if there is net buying (selling) pressure. Ramachandran and Tayal (2021) examine the impact of short-sale constraints on the pricing of options. Zhan et al. (2021) uncover return predictability in the cross section of delta-hedged equity options based on stock-based information, such as prices, profit margins, and firm profitability.

Heston et al. (Forthcoming) investigate the phenomenon of option momentum and reversal. In contrast to their study, which focuses on extracting trading signals from the insight that options that appreciated over a certain past horizon tend to continue to do so in the future, we are agnostic about which characteristics explain future option returns and instead propose a way to extract information simultaneously from 273 option and stock characteristics. Finally, Goyenko and Zhang (2021) apply machine learning techniques to analyze which characteristics drive option and stock returns.

We also extend the literature on the usage of machine learning techniques in empirical asset pricing. So far, the majority of papers apply machine learning models to predict the cross section of individual stock returns.10 Rapach, Strauss, and Zhou (2013) use Lasso in predicting market returns across countries, while Moritz and Zimmermann (2016) apply tree-based conditional portfolio sorts to examine the relation between past and future stock returns. Kelly, Pruitt, and Su (2019) apply instrumented principal component analysis (IPCA), detailed in Kelly, Pruitt, and Su (2020), to model the cross section of returns that allow for latent factors and time-varying loadings.

Gu, Kelly, and Xiu (2020) perform a comparative analysis of machine learning methods to measure equity risk premia based on a large set of stock characteristics. The authors use a broad set of stock characteristics, following Green, Hand, and Zhang (2017), whereas Murray, Xiao, and Xia (2021) focus solely on historical price data. Neuhierl et al. (2021) examine the predictive power of option characteristics for the cross section of stock returns. Kozak, Nagel, and Santosh (2020) impose an economically motivated prior on stochastic discount factor coefficients that shrinks contributions of low-variance principal components for the cross section of stock returns, and Chen, Pelger, and Zhu (Forthcoming) add to these insights, using deep neural networks to estimate an asset pricing model for individual stock returns. Martin and Nagel (2022) show that asset returns may appear predictable in-sample when analyzing the economy ex-post and stress the importance of out-ofsample tests. Feng, Giglio, and Xiu (2020) propose a new model selection method that accounts for model selection mistakes that produce a bias due to omitted variables, and Lettau and Pelger (2020) construct a new estimator that generalizes principle component analysis by including a penalty on the pricing error in expected returns. A nonparametric method to dissect characteristics based on the adaptive group Lasso is proposed by Freyberger, Neuhierl, and Weber (2020). Giglio, Liao, and Xiu (2021) perform “thousands of alpha tests” to develop a new framework to rigorously perform multiple hypothesis testing in linear asset pricing models. Grammig et al. (2020) contrast theory-based and machine learning methods for measuring stock risk premia.

The aforementioned studies have mainly focused on the cross section of U.S. stocks. Leippold, Wang, and Zhou (2021), instead, employ machine learning algorithms to analyze return prediction factors in the Chinese stock market. Recent research also expands the application of machine learning models for the prediction of other asset classes: Kelly, Palhares, and Pruitt (Forthcoming) propose a conditional factor model for corporate bonds returns based on the IPCA approach. Hull, Li, and Qiao (2023) build a predictive model of breakeven implied volatilities of S&P500 index options. Bali et al. (2021) find that machine learning models substantially improve the out-ofsample performance of stock and bond characteristics when predicting the cross section of corporate bond returns. Bianchi, Büchner, and Tamoni (2021) apply similar techniques to Treasury securities, whereas Filippou et al. (2020) employ them in the context of exchange rates. DeMiguel et al. (2021) show that machine learning helps to select future outperforming mutual funds and Wu et al. (2021) establish similar conclusions for hedge funds. Finally, Li and Rossi (2020) apply machine learning to select mutual funds on the basis of their exposure to a large set of stock characteristics.

Our paper is the first to predict the cross section of individual option returns using a large set of linear and nonlinear machine learning models and the largest set of characteristics to date. We also provide a comprehensive investigation of the economic underpinnings of option return predictability and offer important insights into the cross-sectional pricing of equity options with machine learning and big data. Finally, we introduce an expected return benchmark for delta-hedged options based on the instrumented principal component analysis and show that an ensemble of nonlinear machine learning models produces abnormal economic and statistical profits to the benchmark.

2. Importance of Nonlinearities and Interactions for Predicting Option Returns

The existent academic literature on financial derivatives does not provide clear theoretical guidance on what delta-hedged expected returns should look like. In its simplest form, the Black and Scholes (1973) world, delta-hedged excess returns should be zero. In this section, we shortly discuss what to expect when predicting delta-hedged option returns with characteristics and why we believe that nonlinearities and interactions between predictors should matter in this task.

First, both (delta-hedged) call and put options are financial assets whose payoff structure depends on whether the respective underlying is above or below a predefined strike price. As a consequence, the returns of these securities are characterized by strong nonlinearities. Table 1 documents that delta-hedged option returns display high absolute skewness and kurtosis as well as significantly fail the Jarque-Bera test for conforming to a normally distributed random variable.

Second, it has been known since Heston (1993) that option prices have nonconstant implied volatilities, so that log returns are not normally distributed under the risk-neutral measure. Thus, option prices may be significantly affected by time-varying skewness and kurtosis. The practical consequence is that using the Black-Scholes formula that assumes zero skewness and excess kurtosis distorts option prices. To address departures from normality in the empirical return distribution, earlier studies rely on nonlinear Gram-Charlier expansions that allow for additional flexibility over a normal density by introducing adjustments for observed skewness and kurtosis of the assumed distribution.11 Thus, standard linear models might be unable to capture significant departures from normality in option return distribution. Nonlinear models, on the other hand, provide a more accurate characterization of the skewed, fat-tailed distribution of option returns.

Third, we expect that interactions between variables—in addition to nonlinearities—should matter for predicting future delta-hedged option returns. As an example, Black-Scholes option prices (with constant volatility estimates) will be distorted when implied volatility jointly varies with different predictor variables. To understand how interactions can affect the prediction of option returns, we examine the monthly aggregate time-series of options’ implied volatilities and the underlyings’ idiosyncratic volatility in our sample. Figure IA1.1 in the Internet Appendix plots the rolling correlation (over a time horizon of 60 months) between both variables.

Over our sample period, the average correlation between these two variables is close to zero (−0.03). However, we observe that the correlation fluctuates over time between a minimum of −0.70 and a maximum value of 0.70. As a consequence, it is likely not only that each set of characteristics predicts future delta-hedged option returns, but that it is the interaction between them that particularly matters. Previous research on the topic supports this idea: Ramachandran and Tayal (2021) document a monotonic relation between short-sale constraints and delta-hedged put returns for overpriced stocks. Hence, the documented predictability is an interaction between three characteristics: the level of mispricing of the underlying stock, the level of short-sale constraints of the underlying stock, and if the option is a put or a call option.

Fourth, results from asset pricing theory show that “realistic” preferences of the representative investor lead to a nonlinear pricing kernel for the cross section of asset returns. More specifically, Pratt and Zeckhauser (1987), Kimball (1993), and Dittmar (2002) show that a nonlinear pricing kernel is the outcome if an investor’s four derivatives of the utility function have altering signs about final wealth, that is, investors show nonsatiation, they are risk-averse, they like skewness, and they dislike kurtosis. Using a different theoretical setup, Bekaert, Engstrom, and Xu (2021) document that time-varying risk aversion and aversion to economic uncertainty of investors translate into expected asset returns that can only be poorly captured by assuming linearity of the pricing kernel. Supporting these theoretical considerations, Büchner and Kelly (2022) find that unconditional factor exposures appear to be nonlinear in the moneyness domain of S&P500 index option returns. Hence, we also expect that forecasts for individual option returns should take into account nonlinearities and interactions between predictor variables to accurately translate investor preferences.

Due to all these reasons, machine learning methods, which allow for nonlinearities and interactions of predictor variables, seem particularly promising for forecasting delta-hedged option returns and for deriving trading strategies based on predictability. In our empirical analysis, we also propose an adequate benchmark for expected option returns. Precisely, we compare our results to (i) a newly constructed benchmark based on instrumented principal component analysis, (ii) forecasts based on univariate predictions, and (iii) different asset pricing models consisting of stock- and option-based risk factors.

3. Methodology and Performance Evaluation

In its most general form, we express future option returns as the sum of expected returns and the error term with zero mean:

$$\tag{1}$$

The central element we aim to estimate is a functional representation g(zi,s,t ), which links expected future returns Et [ri,s,t+1] to characteristics zi,s,t of option i on underlying s:

$$\tag{2}$$

Methods considered: Following the growing literature on machine learning algorithms for predicting asset returns (Gu, Kelly, and Xiu 2020), we compare a variety of machine learning methods with increasing complexity, and contrast the implications of linear and nonlinear models. For penalized linear models, we consider Lasso (Tibshirani 1996), Ridge (Hoerl and Kennard 1970), and Elastic Net regressions (Zou and Hastie 2005 ENet). For linear dimension reduction techniques, we use principal component (PCR) and partial least squares regressions (PLS). To model nonlinearities, we consider tree-based methods: random forests (Breiman 2001 RF), gradient boosted tree regressions (see Friedman 2001 GBR) and gradient boosted tree regressions with dropout (Vinayak and Gilad-Bachrach 2015 Dart), as well as deep feed-forward neural networks (FFN) as universal function approximators (Hornik, Stinchcombe, and White 1989). Internet Appendix IA3 provides a detailed description of these methods.

Forecast ensembles: We furthermore form ensembles of the five linear (L-En) and the four nonlinear models (N-En) to combine the predictive power of multiple models (Goyal and Welch 2008). We consider a simple ensemble that equally weights the predictions of each method included. Building on the insights of Bates and Granger (1969), Rapach, Strauss, and Zhou (2010) document large benefits in economic forecasts using this type of ensemble. Denote the forecast of a given model by rˆi,s,t+1. Then, the ensemble forecasts for t + 1 will be:

$$\tag{3}$$

where contains the target models and J denotes the number of models in the set.

The design decision to create an ensemble of linear and one of nonlinear models allows us to directly analyze the informativeness of modeling nonlinear interactions and combine the predictive power of multiple methods.

Assessing predictive power: We use the standard out-of-sample R2 statistic to gauge the predictive power over single-equity option returns (Gu, Kelly, and Xiu 2020):

$$\tag{4}$$

R2OS measures the reduction in the mean squared forecast error (MSFE) compared to a naive benchmark of zero excess returns for all options.

We evaluate the predictive power on a testing sample, which is disjoint from the data used to estimate the model parameters and hyperparameters (such as the magnitude of the Lasso penalty). More specifically, we start by estimating model parameters on a training sample 1 of 5 years (January 1996–December 2000). We then perform an extensive hyperparameter optimization validating the method’s fit in the next 2 years 2 (2001–2002). Lastly, for each method, we use an equal-weighted ensemble of the eight models with those hyperparameter combinations yielding the best fit in the validation sample to assess the predictive power in the 1-year testing sample 3 (2003). We keep the models fixed for 1 year and replicate this procedure extending the number of years in the training sample by 1 year in each iteration, for a total of 18 out-ofsample years (2003–2020). Internet Appendix IA4 details the procedure we use to estimate the models, the libraries used for each model type, and the setup of the hyperparameter optimization.

In cross-sectional asset pricing tests, our main objective is not to forecast time-series variation in future returns, but rather cross-sectional return spreads in the testing sample. To focus on this cross-sectional variation, Han et al. (2021) propose a cross-sectional out-of-sample R2,

$$\tag{5}$$

which effectively compares the resulting cross-sectional return spread of a candidate model, (rˆi,s,t+1 − ¯rˆi,s,t+1), with the realized return spread in the testing sample, (ri,s,t+1 − ¯ri,s,t+1).

We test the statistical significance of each model’s forecasts, following Clark and West (2007), by comparing the resulting forecasts with a naive benchmark of always forecasting an excess return of zero:

$$\tag{6}$$

where c¯ (j ) and σˆ (j ) c denote the time-series average and Newey and West (1987) standard error of the mean difference between squared forecast errors:

$$\tag{7}$$

Here, nT3 is the number of observations in the testing sample and eˆ (j )i,s,t+ ei,s,t+1 is the forecast error on option i on underlying s at time t +1 for method j . We use 12 lags for the standard errors, coinciding with the number of months we keep model parameters fixed for each slice of the testing sample.

Forecast comparison: To compare the forecasts of two methods, we use the modified Diebold and Mariano (1995) (DM) test, which accounts for potential cross-sectional dependence in equity option returns.12

The DM test statistic for a comparison between methods 1 and 2 is defined as:

$$\tag{8}$$

where d¯(1,2) and denote the time-series average and Newey and West (1987) standard error of the mean difference between squared forecast errors d (1,2):

$$\tag{9}$$

We also use correlations as a secondary method to assess the similarity between the forecasts of different methods. Formally, the forecast correlation is defined as:

$$\tag{10}$$

where σ (eˆ (1)t+1) and σ (eˆ (2)t+1) denote the standard deviations of forecast errors for models 1 and 2, respectively, and Cov(eˆ (1)t+1,eˆ (2)t+1) (1) ^(2) denotes their covariance. Divergence in the forecasting power of two methods provides a high-level view on why some methods outperform.

4. Data and Variable Definitions

We first outline the data sources used and then provide summary statistics for the option sample and the sample of underlying optionable stocks. Our primary data source is OptionMetrics IvyDB, which provides historical prices for all U.S. single equity options. We also use the interpolated volatility surface data from OptionMetrics. Whereas option returns are calculated using historical option prices, the interpolated volatility surface data are only used for constructing option-based characteristics. Due to the starting date of this database, our sample covers the period from January 1996 through December 2020.

Historical prices and accounting data for underlying stocks are obtained from CRSP and Compustat. We retain only underlyings with share codes 10 or

11 and exchange codes 1, 2, 3, 31, 32, and 33, that is, stocks listed on the NYSE, NYSE American (formerly AMEX), or NASDAQ. Contrary to previous studies (see Zhan et al., 2021), we purposely do not remove stocks with nominal prices below \$5 per share, as Eisdorfer, Goyal, and Zhdanov (2022) find that options trading on stocks with a low nominal price tend to be overpriced. Information on stock splits and dividend payments is taken from OptionMetrics and crosschecked with CRSP. We match these databases using the linking algorithm developed by WRDS. Daily risk-free rates are taken from Kenneth French’s online data library.13

Option returns are notoriously noisy, especially for underlyings with few outstanding option contracts and relatively low option trading activity. We therefore rely on a variety of standard filters established in the literature to assure the consistency of our analyses (Goyal and Saretto, 2009, Cao and Han, 2013, Zhan et al., 2021).14 First, we exclude all options for which OptionMetrics does not provide an implied volatility and Greeks. Second, we disregard options on stocks that have a dividend scheduled during the investment period. Third, we eliminate options with zero volume over the last 7 calendar days. Fourth, to avoid any biases due to microstructure noise, we remove options for which the bid price is zero, the ask is smaller or equal to the bid, the mid price is below \$0.125, or the relative bid-ask-spread is above 50%. As a fifth step, we make sure that American option bounds are not violated. Sixth, we retain only options with a standard third-Friday expiration, such that we exclude short-term options with less than 2 weeks to expiration. Finally, we check for the convexity of option prices per underlying, following Bollerslev, Todorov, and Xu (2015). Specifically, we retain only those observations for a given maturity τ , for which the difference between the prices of two neighboring call (put) options with strike price K1 < K2 is ≤ 0 (≥ 0).

4.1 Option returns

Our main variable of interest is the excess return of buying an option that we delta-hedge on a daily rebalancing schedule. We consider delta-hedged option gains, following Bakshi and Kapadia (2003), as the value of a selffinancing portfolio consisting of a long option, hedged by a position in the underlying such that the portfolio is locally immune to changes in the stock price. To establish notation, consider the partition = {t = t0 < ··· < tN = t +τ } of the interval from t to t +τ . Assume that the long option position is hedged discretely N times at each of the dates tn,n = 0,...,N −1 . The discrete delta-hedged option gain over the period [t,t +τ ] is then given by:

$$\tag{11}$$

where Vt denotes the price of the option at time t ; rn is the risk-free rate at tn; an is the number of calendar days between rehedging dates tn and tn+1, which we set to a =1; and is the observed delta of the option. We consider gains for investment horizons of 1 calendar month, or until maturity if the option expiration falls within the investment month.15 When an individual stock exhibits large price movements over the investment horizon, establishing initial delta-neutrality may still expose the investor to substantial sensitivity to future movements in the underlying stock. Tian and Wu (Forthcoming) estimate that one-time delta-hedging at initiation removes an average of about 70% of the directional risks embedded in the option, whereas daily deltahedging manages to eliminate upwards of 90% of these risks. Hence, we opt for delta-hedging at the end of each trading day. Moreover, daily delta-hedging enables us to understand how characteristics relate to the pricing of higherorder risks embedded in options, detached from predictors for stock returns. Finally, we define option returns, following Cao and Han (2013) and Zhan et al. (2021), as:

$$\tag{12}$$

4.2 Summary statistics

After the data filters discussed earlier, our sample comprises 4,867,767 options on 6,942 unique underlyings, for a total of 11,983,005 option-month observations for the period January 1996–December 2020. Our sample is made up of roughly 54% call and 46% put options. Panel A of Table 1 shows that the average monthly delta-hedged option return is 0.17%, whereas the median monthly delta-hedged option return is −0.37%. The average moneyness is 1.03 and the average implied volatility is 47.99%. The average (median) days to maturity are 173 (113), while every fourth option exhibits a time-to-maturity of less than 50 days. As panels B and C depict, the median monthly deltahedged option returns are slightly positive for call options at an average 0.44% per month, but strongly negative for put options, −0.13%. The median return, however, is negative for both puts and calls. Panel D in Table 1 shows summary statistics for the years of 1996 through 2002, which are used in the training step of the machine learning models, while panel E gives the summary statistics only for the testing subsample from 2003 through 2020. Average monthly deltahedged option returns tend to be slightly more negative for the more recent time period. Moreover, implied volatility and moneyness are lower and days to maturity higher for the testing period from 2003 to 2020.

Table IA7.1 in the Internet Appendix reports summary statistics of the 6,942 stocks in our sample. Our sample includes on average 1,706 optionable stocks per month, which comprise 76% of the total market capitalization of the U.S. equity market. Moreover, our sample comprises large stocks with representative volatility, given that the average size and volatility percentile within the total stock universe are 71 and 45, respectively. Finally, the Fama-French 12-industry distribution in our sample is comparable to the total stock sample, as is evident from panel C in Table IA7.1.

4.3 Option and equity characteristics

Throughout this paper, we differentiate between different parts of the time-tomaturity and moneyness domain of options, which we refer to as “buckets.” Specifically, we separately consider predictability for short- and long-term options (≤ vs. > 90 days to maturity), in-the-money (ITM: mstand > 1 for puts, mstand < 1 for calls), and out-of-the-money (OTM: mstand < 1 for puts, mstand > 1 for calls) calls and puts, and at-the-money options (ATM: −1 ≤ mstand ≤ 1), as well as time-to-maturity and moneyness combinations. The moneyness buckets are based on standardized moneyness, that is, mstand = log(K/S)/(σ atm τ ), where σ atm is the at-the-money implied volatility for time to maturity τ . The moneyness and time-to-maturity of option contracts change rapidly. Therefore, it is unreasonable to assume that flow measures, such as option volume, derived from a particular option contract over a historical period will be valid for the same contract in the next month. The defined buckets allow us to compute these flow measures for option contracts, as we abstract from the impact of changing moneyness and fleeting time-to-maturity. Table IA7.3 in the Internet Appendix shows that, on average, 15 long term options and 11 short term options belong to each underlying stock per month, with the majority located at the current price of the underlying.

We build a comprehensive set of option-based characteristics, motivated by earlier studies on the cross-section of option and/or stock returns. Out of the 80 we compute, 43 characteristics operate on the level of the underlying (e.g., iv-rv-spread; Goyal and Saretto 2009), 20 on the level of option buckets (e.g., the Amihud 2002, illiquidity measure), and 17 on the level of individual option contracts (e.g., the option’s time-to-maturity). Internet Appendix IA5 provides a detailed description of the option-based characteristics.

As we are also interested in the performance of stock-based characteristics for predicting option returns, we include the 94 stock characteristics proposed by Green, Hand, and Zhang (2017). We enrich this set by adding 90 industry dummies, based on the first two digits of the SIC code, four seasonal returns for each underlying (Heston and Sadka 2008; Keloharju, Linnainmaa, and Nyberg 2016), the bear-beta factor proposed by Lu and Murray (2019), and the previous period’s return. Finally, we add stock-based characteristics motivated by the literature on the cross section of option returns, but that are not included in Green, Hand, and Zhang (2017). These comprise default risk (Vasquez and Xiao 2021), the underlying’s close price (Eisdorfer, Goyal, and Zhdanov 2022), and realized volatility (Cao et al. 2019).16

Table 1 Delta-hedged option return summary statistics

	Mean	Sd	10-Pctl	Q1	Q2	Q3	90-Pctl	Skew	Kurt	JB
A:All Options (N=12,136,401)
Delta-Hedged Return	0.11	602.77	-4.45	-1.98	-0.37	1.26	4.46	1.31	10.81	0.0
Days to Maturity	172.88	179.68	21.0	50.0	113.0	204.0	444.0	1.84	3.07
Moneyness	1.03	0.37	0.77	0.89	1.0	1.11	1.29	2.66	17.06
Implied Volatility	47.89	26.17	23.48	30.23	41.25	58.18	80.48	1.81	5.05
Absolute Delta	0.46	0.25	0.13	0.25	0.45	0.66	0.82	0.17	-1.01
B:Call Options (N=6,559,893)
Delta-Hedged Return	0.35	819.87	-4.91	-2.08	-0.33	1.52	5.22	1.26	11.59	0.0
Days to Maturity	176.75	182.37	21.0	50.0	113.0	204.0	449.0	1.81	2.91
Moneyness	1.06	0.28	0.82	0.93	1.03	1.15	1.33	2.17	11.13
Implied Volatility	47.37	24.95	23.27	30.06	41.12	57.96	79.62	1.58	3.52
Absolute Delta	0.51	0.24	0.18	0.32	0.51	0.71	0.84	-0.03	-1.0
C:Put Options (N=5,576,508)
Delta-Hedged Return	-0.17	4.37	-3.99	-1.88	-0.41	1.0	3.62	1.46	8.87	0.0
Days to Maturity	168.33	176.35	21.0	50.0	112.0	204.0	417.0	1.86	3.18
Moneyness	0.99	0.45	0.72	0.85	0.96	1.06	1.22	3.69	30.27
Implied Volatility	48.51	27.51	23.73	30.43	41.39	58.43	81.57	2.05	6.72
Absolute Delta	0.4	0.25	0.1	0.2	0.36	0.58	0.78	0.47	-0.75
D: All Options 1996-2002 (N=2,027,277)
Delta-Hedged Return		1.34 1474.68	-5.96	-2.41	-0.19	2.31	7.02	1.1	11.63	0.0
Days to Maturity	152.63	174.13	22.0	50.0	108.0	176.0	351.0	2.36	5.69
Moneyness	1.1	0.54	0.78	0.9	1.02	1.17	1.42	3.99	28.44
Implied Volatility	65.39	29.68	32.22	42.95	60.67	82.31	103.98	0.99	1.49
Absolute Delta	0.49	0.24	0.18	0.31	0.48	0.68	0.83	0.11	-0.94
E: All Options 2003-2020 (N=10,109,124)
Delta-Hedged Return	-0.14	9.32	-4.19	-1.92	-0.4	1.1	3.97	1.3	9.54	0.0
Days to Maturity	176.94	180.5	21.0	50.0	113.0	206.0	449.0	1.74	2.62
Moneyness	1.02	0.33	0.77	0.89	0.99	1.1	1.27	1.97	11.16
Implied Volatility	44.38	23.91	22.7	28.9	38.77	52.96	71.62	2.15	7.77
Absolute Delta	0.45	0.25	0.13	0.24	0.44	0.65	0.82	0.19	-1.02

The table reports descriptive statistics for delta-hedged option returns. Panel A reports statistics of returns and option characteristics over the period from 1996 to 2020. Delta-hedged option returns are measured over a period of 1 calendar month, or until option maturity. Delta-hedging is performed daily. Days to maturity is defined as the number of calendar days until option expiration. Moneyness is the ratio between the underlying’s stock price and the option’s strike price. Option implied volatility is provided by OptionMetrics. Absolute delta follows the model of Black and Scholes (1973). Skew denotes skewness. Kurt denotes excess kurtosis. JB denotes the p-value in percent of the Jarque-Bera test if delta-hedged option returns conform to a normal distribution. Panels B and C depict statistics for call and put options, respectively, whereas panel D reports summary statistics for the period 1996 to 2002 used exclusively in the training step of the models. Panel E reports summary statistics for the entire out-of-sample period, comprising the years 2003 through 2020.

We are left with a total of 273 characteristics, which can be broadly classified into 12 groups. Accruals, Industry, Investment, Profitability, Quality, and Value exclusively include stock-based characteristics and loosely follow the classification proposed in Jensen, Kelly, and Pedersen (Forthcoming) and Green, Hand, and Zhang (2017). Classification group Contract contains five option-based characteristics, the time-to-maturity, moneyness, implied volatility, and put and call identifier, and thus combines information about the location of the respective option on the underlying’s implied volatility surface. We therefore assume that these characteristics play a pivotal role in predicting future option returns as a sort of reference point, much as the market return in traditional factor models (Fama and French 1993). Groups Frictions, Illiquidity, Informed Trading, Past Prices, and Risk contain both stock- and option-based information and highlight the predictions’ dependence on informational frictions. Internet Appendix IA6 provides a detailed list of all 273 characteristics, their origin from the literature, the primary information source (option- vs. stock-based), and the feature group to which we have assigned them.

5. Predicting Option Returns

5.1 Predictability comparison

Figure 1 shows the out-of-sample R2 for the pooled testing sample from January 2003 through December 2020 using the nine machine learning methods and the two ensembles. Nonlinear models routinely outperform the predictability uncovered by linear models for option returns. The R2 for the linear models on the full sample are all negative, ranging from −0.61% for PLS to −0.18% for elastic net regressions. The Clark and West (2007) test statistics indicate that none of these predictions outperform a naive benchmark of predicting delta-hedged returns of zero.17 Nonlinear models, in contrast, manage to uncover substantial predictability in single-equity option returns. GBR and Dart generate the highest out-of-sample R2, at or above 2% for the pooled sample. All but FFN generate forecasts statistically better than the naive benchmark.

In addition to the full sample, we examine predictability for future call and put option returns separately. For most models, we find that forecasts of future put returns yield higher R2OS. The best call and put return predictions are both made by GBR, with R2OS ≥ 2.0%; the worst call predictions are generated by Lasso; and the worst put predictions by Ridge. Linear models fail to adequately uncover a relation between characteristics and future returns. While FFN is the most promising model in Gu, Kelly, and Xiu (2020) for stock returns, we find that it uncovers low predictability in the case of delta-hedged option returns. Given that FFN is the method with the highest potential complexity, this suggests that the complex structure does not generalize well to the testing sample of option returns. In contrast, tree-based methods tend to outperform, such that histogram-based estimation including nonlinear interactions trumps model complexity in this market.18

Figure 1 model comparison
linear (L-En) and nonlinear (N-En) ensemble methods. We separately document the predictive power for all options and for calls and puts. ***, **, * below the bars denotes statistical significance at the 0.1%, 1%, and 5% level as defined in Equation (7) for the sample of “all” options. The testing sample spans the years 2003 through 2020.

Ensembles have been shown to improve the accuracy and consistency of the predictions and are a staple in modern machine learning estimation (e.g., Krizhevsky, Sutskever, and Hinton 2012; Lakshminarayanan, Pritzel, and Blundell 2017).19 The linear ensemble L-En produces significantly better forecasts than any of the individual linear models, notably managing to produce positive R2OS for call and put returns. The predictability uncovered is comparable to that of FFN, but again not statistically significant. The level of predictability at roughly 0.9% is about twice as high as the levels of predictability of stock returns found by Gu, Kelly, and Xiu (2020). Within the class of nonlinear methods, no model outperforms the ensemble, highlighting the importance of averaging predictions of multiple methods. Like all treebased methods, the resulting predictions comfortably outperform the naive benchmark of zero excess returns.

Figure 2
considered, as well as the linear (L-En) and nonlinear (N-En) ensemble methods. We separately document the predictive power for all options and for calls and puts. ***, **, * below the bars denotes statistical significance at the 0.1%, 1%, and 5% level, as defined in Equation 7 for the sample of “all” options. The testing sample spans the years 2003 through 2020.

We are ultimately interested in how far the models uncover cross-sectional return spreads in our option sample. For this, Figure 2 compares the crosssectional R2OS;XS defined in Equation 5. Interestingly, while none of the individual linear models generated predictability on the full sample, especially penalized regressions are able to generate realistic return spreads. For Lasso and ENet, these are statistically significant at the 5% and 1% level, respectively. The cross-sectional pricing power of all nonlinear models is highly significant, and, surprisingly, FFN manages to produce R2OS;XS comparable to the treebased methods, despite failing to adequately predict the average level of future returns. This exercise clearly highlights the benefits of using ensemble models in that the cross-sectional predictive power increases both for L-En and N-En. The nonlinear ensemble generates the highest cross-sectional predictability for all subsamples considered with an R2OS;XS of 3.2%.

Most recent studies overlook the ability to zoom in on the predictive power of the models considered and provide an intuition for how stable the resulting predictions are. The focus mostly lies on pooled out-of-sample predictability, as in Figures 1 and 2. Instead, we also show the dispersion of R2OS and R2OS;XS over time. While the pooled approach weights the predictions of each year by the number of option-month observations contained, we now provide estimates of the predictive power per year, which allows us to investigate the stability of the forecasts over time. Stable predictability is imperative for investors who wish to use the model forecasts in their investment decisions.

Figure 3
Time-series dispersion of R2 and R2 OS XS
The figure shows the dispersion of annual out-of-sample R2 defined in Equation 4 in the upper panel and crosssectional out-of-sample R2 defined in Equation 5 in the lower panel. We show the 5th and 95th percentile R2 in the whiskers, the interquartile range in the boxes, as well as the mean (circles) and median (bar).

The upper panel of Figure 3 adds three main insights to the pooled R2OS consideration above. First, the predictive power of all models fluctuates significantly over time. Linear models produce the largest dispersion, with the possibility of highly negative R2. Second, most models exert an interquartile range of their predictive power that is either symmetric around the median or more exposed to the downside. Notable exceptions to this are GBR and Dart, which manage to produce the best predictability most consistently, with an interquartile range of 1.5% to 4.0%. Leveraging the informational content uncovered by these tree-based models potentially grants large benefits to investors. Following this intuition, we find that ensemble models produce more stable forecasts. The minimum to maximum predictability for L-En ranges from −7.6% to +4.4% and for N-En between −0.7% and +6.9%. The forecasts of N-En are the most stable over time, producing significant predictive power for all years in the testing sample. Cross-sectional predictability is much less disperse (lower panel of Figure 3) and is generally increasing in the model complexity and the ability to model nonlinear interaction effects between characteristics. N-En is once more the most stable model, with the highest median and mean level of predictability over time.

Table 2 Forecast comparison

	Lasso	ENet	PCR	PLS	L-En	GBR	RF	Dart	FFN	N-En
A: Diebold and Mariano (1995) forecast comparison
Ridge	1.21	1.91	-0.02	-0.36	3.29	2.87	2.21	2.01	3.58	3.53
Lasso		1.67	-0.36	-0.67	2.43	2.27	1.68	1.59	3.06	2.78
ENet			-0.56	-0.89	2.20	2.22	1.56	1.52	2.06	2.74
PCR				-1.34	2.26	6.68	7.27	3.11	1.24	7.25
PLS					2.79	7.15	9.48	3.47	1.58	8.14
L-En						2.03	1.04	1.02	-0.23	2.79
GBR							-2.51	-0.53	-1.30	1.12
RF								0.57	-0.74	3.36
Dart FFN									-0.83	0.86 1.65

	B:Forecast correlation
Ridge Lasso	0.80	0.84	-0.03 -0.02	0.53	0.91 0.91	0.48 0.41	0.45 0.40	0.47 0.39	0.77 0.78	0.61 0.56
ENet		0.95		0.51 0.53	0.92	0.43	0.42	0.41	0.78	0.57
			-0.02				0.27	0.15	0.03	0.17
PCR				0.25	0.13	0.18	0.54	0.52	0.58
PLS					0.76	0.51	0.54	0.53	0.83	0.61
L-En						0.54	0.86	0.91	0.59	0.68 0.95
GBR								0.79	0.57	0.89
RF									0.58	0.93
Dart FFN										0.77

Panel A of the table shows Diebold and Mariano (1995) test statistics, following Equation 8, for the nine models and two ensembles considered in the paper. A positive number indicates that the model in the column outperforms the row model. If it is highlighted in light blue (blue), this outperformance is statistically significant at the 1% level (5% level). Panel B shows forecast correlations defined in Equation 10. Here, highlighting in light blue (blue) denotes large values with a cutoff at 90% (70%).

Forecast comparison. We now turn to Diebold and Mariano (1995) tests to iteratively compare the forecasts of two competing models following Equation 8. Statistical significance at the 1% (5%) level is highlighted in light blue (blue). The first row of Table 2 shows that only L-En manages to statistically outperform the predictions made by Ridge regressions within the class of linear models. In comparison, all nonlinear models manage to beat the predictions by Ridge. Overall, we find that the forecasts by L-En are significantly better than the forecasts any of the other linear models produce, highlighting the necessity of adequately pooling the forecasts by multiple methods.

Within the class of nonlinear models, only GBR manages to outperform L-En comfortably, with a t -statistic of 2.03. Interestingly, the forecasts of GBR and Dart, which have produced the highest single-model predictability, are indistinguishable from one another. FFN manages to outperform Ridge, Lasso, and ENet, but none of the nonlinear models.20 The nonlinear ensemble N-En produces forecasts that beat any of the other models but Dart and GBR, which are its most vital inputs. Comparing the performance of L-En and N-En, we find that the forecasts of the latter are more informative, with a t -statistic of 2.79. While the forecasts of GBR and Dart do not manage to statistically beat those of FFN, N-En manages to do so at the 10% level (t-stat = 1.65).

The forecast correlation analysis in panel B of Table 2 confirms these insights. First, penalized linear regression methods yield similar predictions, which is expected given that Lasso and Ridge are special cases of ENet. Notably, L-En produces forecasts highly correlated with either of these methods (ρ >90%), as does FFN (ρ >75%), suggesting that the underlying process proposed by FFN is quite similar to a regularized linear function. Second, tree-based methods are a class of their own, showing correlations of ρ >75% only among themselves. Given their unique setup of identifying quantile splits in the input characteristics to relate to option returns, this is not surprising. At the same time, these methods, especially boosted treebased methods (GBR and Dart), outperform all other models. Consequently, predictions by N-En share many of their properties (ρ >90%). However, the ensemble predictions are also highly correlated with the two other nonlinear methods, with ρN-En,FFN = 77% and ρN-En,RF = 89%.

Impact of Nonlinearities. The results underscore the usefulness of forming forecast ensembles of many models. Therefore, from now on, we will compare the linear (L-En) and nonlinear (N-En) ensemble methods to understand the implications of modeling nonlinear interaction effects.

In this section, we highlight the usefulness of modeling nonlinear interaction effects between option characteristics. The left panel of Figure 4 compares the monthly R2 for N-En in dark blue and with L-En in light blue. The figure provides multiple insights: First, both ensembles tend to beat a naive benchmark of predicting zero delta-hedged returns in most months. Second, N-En is less prone to experience predictability crashes. While L-En does a poor job of predicting future option returns during and after turbulent times (2008–2012 for example), N-En predicts returns much more consistently. Third, for N-En, the predictive power has stayed roughly constant over time, confirming not only that nonlinear models enlarge the information embedded in option characteristics, but also that the resulting predictability patterns are highly persistent over time.21 At the same time, this suggests that our methods pick up more than just plain mispricing, especially in recent years, under the assumption that modern options markets have become informationally more efficient.

The panel on the right shows a scatter plot of the resulting R2OS of L-En on the x-axis and N-En on the y-axis. The green-shaded area represents a

Comparing linear and nonlinear ensembles Comparinglinearand nonlinearensembles

Figure 4

The left panel of the figure shows monthly R2OS for the testing sample from 2003 through 2020 for the linear L-En on the x-axis and for N-En and the y-axis. The green-shaded area represents a relative outperformance in terms of predictability for N-En, while the red-shaded area represents the opposite. The red circles represent the COVID-19 selloff and subsequent recovery from December 2019 through December 2020.

relative outperformance of the nonlinear ensemble, whereas the red-shaded area represents the opposite. For 69.8% of the months in our sample, we find that N-En outperforms L-En and if so, quite comfortably. In the figure, we have also highlighted in red circles the period of December 2019 through the end of our sample in December of 2020, which are the months surrounding the COVID-19 selloff in February and March 2020, and the subsequent recovery. Nonlinear ensembles were better able to deal with this huge exogenous shock, with a performance on par with or exceeding that of L-EN.22 The R2 for N-En dips slightly below zero only in January and February and reaches precrisis levels of 4.8% afterwards. While the pandemic-driven selloff constituted a large exogenous shock to financial markets, the relation between option characteristics and future returns quickly went back to normalcy after the initial reaction. This speaks in favor of using nonlinear machine learning methods to identify persistent predictability patterns in the options market, especially given that the sample we used to train the models to uncover predictability patterns in option returns during the COVID-19 selloff ended in December of 2017. In Figure IA8.1 in the Internet Appendix, we repeat the exercise using the crosssectional out-of-sample R2OS;XS defined in Equation 5. N-En beats the linear ensemble in 86.0% of the months, with a positive R2OS;XS in more than 90% of months.

Table 3 Trading on machine learning predictions

	L-En				N-En
	Pred	Avg	SD	SR	Pred	Avg	SD	SR	N vs.L
Lo	-1.351	-1.087	1.398	-0.778	-1.730	-1.649	1.942	-0.849	***
2	-0.775	-0.528	1.535	-0.344	-0.781	-0.700	1.529	-0.458
3	-0.542	-0.365	1.434	-0.255	-0.460	-0.415	1.368	-0.303
4	-0.369	-0.259	1.466	-0.177	-0.280	-0.266	1.255	-0.212
5	-0.224	-0.196	1.497	-0.131	-0.155	-0.166	1.295	-0.128
6	-0.092	-0.122	1.509	-0.081	-0.050	-0.119	1.325	-0.090
7	0.038	-0.061	1.494	-0.041	0.052	-0.075	1.425	-0.053
8	0.174	-0.027	1.510	-0.018	0.166	-0.031	1.491	-0.021
9	0.337	0.046	1.436	0.032	0.324	0.090	1.555	0.058
Hi	0.637	0.216	1.485	0.146	0.791	0.391	1.835	0.213
H-L	1.988	1.303	1.270	1.026	2.521	2.040	1.598	1.277	***
		***		***		***		***
call	1.864	1.400	1.614	0.867	2.596	2.290	1.941	1.180	***
put	1.943	1.232	1.274	0.967	2.264	1.971	1.663	1.185	***

The table shows the returns to option portfolios sorted by the predictions made by the linear (L-En) and nonlinear ensemble (N-En) methods. Each contract is weighted by its dollar open interest at the time of investment. Pred denotes the average predicted return within the respective portfolio; Avg the average realized return; SD the standard deviation of realized returns; and, finally, SR the realized Sharpe ratio. All values are given per month. The last column (N vs. L) gives the significance of comparing the mean realized returns for N-En and L-En. ***, **, * correspond to significance at the 1%, 5%, and 10% level, respectively.

5.2 Machine learning portfolios

To gauge whether the predictability of machine learning methods is also economically significant, we follow Gu, Kelly, and Xiu (2020) and form portfolios using machine learning forecasts. Specifically, each month, we sort individual equity options into 10 decile portfolios based on the machine learning models’ (L-En and N-En) expected return forecasts. Then, we calculate the 1-month-ahead average realized return of individual equity options in each decile. Finally, we compute the average long-short portfolio return of a zero-net investment portfolio by buying options with the highest expected return forecast (decile 10) and financing this investment by writing options with the lowest expected return forecast (decile 1). Each contract is weighted by its dollar open interest at the time of investment.

Table 3 shows the average predicted and 1-month-ahead realized portfolio returns. For both ensemble classes, the average predictions are slightly larger than the returns they actually realize, but the predicted and realized return spreads between the lowest and highest predicted return portfolio is much greater for N-En. The per-month realized high-minus-low return generated by N-En for the testing sample of 2003 through 2020 is 2.04% with a Sharpe ratio of 1.28.23 Figure IA10.1 in the Internet Appendix shows that N-En generates sizable returns each year. Realized Sharpe ratios consistently exceed 0.5 per month and often exceed 1.5. Figure IA19.2 replicates this analysis for put and call options separately. We find that N-En manages to produce significant return spreads for each year in the sample and for both put and call options.24

Figure 5
Trading on machine learning predictions vs. return benchmark
The figure shows the year-by-year performance of the trading strategy following predictions by the linear (L-En) and nonlinear ensemble N-En, as well as a benchmark relying on IPCA by Kelly, Pruitt, and Su (2019). We show average realized monthly returns of decile-based long-short portfolios.

Option Return Benchmark. We have highlighted in Section 2 that there is little theoretical guidance as to what delta-hedged option returns should look like. As a simplistic benchmark, excess returns on delta-hedged options are zero in the world of Black and Scholes (1973). To provide a more realistic benchmark, we proceed in two ways. First, akin to the capital asset pricing model for risky securities, we propose a linear factor model for delta-hedged option returns with two factors. Following the decomposition in Internet Appendix IA2, which expresses expected delta-hedged option returns as a function of expected returns of the underlying and expected raw option returns, we include the excess return on the value-weighted equity market (CRSP) index (rM), as well as the excess return on an option market index, which we calculate as the dollar open interest-weighted delta-hedged return of all options available at time t (rO ). Since options are short-lived, we cannot reliably estimate betaexposures to the two risk factors on a per-option basis. Thus, in the spirit of Büchner and Kelly (2022), we make use of instrumented principal component analysis (IPCA), proposed by Kelly, Pruitt, and Su (2019), to instrument the time-variation in betas by observable option- and stock-level characteristics.25 Since our objective is to provide a simple and interpretable benchmark for expected returns in the options market, we use the stock and option market factors detailed above instead of estimating latent factors from cross-sectional option returns, that is, ft+1 = [rMt+1,rOt+1]. Then, we express next-period’s option returns as:

$$\tag{13}$$

After we estimate α,β and f on an expanding sample at the start of each January, we obtain expected option returns in an out-of-sample fashion analogous to the procedure used for N-En and L-En, by fixing the factor means observed until time t, denoted by f :

$$\tag{14}$$

With expected delta-hedged option returns at hand, we again invest in the options in the highest expected return decile and short options in the lowest expected return decile. The comparison of year-by-year realized monthly returns for L-En, N-En, and the aforementioned benchmark shown in Figure 5 produces a number of interesting results. First, realized returns of both machine learning ensembles, as well as the benchmark, are consistently positive in each year. Second, each method produces the largest return spreads in years of crises, namely 2008 and 2020. Third, the nonlinear ensemble outperforms both the competing linear ensemble, as well as the proposed factor-based benchmark by a large margin. N-En’s average monthly return amounts to 2.04%, which compares well to L-En’s 1.30% and the benchmark’s 1.20%. In fact, N-En manages to outperform L-En and the benchmark at the 1% significance level, whereas the average realized returns of L-En and the benchmark are statistically indistinguishable from one another.

The second benchmark we propose mimics an abstract quantitative investment strategy by performing univariate sorts on stock- and optionlevel characteristics. With this, we investigate which characteristics provide independent information about delta-hedged option returns (Green, Hand, and Zhang 2017).26 To do so, we sort options into decile portfolios by each of the 80 characteristics described in Internet Appendix IA5 as well as the 94 stock-level characteristics we obtain from Green, Hand, and Zhang (2017).27 We then form long-short portfolios as the difference between the highest and lowest decile, wherein option returns in each decile portfolio are weighted by the contract’s dollar open interest at trade initiation. We order the sorting in a way that assures that the average return over the testing sample between 2003 and 2020 of the long-short portfolios is positive.

The upper panel of Figure 6 shows a histogram of the average monthly returns that can be achieved using this approach. Three characteristic-sorted long-short portfolios manage to produce monthly returns north of, but close to, 1%. These include the option’s implied volatility, the maturity-specific atthe-money volatility, as well as pzeros, an illiquidity measure based on the proportion of zero return days of individual option buckets (Lesmond, Ogden, and Trzcinka 1999). The average return of the long-short portfolios amounts to 0.25% per month, with 26 portfolios averaging returns above 0.50%. In comparison, the red (gray) line denotes the average realized return of N-En (L-En) of 2.04% (1.30%), demonstrating a significant outperformance of both methods but especially of the nonlinear machine learning method when compared to the crude portfolio sorting approach. When comparing the realized returns of each univariate sort and the two ensembles, we find that L-En significantly (at the 5% level) dominates all but the sorts on the implied volatility, the at-the-money implied volatility, as well as pzeros, whereas N-En outperforms all of the univariate sorts. In the lower panel of Figure 6, we show the realized Sharpe ratios, confirming these findings. Both benchmark analyses highlight the efficacy of the proposed machine learning methods and stress the importance of allowing for nonlinearities and interaction effects when predicting option returns.

Figure 6
Average realized returns and Sharpe ratios of characteristic-sorted long-short portfolios
The figure shows a histogram of the average realized returns (upper panel) and Sharpe ratios (lower panel) of characteristic-sorted long-short portfolios, using both option- and stock-level characteristics. Each month, we sort options into decile portfolios based on each of the 80 characteristics described in Internet Appendix IA5 and the 94 stock-level characteristics obtained from Green, Hand, and Zhang (2017), and calculate long-short portfolio returns as the difference between the highest and lowest decile, wherein the options of each decile portfolio returns are weighted by each contract’s dollar open interest at trade initiation. We order the sorting so that the average long-short returns are positive over the testing sample, between 2003 and 2020.

Portfolio Decomposition. Table 4 provides summary statistics for the decile portfolios based on the nonlinear ensemble N-En. The high and low portfolios tend to include options from NU = 502 and NU = 637 underlyings, respectively. The portfolios use options from the least number of individual stocks, but still rely on average on options from about a third of the stocks included in the sample. While we find little change in the average moneyness of the included options, the low portfolio includes more short-term and fewer call options (% Calls = 0.44), while the high portfolio includes more long-term call options (% Calls = 0.78). The relative bid-ask spreads of the options are also highest for the extreme portfolios, while the outstanding open interest is lowest for options in these portfolios. The delta (of the unhedged option), theta, and vega are increasing in the expected option return, while gamma is roughly comparable across portfolios. Focusing on whether the predicted return direction is correct, s(r) = s(rˆ), we find the highest accuracy for Low with 69% of the return directions correctly predicted. The ratio first decreases and then starts to increase once more for portfolio 9 and the high portfolio. For the latter, it stands at 52%. Figures IA10.4 to IA10.6 in the Internet Appendix provide a visual representation of the portfolio decomposition, split by the put and call contracts included.

Table 4 N-En portfolio decomposition

	Lo	2	3	4	5	6	7	8	9	Hi
NU	637.04	801.99	829.53	813.09	783.93	760.82	742.42	708.65	642.98	502.24
m	1.04	1.01	1.00	1.00	1.01	1.01	1.01	1.02	1.03	1.05
ttm	125.12	154.03	173.00	182.08	187.37	189.01	187.04	183.64	182.52	199.59
>0	0.31	0.35	0.38	0.39	0.41	0.42	0.43	0.44	0.47	0.52
s(r)=s()	0.69	0.65	0.62	0.60	0.57	0.53	0.49	0.46	0.48	0.52
% Calls	0.44	0.40	0.42	0.42	0.45	0.49	0.55	0.62	0.69	0.78
Spread	0.20	0.16	0.14	0.12	0.11	0.11	0.11	0.12	0.12	0.14
OI	0.06	0.08	0.10	0.11	0.12	0.12	0.12	0.11	0.10	0.08
Delta	0.03	0.00	0.01	-0.00	0.00	0.03	0.08	0.15	0.21	0.29
Gamma	0.01	0.01	0.02	0.02	0.02	0.02	0.02	0.02	0.02	0.02
Vega	0.15	0.16	0.18	0.19	0.20	0.20	0.21	0.20	0.20	0.21
Theta	-0.32	-0.19	-0.15	-0.13	-0.12	-0.11	-0.11	-0.11	-0.12	-0.12

The table shows summary statistics for the 10 machine learning portfolios following the nonlinear ensemble N-En. All measures are averaged over time. NU denotes the number of individual stocks underlying the options in the portfolios, m the moneyness, and ttm the time-to-maturity. r >ˆ 0 denotes the share of positive returns in the portfolio and s(r)=s(rˆ) the share for which N-En correctly predicted the realized return’s sign. % Calls is the share of call options in the portfolio; Spread the average option bid-ask spread; OI the relative open interest; and delta, gamma, vega, and theta the respective option Greeks. Gamma is expressed for a 1% move in the underlying stock (gamma × S ) and vega and theta in terms of the underlying stock price ( x for x ∈ [vega,theta]).

Figure IA10.7 in the Internet Appendix shows the relative share of call options in the machine learning portfolios for each year in our testing sample. Over time, the call share of the high portfolio has decreased, especially after the financial crisis. For the low portfolio, it has stayed roughly constant over time. Another interesting aspect of the machine learning portfolios is that they allow one to analyze how far N-En exploits expected return differentials of options across or within underlyings. Figure IA10.8 shows the share of options of a given underlying that end up in the same portfolio. We find that this measure is highest for the high and low portfolios at slightly below 50%, suggesting that N-En exploits return differentials of options across underlyings. To further shed light on this, we consider alternations to the trading strategy. Enforcing that all options of an underlying are ranked in the same decile portfolio yields a monthly average excess return of 1.93% for N-En (Table IA10.2 in the Internet Appendix). Specifically, for each underlying, we compute the average expected return of all options in a given month and use this average to sort underlyings into 10 portfolios. Building decile portfolios on the underlying level and then averaging across underlyings yields a monthly excess return of 1.02% for N-En (Table IA10.3), suggesting that N-En primarily exploits return differences across underlyings as opposed to within the same underlying. However, as the unrestricted full sample return spread amounts to 2.04% per month, N-En tends to combine within and across underlying return predictability.28

Prediction Persistence and Turnover. How likely is it that securities selected in portfolio i at month t remain in the same portfolio at month t +1? While we cannot answer this question for individual option contracts, given that their moneyness and time-to-maturity change rapidly, we can provide indicative evidence for this on the options bucket level. We focus on how the portfolio mode for all stock-bucket combinations in our sample changes from one month to another.29 With this, we can understand more about the persistence of the machine learning predictions. Figure IA10.9 provides the results to this exercise: we first note that the diagonal is the lightest-shaded area, highlighting that moving from one to the same or a neighboring portfolio is the most likely transition. Second, the lightest areas overall are at the high and especially the low portfolio. The options of a given stock-bucket combination of the low and high portfolios exhibit the strongest persistence, where the low portfolio is again more persistent than the high portfolio.

Importance of Complexity. Table 3 shows that the difference in the highminus-low returns between L-En and N-En is highly significant at 0.74% per month. This spread is primarily driven by the low portfolios, with significantly lower realized returns for the options shorted by N-En. At the same time, the average prediction error for this portfolio is much smaller for N-En, whereas the errors are comparable for the high-decile portfolio. These findings suggest that nonlinearities seem to play a particularly important role in uncovering options with negative returns over the next month.

We investigate the importance of nonlinearities and complexity in general for the predictions of N-En per decile portfolio in Table 5. Complexity comprises three facets: the number of influential characteristics, the importance of nonlinearities, and the degree of interaction effects between characteristics. For the low portfolio, fewer characteristics are important on average. However, the predictions of expected option returns rely much more on nonlinearities for the low portfolio. We assess this by fitting a linear model to the functional form of the impact of each characteristic on expected returns and calculating the standard deviation of the residuals from the linear fit.

The importance of complexity for the N-En decile portfolios

	Nbr Imp.Features	Nonlinearities	Interactions (Std)	Interactions (IQR)
Lo	17.859	2.256	1.832	2.613
2	22.170	0.979	0.653	0.850
3	22.374	0.825	0.491	0.624
4	22.030	0.798	0.453	0.582
5	21.625	0.780	0.443	0.578
6	21.450	0.771	0.443	0.573
7	21.610	0.764	0.448	0.572
8	21.735	0.773	0.459	0.578
9	22.049	0.810	0.497	0.625
Hi	22.791	0.971	0.756	0.978

The table shows the importance of complexity for the predictions made by the nonlinear ensemble (N-En) method. The importance of complexity is measured for each decile portfolio. Complexity constitutes three aspects: the number of important features (Nbr Imp. Features), the degree of nonlinearities (Nonlinearities), and the degree of interaction effects between characteristics. The degree of nonlinearities is assessed by the standard deviation of residuals obtained from fitting a linear function to the functional form of each characteristic. The functional form is estimated via SHAP values. The degree of interactions is assessed by the local dispersion of SHAP values for each characteristic. Dispersion is either measured by the standard deviation (Interactions (Std)) or the interquartile range (Interactions (IQR)). Higher values denote a higher degree of nonlinearities and interaction effects, respectively.

Besides nonlinearities, Table 5 reveals that interaction effects play an important role in forming predictions for the low portfolio, represented by a higher degree of local dispersion. At the same time, the portfolio with the second-highest dependence on interaction effects is the high portfolio, highlighting the importance of including nonlinearities and interactions when predicting option returns of both legs of the spread portfolio.

Impact of Transaction Costs. So far, we have assumed that the investor can buy and sell each option at the mid point between bid and ask, that is with zero effective spreads. Prior research has shown that transaction costs in option markets can be large (Ofek, Richardson, and Whitelaw 2004). We now turn to the impact of trading at different transaction prices by changing the ratio of effective to quoted spreads. We measure transaction costs by realized effective spreads, which we vary between 25% and 100% of the quoted spread of the option contract or the underlying. Additionally, we consider the impact of having to post different margins for long and short option positions. Details on the calculation of margin requirements are provided in Internet Appendix IA6.1. When incorporating transaction costs into the sorting step, which places options into the respective decile portfolios, we adjust predicted returns by expected transaction costs before sorting into decile portfolios. In case an option does not mature at the end of the investment period, we assume that the observed spread at trade initiation also applies to unwinding the position after a month. For the expected transaction costs of the hedging portfolio, we assume that we pay on average three times the bid-ask spread of the underlying, quoted at trade initiation and multiplied by the absolute value of the option’s delta.30

Trading on machine learning predictions: transaction costs

Eff. Spread	L-En			N-En			N vs.L
	H-L	t	SR	H-L	t	SR
Option Costs
25%	0.591	(7.82)	0.514	1.271	(8.06)	0.801	***
50%	0.349	(4.35)	0.286	0.992	(5.47)	0.538	***
75%	0.183	(2.28)	0.148	0.806	(3.89)	0.408	***
100%	0.047	(0.56)	0.037	0.591	(2.79)	0.288	***
Option and Delta-Hedging Costs
25%	0.541	(7.73)	0.460	1.235	(8.27)	0.782	***
50%	0.303	(3.67)	0.233	0.916	(5.38)	0.503	***
75%	0.130	(1.48)	0.098	0.684	(3.93)	0.352	***
100%	0.009	(0.11)	0.007	0.500	(2.87)	0.245	***
Option and Delta-Hedging Costs with Long/Short Margin Requirements
25%	0.747	(5.81)	0.304	1.711	(6.22)	0.593	***
50%	0.312	(2.19)	0.121	1.218	(4.64)	0.400	***
75%	0.034	(0.22)	0.013	0.712	(2.75)	0.227	**
100%	-0.244	(-1.30)	-0.086	0.352	(1.29)	0.114	**

The table shows the returns to option portfolios sorted by the predictions made by the linear (L-En) and nonlinear ensemble (N-En) methods after accounting for transaction costs through effective spreads. Effective spreads are defined as a fraction of the quoted spread. We consider effective spreads for option prices (upper panel) and option prices and the underlyings’ prices (middle panel). Effective spreads for the underlyings’ prices account for delta-hedging costs. We additionally consider the impact of having to post different margins for long and short options positions (lower panel). Details on margin requirements are given in Internet Appendix IA6.1. Predictions made by L-En and N-En are adjusted by expected transaction costs before sorting in the high-minuslow portfolios. Each contract is weighted by its dollar open interest at the time of investment. H-L denotes the average monthly realized returns of the high-minus-low portfolio. t denotes the corresponding t-statistic and SR the resulting monthly Sharpe ratio. The last column (N vs. L) gives the significance of comparing the mean realized returns for N-En and L-En. ***, **, * correspond to N-En beating L-En significantly at the 1%, 5%, and 10% level, respectively.

Results are reported in Table 6. The upper panel shows results if we only consider transaction costs of the option position. In the middle panel, we consider both the transaction costs for trading the option and setting up and maintaining the hedge portfolio. Finally, in the lower panel, we additionally account for margin requirements of long and short hedged positions in the options market. Table 6 shows that predictions by N-En yield statistically significant positive monthly returns for all levels of transaction costs except for when we assume that the investor has to pay 100% of the quoted spreads on all positions and incorporate long and short margin requirements. In contrast, high-minus-low return spreads following L-En’s predictions turn insignificant for moderate to high levels of transaction costs. Trading on N-En’s predictions is always superior to L-En’s predictions gauging by the average monthly returns, the Sharpe ratio, and the statistical assessment of this outperformance. Finally, Table 6 shows that delta-hedging costs are generally of second order importance compared to transaction costs of trading the option, whereas margin requirements in combination with high levels of transaction costs are an important impediment to consider.

Muravyev and Pearson (2020) argue that effective spreads are much lower than the ones we measure using OptionMetrics’ end-of-day quoted spreads since investors have the ability to time trades over the entire trading day. Using high-frequency data on single-stock options, the authors find that effective transaction costs are about a quarter smaller than their conventional estimates. Moreover, considering the impact of execution timing, the reduction can be increased to more than 60%. We are therefore confident that the signals N-En generates are exploitable.

Feature group importance for nonlinear forecast ensemble
The figure shows the feature group importance for the 12 feature groups defined in Internet Appendix IA6 for the nonlinear (N-En) ensemble. We measure the importance using SHAP values. The group importance is the sum of the resulting SHAP values for all features included in a given group. The values are scaled such that they sum to one. The bars represent the mean feature group importance for the entire testing sample, the dots the dispersion of the group importance for the months in the testing sample. The abbreviations used: Acc=Accruals, Prof=Profitability, Q=Quality, Inv=Investment, Ill=Illiquidity, Info=Informed Trading, Val=Value, C=Contract, Past=Past Prices, Fric=Frictions, Ind=Industry.

5.3 Which characteristics matter?

We next analyze the importance of option characteristics and groups built thereof. Optimally, we would reestimate the model after excluding the characteristics in each group sequentially. This approach, however, is infeasible, given the large number of characteristics (a total of 273) in our estimation and, hence, the large computational burden required.31 Instead, we use SHAP values, which approximate the effect of this feature exclusion and are based on cooperative game theory.

Characteristics Groups. The relative feature group importance for N-En is provided in Figure 7, in which the groups are sorted by their total importance over the entire testing sample. Contract-based characteristics are the most important predictors of future option returns. Knowing where an option lies on the underlying’s implied volatility surface and knowing where that implied volatility surface lies relative to the market is essential when making option return forecasts. Measures of illiquidity and risk are the next-most important predictors. Of secondary importance, but nevertheless aiding in the prediction process for N-En, are characteristics in the groups’ Past Prices, which includes measures of stock and option momentum.32 The dots in the figure represent the group importance during each month in the testing sample. Occasionally, we find that illiquidity-based features are the most important. For most months, however, contract-based information exerts the highest influence on N-En’s predictions. Finally, N-En relies mostly on information drawn from options to make its predictions. The impact of most groups comprised solely of stock characteristics is very small in comparison (e.g., accruals or investment).

Single Characteristics. Figure 8 shows the 10 most important characteristics, along with the dispersion of their relative importance across the testing sample. The most influential characteristic by far is the implied volatility of the option (iv), followed by the bid-ask spread of the underlying stock (baspread). Industry momentum (indmom), the option’s delta (delta), the maturity-specific at-the-money implied volatility (atm_iv), and the variance risk premium (ivrv) are also highly important. The reliance also on stock-based characteristics corroborates the findings by Cao et al. (2019).33

Characteristic Sensitivity. Next, we seek to understand how the most important characteristics affect the direction of the predictions made by N-En. For this, Figure 9 shows a bee-swarm plot highlighting the change in the return prediction due to a given characteristic. For example, a higher implied volatility leads to a more negative prediction of delta-hedged returns. Negative relations are also present for the difference between implied volatility and realized volatility and the volatility of liquidity on the stock level (std_dolvol). In contrast, higher values of the underlying’s bid-ask spread or industry momentum positively affect return predictions. Figure IA11.4 in the Internet Appendix shows that the above relations are stable over time in their directional assessment. However, the magnitude with which changes in individual characteristics affect the return predictions varies substantially over time. Moreover, Figure IA11.5 provides further insights into the functional from of the impact of single features on L-En’s and N-En’s predicted deltahedged returns. The impact for N-En is highly nonlinear for each characteristic among the 10 most important. On the contrary and as expected, we observe a linear relation for L-En. Additionally, Figure IA11.5 reveals that there are instances in which L-En’s predictions appear as averages of N-En’s predictions (e.g., for delta), restricted by the imposed linear functional form. On the other hand, we observe instances with an entire level shift in the functional form (e.g., for rv and mid).

Most important characteristics for N-En
The figure shows the 10 most influential characteristics for the predictions of the nonlinear ensemble (N-En) following the importance using SHAP values. The values are scaled such that they sum to one across all 273 characteristics. The bars represent the mean feature importance for the entire testing sample, the dots the dispersion of the importance for the months in the testing sample. The feature group associated with the feature as defined in Internet Appendix IA6 is given in parentheses. The abbreviations used: C=Contract, Ill=Illiquidity, Past=Past Prices.

Sensitivity to Volatility and Jump Risk. A large part of the literature manifests that options incorporate volatility and jump risk premia. Cremers, Halling, and Weinbaum (2015) and Dew-Becker, Giglio, and Kelly (2021) show that some options are driven by volatility risk, while others are driven by jump risks, conditional on their time-to-maturity and moneyness. Following the previous literature, we identify four (five) proxies for volatility (jump) risks in our set of characteristics.34 We compute the importance of the nine characteristics in terms of their ranking using absolute SHAP values for the entire sample, as well as the ranking for individual option buckets, relative to the rank of the full sample. Table IA11.1 in the Internet Appendix reports these results. In general, we find that jump risk proxies impact short-term and especially out-of-the-money options the most. In contrast, the impact of many volatility risk proxies is comparable across buckets, with an increased impact of vega for short-term options. Knowing about a stock’s variance risk premium (ivrv) is vital for predicting all options. Note that this proxy is also part of the 10 most important features that N-En identifies.

Impact of the most important characteristics on predicted delta-hedged returns for N-En The figure shows the impact of the 10 most influential characteristics on the predictions of the nonlinear ensemble (N-En). Feature impact is measured by SHAP values. The feature group associated with the feature as defined in Internet Appendix IA6 is given in parentheses. The abbreviations used: C=Contract, Ill=Illiquidity, Past=Past Prices.

To provide an intuition for how options with a different moneyness-maturity combination depend on volatility and jump risks, Figure 10 visualizes the functional form of how sensitive N-En’s predictions are to changes in the most important volatility (ivrv) and jump risk proxies (gamma). The predicted deltahedged returns are uniformly decreasing for higher values of ivrv. Interestingly, we find a strong tail dependence in this pattern: For the largest values of ivrv, we find an amplified impact on the return prediction, depressing predicted returns by more than −4% for the stocks within the five highest ivrv percentiles. We find little variation in this functional form across option buckets.

In contrast, the relation between predicted delta-hedged returns and tail risk shows more variation across different buckets. We generally find that low gamma values lead to predictions of lower returns, but a positive impact on the predictions for large gamma values. We can furthermore show that shortterm at-the-money options exhibit the largest and most diverse sensitivity to changes in gamma. The functional form for long-term options differs little for different moneyness regions. Figure IA11.6 in the Internet Appendix replicates these graphs for the second-most important volatility (vega) and jump risk proxy (tlm30). We find a highly nonlinear dependence of N-En’s predictions on both proxies, with striking differences across option buckets.

Figure 10
Relative effect of volatility and jump risk premia per bucket
The figure shows the effect of features proxying for volatility and jump risk, respectively, on the predictions of the nonlinear ensemble (N-En) per bucket. The impact of the feature is measured using SHAP values. ivrv is the volatility risk premium, following Bali and Hovakimian (2009), and gamma is the option’s gamma.

5.4 Impact of the information set

Instead of analyzing the importance of single characteristics, we investigate how delta-hedged option return predictability changes when using only a subset of characteristics. We reestimate each model using only stockbased characteristics (S), option-based characteristics (O), or only those characteristics operating on the bucket or individual contract level (B+I) and contrast the resulting predictability with that of the full information set.35

Figure 11 provides the resulting full-sample R2OS values. Using all available information produces the highest R2OS, consistent with the idea that more information leads to better forecasts if the models used are sufficiently able to capture this information. Less weight is put on uninformative characteristics and important nonlinear interactions between them are used advantageously. Restricting to only option-based information (O) is our second choice. The benefit of adding stock-based to the 80 option-based characteristics is substantial, given that the R2 drops from 2.45% to 1.99% (both significant at the 1% level) for the whole sample if we exclude the whole stock-based characteristics block. Only considering the subset stock-based characteristics (S), however, is detrimental to uncovering option return predictability. The outof-sample R2 drops to 0.11%. The benefit of option-derived characteristics is huge when making informed forecasts of future delta-hedged option returns. As an additional check, we also consider whether option-contract and optionbucket information is sufficiently informative, which would render the addition of option-based characteristics for the underlying pointless. We strongly reject this idea, given that the inclusion of option-based characteristics for the underlying boosts out-of-sample predictability R2OS from 1.69% (B+I) to 1.99% (O). Consistent with the feature group importance shown in Figure 7, contract-based characteristics are highly important, but alone do not suffice to forecast future single-equity option returns.36

Figure 11
Restricting the information set for N-En
The figure shows the out-of-sample R2 defined in Equation 4 for N-En with restricted access to the full set of characteristics. The full model is shown in the left bar for reference, and is compared with models using all option-based information (O), models using only bucket- and individual contract-based information (B+I), and models using only stock-based information (S). The distinction of the information source is provided in Internet Appendix IA6. ***, **, * below the bars denotes statistical significance at the 0.1%, 1%, and 5% level as defined in Equation (7) for the sample of “all” options. The testing sample spans the years 2003 through 2020.

We use Diebold and Mariano (1995) forecast comparisons to assess whether the forecasts of N-En using more information are statistically more informative. For completeness, we also consider the linear ensemble (L-En) once more to understand to what degree allowing for nonlinearities helps when using only a restricted set of characteristics.

Table 7 Forecast Comparison for Characteristic Sets

	L-En			N-En
	0	S	full	B+I	0	S	full
A:Diebold and Mariano (l995) forecast comparison
L-En: B+I	3.07	-2.87	4.39	2.26	2.80	0.05	3.48
L-En:O		-4.82	3.59	1.84	2.47	-0.63	3.23
L-En: S			7.04	3.42	4.08	1.24	4.78
L-En: full				1.17	1.88	-1.74	2.79
N-En: B+I					2.38	-3.97	3.26
N-En: O						-5.13	2.13
N-En: S							7.42
B:Forecast correlation
L-En: B+I	0.90	0.50	0.80	0.62	0.57	0.33	0.55
L-En:O		0.47	0.86	0.60	0.65	0.33	0.62
L-En: S			0.58	0.32	0.31	0.61	0.35
L-En: full				0.55	0.61	0.41	0.68
N-En: B+I					0.87	0.38	0.79
N-En: O						0.38	0.85
N-En: S							0.53

Panel A of the table shows Diebold and Mariano (1995) test statistics defined in Equation 8 to compare the forecasts for models with restricted access to the full set of characteristics. The full model is compared with models using all option-based information (O), models using only bucket- and individual contract-based information (B+I), and models using only stock-based information (S). The distinction of the information source is provided in Internet Appendix IA6. Significance at the 1% (5%) level is highlighted in light blue (blue). Panel B shows forecast correlations defined in Equation 10. Here, highlighting in light blue (blue) denotes large values with a cutoff at 90% (70%).

Panel A of Table 7 reports the results. For the linear and nonlinear ensemble, we find that adding more information is always worthwhile. At the same time, we find a clear hierarchy, which puts the informational content of option-based characteristics above that of stock-based characteristics. Models using both bucket and contract information (B+I) in their respective ensemble class beat those relying solely on stock-based information (S), but are outperformed by models also leveraging the information inherent in option-based information about the underlying (O). The full model for L-En and N-En performs significantly better than all three alternative model specifications. Comparing the forecasts of the linear and nonlinear ensemble, we find that N-En restricted to option-based information (O) manages to surpass all L-En models at the 10% significance level or better. In contrast, the full L-En model only manages to provide marginally more accurate forecasts than N-En restricted to stock-based characteristics (S). In line with our intuition that (i) more information is always better and (ii) nonlinear interactions are most valuable when making informed investment decisions in the options space, we find that the full nonlinear ensemble manages to outperform all four L-En specifications.

The findings from the Diebold and Mariano (1995) comparison carry over to evidence from forecast correlations in panel B of Table 7. We find the largest similarity in the resulting predictions for linear or nonlinear ensembles estimated on option-based information (both B+I and O) and the full information set (ρt+1 > 0.7). The forecast correlation of models using only option- and only stock-based characteristics is particularly low.37

5.5 Robustness checks

5.5.1 Trading on machine learning portfolios: Earnings announcements. Engelberg, McLean, and Pontiff (2018) document that stock market anomalies are 50% higher on corporate news days and six times higher on earnings announcement days, indicating that stock return predictability is more consistent with a mispricing explanation or when driven by biased expectations that are partly corrected upon news arrival.

To gauge the effect of earnings announcements and news on return spreads in our setting, we use a subsample analysis similar to Zhan et al. (2021), but on the set of weekly short-term at-the-money options (Section 5.5.4). Precisely, in each month, one subsample contains short-term at-the-money options whose underlying stocks experience an earnings (news) announcement during that week, while the other subsample contains short-term at-the-money options on stocks without an earnings (news) announcement during that week. News days are identified using the Dow Jones version of Ravenpack News Analytics. News is only recorded if the relevance score is 100 and if it is highly positive (sentiment score above 0.75) or highly negative (sentiment score below 0.25).

Figure IA10.3 in the Internet Appendix shows the realized return spreads of the high-minus-low portfolio for the different subsamples and produces a number of interesting results: First, the realized return spread is nearly three times larger for earnings announcement weeks for predictions made by both L-En and N-En. Second, the returns of the high-minus-low portfolio are still significantly different from zero for non-earnings-announcement weeks. Third, N-En yields higher high-minus-low portfolio returns regardless of whether earnings are announced or not. Fourth, we find a similar, but less pronounced, effect for news announcements as the realized returns of high-minus-low portfolios are approximately 50% higher for news weeks. These findings suggest that our uncovered option return predictability is at least partially driven by option mispricing.

Subsamples. We investigate to what degree the profitability of our machine learning portfolios changes with the state of the economy. The results in Table IA10.5 show that realized returns are generally amplified during bad states of the economy and that N-En manages to outperform L-En across different market phases. This holds for a wide range of measures approximating the state of the economy, market volatility, uncertainty, and investor sentiment.

Risk Attribution. A possible explanation for our results is that the machine learning models are best at predicting the most risky option positions, which should translate into higher future realized returns. To understand whether this is the case, we compute risk-adjusted returns for the Hi-Lo portfolios for N-En, either for the pooled sample of all options or split by option buckets. We consider a wide range of candidate models, which have been proposed by earlier studies to explain the returns of a variety of financial instruments, including the CAPM; the Fama and French (2015) five-factor model enhanced by the momentum factor of Carhart (1997) (Fama and French 2018) and additionally by the liquidity factor of Pástor and Stambaugh (2003); a model using the Agarwal and Naik (2004) option-market factors; a model with the intermediary leverage bearing capacity of Grünthaler, Lorenz, and Meyerhof (Forthcoming); and, finally, the factor model of Bali, Chabi-Yo, and Murray (2022) proposed for stock returns based on option prices.

The results are provided in Table IA10.6. Risk-adjusted returns for all option types are virtually unchanged compared to average raw returns. The risk exposure picked up by the candidate models does not suffice to explain the return spreads generated by the nonlinear machine learning methods.

5.5.2 Variations in the training window: Fixed-length training window. As in Gu, Kelly, and Xiu (2020), we train the models using an expanding training sample. We refit the models each year and correspondingly increase the size of the training window by 1 year after each iteration. As a robustness check, we consider a rolling training sample with a fixed length of 10 years. Figure IA13.1 depicts the statistical performance for N-En. The R2 declines from 2.5% to 1.75% for the testing sample between 2008 and 2020, when compared to using an expanding training sample.38 The reduction is also prevalent for calls and puts, as well as for the cross-sectional predictability, for the expanding training sample, the realized Sharpe ratio is in fact higher using a rolling training scheme, driven by a lower fluctuation of realized returns (see Table IA13.1). However, neither model statistically dominates the other, neither for the full sample nor individually for put or call options.

Excluding vs. including the Great Financial Crisis. Next, we reestimate the models excluding information about option returns during the financial crisis in 2008 and 2009. Subsequently, we compare the resulting performance with our baseline model specification for the testing years of 2010 through 2020. We observe a slightly higher statistical performance (Figure IA13.2) and return spread of the Hi-Lo portfolio when including 2008 and 2009 in the training sample (Table IA13.2). However, the resulting return spreads of the high-minus-low portfolios are statistically indistinguishable.39

Summing up, these results suggest that the model’s ability to estimate a meaningful connection between option characteristics and future option returns does not materially hinge on the way we set up the training and validation samples and instead is robust to alterations of it.

5.5.3 Restricting to the 500 largest CRSP stocks. Next, we focus only on the 500 largest stocks of the entire CRSP universe determined by market capitalization of individual stocks in each month. While the overall level of predictability R2OS is lower, the return predictions made by the nonlinear ensemble remain useful. N-En yields a positive R2OS and a significantly positive R2OS;XS (Figure IA14.1). This compares well to L-En, which fails to yield significant predictability. The average realized excess returns of N-En’s predictions are 1.73% per month with a monthly Sharpe ratio of 0.76, significantly outperforming the high-minus-low portfolio returns of the L-En (Table IA14.1). These conclusions are robust to considering call and put options separately.

5.5.4 Weekly investment period. In this section, we consider a shorter investment period of 1 week instead of 1 month. We focus the analysis on short-term at-the-money options, which are most actively traded and have low transaction costs (Garleanu, Pedersen, and Poteshman 2009; Zhan et al. 2021).

Figure IA15.1 reports the resulting predictability of the linear and nonlinear ensemble when predicting weekly option returns. We find higher predictability compared to the sample of monthly option returns. While L-En consistently yields out-of-sample R2 at or above 3%, predictability of N-En is almost twice that high. These findings are robust to the cross-sectional R2OS;XS and hold regardless of whether we consider puts, calls, or both. In line with higher predictability, we also document an improvement in economic performance (Table IA15.1). Both L-En and N-En generate weekly Sharpe ratios above 1.3, doubling the monthly Sharpe ratios obtained in the baseline results. Furthermore, both N-En’s short and long portfolios yield significantly higher returns than predictions based on L-En, stressing the importance of modeling nonlinear and interaction effects for option returns also at higher frequencies.

5.5.5 Different return definitions: Margin-adjusted returns. The main analysis throughout the paper uses delta-hedged gains scaled by the cash requirement of opening the delta-hedged option position. Garleanu and Pedersen (2011) and Hitzemann et al. (2021) show that margin requirements for option positions are large and suggest evaluating returns using margin requirements as the denominator in the return definition of Equation 12. We consequently adopt the CBOE minimum margin for customer accounts. Details on the calculation of margin requirements are given in Internet Appendix IA16.1.

Refitting the models on returns scaled by margin requirements of long options positions leaves our baseline results mostly unchanged. Statistical performance is shown in Figure IA16.1. Table IA16.1 reports the economic performance with high-minus-low portfolios before transaction costs, whereas Table IA16.2 includes transaction costs and accounts for margin requirements of short option positions.

Delevered Returns. Besides margin-adjusted returns, we assess how the trading strategy profits change if we account for time-variation in the leverage of the traded options, following Frazzini and Pedersen (2022). Table IA16.3 shows average realized excess returns and Sharpe ratios of the decile portfolios sorted by N-En’s predictions after we account for the embedded leverage. We account for leverage in three ways: First, we scale the realized profits of each portfolio by the average leverage of the options contained in portfolio p. We find that realized returns when expressed per unit of leverage decrease to 0.55% per month. However, the Sharpe ratio is virtually unchanged at 1.25 (1.28 with leverage), suggesting that risk-adjusted profits are not driven by differences in the average leverage of the high or low portfolio. Separate results for calls and puts confirm this finding. Second, we account for time-variation of the portfolio leverage by scaling the realized returns by the average leverage of portfolio p measured in the month of trade initiation (t ). We find that monthspecific differences in the portfolio leverage result in slightly lower Sharpe ratios for the high-minus-low portfolio (0.95). However, the Sharpe ratio and average realized returns are still highly significant. Lastly, we also account for the time-varying leverage of each individual option contract o. The resulting high-minus-low returns per unit of leverage amount to 1.21% per month with a Sharpe ratio of 0.65. The realized monthly Sharpe ratio decreases by about half if we account for time-variation in the contract-specific leverage, but is still large for unit leverage of both the short and long leg.

5.5.6 Option buckets. A possible objection to our result that nonlinear models outperform linear models is our intention of predicting all options across the moneyness-maturity spectrum. This potentially introduces nonlinear interactions mechanically that would be irrelevant when predicting returns of only at-the-money and short-term contracts.

We address this potential criticism in three ways. First, Figure IA17.1 shows the R2OS comparison between the linear and nonlinear ensemble for each option bucket considered. Table IA17.1 provides the trading strategy results for different option buckets. Predictability is concentrated in short-term options, for which we also find the highest payoffs when following the investments proposed by our machine learning models. Throughout all option buckets but short-term otm calls, we find significantly higher raw and risk-adjusted returns for N-En than for L-En.

Second, we investigate the power of N-En for predicting the returns of option portfolios and compare it with predicting returns for the individual contracts within that portfolio (Figure IA17.2). The construction of portfolios follows the bucket definition in Section 4. It yields one option portfolio per bucket and underlying, weighting each contract by its dollar open interest. The literature commonly uses portfolios or a single most-liquid (call) contract to assess the predictability of option returns. Reasons are limiting the influence of noise and drastically shrinking the size of the estimation problem at the loss of generalizability (see Cao and Han 2013; Zhan et al. 2021; Goyenko and Zhang 2021). We find that we can predict a higher share of portfolio returns within the nonlinear ensemble (which is fitted on individual contracts), especially for short-term options. Overall, however, predictability, measured both by R2OS and

Third, we run a full-on comparison between N-En estimated using all options in our sample and a nested model specializing in short-term atthe-money options. Internet Appendix IA18 shows comparable levels of overall predictability and few drawbacks to predicting returns of all options simultaneously.

6. Sources of Option Return Predictability

Hong and Stein (1999) propose a theoretical model in which gradual diffusion of information among investors explains the observed predictability of asset returns. In their model, at least some investors can process only a subset of publicly available information because either they have limited informationprocessing capabilities or searching over all possible forecasting models using publicly available information itself is costly (Hirshleifer and Teoh 2003), and there are limits to arbitrage (Shleifer and Vishny 1997; Pontiff 2006). Due to investors’ limited attention and informational frictions, new informative signals are incorporated into asset prices partially because at least some investors do not adjust their demand by recovering informative signals from observed prices. As a result of this failure on the part of some investors, asset returns exhibit predictability. In this section, we investigate potential economic mechanisms underlying the sources of option return predictability. In particular, we test whether informational frictions and option mispricing provide an explanation for the observed return predictability in the options market.

As shown in Internet Appendix IA2, the expected return for selling a deltaneutral call (put) is the weighted average of the expected return on underlying stock and the expected return on call (put) option. Hence, we argue that both stock and option characteristics can be viewed as potential determinants of the cross-sectional differences in delta-hedged option returns. Since the literature on option pricing does not provide clear theoretical guidance as to what delta-hedged expected returns should look like, and combined with the considerations put forth in Section 2, we conjecture that the expected return on delta-hedged option positions can be a highly nonlinear function of stock and option characteristics. Thus, it is possible that delta-hedged option return predictability can be driven by information frictions, limits-to-arbitrage, and mispricing in both the underlying equity and option markets.

6.1 Informational frictions

We analyze whether the return predictability is concentrated in options with higher levels of informational frictions. We hypothesize that option return predictability originates partly as a result of informational frictions, such that the information implied from stock- and option-based characteristics is not directly incorporated into option prices.

Rather than relying on a single proxy for information frictions, we follow Atilgan et al. (2020) and construct an arbitrage cost index using a number of indicators known to capture several dimensions of limits-to-arbitrage. First, we build an index on the stock level, for which we include firm size, firm age, idiosyncratic volatility of the underlying, institutional ownership (Ofek, Richardson, and Whitelaw 2004; Nagel 2005; Eisdorfer, Goyal, and Zhdanov 2022), and analyst coverage (Zhang 2006). To construct the arbitrage cost index, we sort stocks in increasing order based on their idiosyncratic volatility. Similarly, we sort stocks in decreasing order based on their level of institutional ownership, analyst coverage, size, and age since lower values of these variables indicate higher arbitrage costs. Each stock is given the corresponding score of its decile rank for each variable. Finally, the arbitrage cost index on the stock level is the sum of the five scores, ranging from 5 to 50. A higher value implies tighter limits-to-arbitrage.

Similarly, we build an alternative arbitrage cost index on the option level. We use option illiquidity, option bucket trading volume, dollar open interest, and margin requirements. Moreover, Tian and Wu (Forthcoming) argue that delta-hedging an option position exposes the option investor to three primary risks: the delta-hedging costs, stochastic volatility risk, and random jump risk. Consequently, we add the bid-ask spread of the underlying stock, volatility of implied volatility, and excess kurtosis of the underlying stock as variables for the index. While constructing the index on the option level, we adjust the sorting mechanism to ensure that higher values indicate stricter limits to arbitrage.

In order to investigate the resulting differences in return predictability, we form quintile splits at time t of the stocks in our sample, either by the arbitrage index on the stock level, or on the option level (Q1–Q5). Subsequently, we contrast the predictability of options written on stocks with different levels of the arbitrage indices. Figure 12 depicts the R2OS values for the subsamples. Higher informational frictions directly translate to higher predictability of option contracts by N-En, confirming our hypothesis. Options on underlyings with the highest levels of information frictions show the highest predictability = 5.32%. Instead, for options written on stocks with the lowest level of information frictions, R2OS is indistinguishable from zero. The predictability uncovered is statistically significant only for the three highest information friction quintiles. Turning to the arbitrage index on the option level, option return predictability is monotonically increasing in the contract-specific limits to arbitrage. In the highest quintile, we find R2OS = 4.00%, which drops to 1.36% in Q1. Figure IA19.1 in the Internet Appendix shows similar findings in the case of cross-sectional out-of-sample predictability.

Figure 12
Predictability conditional on information frictions
an index of informational frictions on the underlying level and an index of informational frictions on the option level. Index construction follows Atilgan et al. (2020). Firm size, firm age, idiosyncratic volatility, institutional ownership, and analyst coverage are used to construct the index on the stock level. The option contract’s bidask spread, margin requirement, dollar open interest, bucket volume, the volatility of the implied volatility, the historical excess-kurtosis of the underlying, and the underlying’s bid-ask spread are taken for the index construction on the option level. Since the level of institutional ownership, analyst coverage, firm age, firm size, dollar open interest, and bucket volume are inversely related to informational frictions, these characteristics are inversely sorted in the indices’ construction. ***, **, * above the bars denotes statistical significance at the 0.1%, 1%, and 5% level, as defined in Equation (7).

Additionally, we perform bivariate portfolio sorts to study how realized option returns depend on informational frictions. We sort options into quintiles based on the friction index on the stock or option level at the end of month t. Subsequently, within each quintile, options are further sorted into quintiles by the 1-month-ahead expected return forecast of N-En.

Panel A of Table 8 presents the results for the 25 (5x5) portfolios sorted by expected returns conditional on information frictions on the stock level. High-minus-low return spreads are significantly positive for each of the stock arbitrage quintiles. However, realized excess returns are an increasing function of informational frictions in the underlying. For the lowest friction levels, average returns amount to 0.75% per month, for the highest level of frictions to 3.34%, which leads to an economically and statistically significant diff-in-diff return spread of 2.59% per month.

Panel B of Table 8 replicates this analysis for the option-level friction score. The high-minus-low return spreads are monotonically increasing in the level of the friction index, confirming our hypothesis that option return predictability is directly related to information frictions. Furthermore, this leads to a significant difference between the high-minus-low spreads of the highest and lowest limits-to-arbitrage quintile of 0.87% per month.

Bivariate portfolios of information frictions and expected returns

	Low Pred.	2	3	4	High Pred.	H-L
Stock Arbitrage Score
Low	-0.650***	-0.273**	-0.168	-0.092	0.103	0.754***
2	-0.891***	-0.335**	-0.199	-0.082	0.221	1.112***
3	-1.125***	-0.402***	-0.190	-0.080	0.349	1.474***
4	-1.371***	-0.436***	-0.191	0.043	0.592**	1.963***
High	-2.529***	-0.857***	-0.313*	0.062	0.813***	3.342***
H-L	-1.878***	-0.584***	-0.145**	0.154**	0.710***	2.588***
Option Arbitrage Score
Low	-1.105***	-0.316**	-0.142	-0.023	0.275	1.380***
2	-1.214***	-0.266*	-0.076	0.038	0.399**	1.613***
3	-1.249***	-0.276**	-0.049	0.089	0.483**	1.732***
4	-1.388***	-0.293**	-0.037	0.122	0.615***	2.003***
High	-1.487***	-0.362***	-0.073	0.188	0.767***	2.255***
H-L	-0.382***	-0.046	0.069	0.211***	0.492***	0.874***

The table shows the returns to option portfolios first sorted by an index of informational frictions, either on the stock level (upper panel) or on the option level (lower panel) and then by the return predictions made by the nonlinear ensemble method. Index construction follows Atilgan et al. (2020). Firm size, firm age, idiosyncratic volatility, institutional ownership, and analyst coverage are used to construct the index on the stock level. The option contract’s bid-ask spread, dollar open interest, bucket volume, the volatility of the implied volatility, the historical excess-kurtosis of the underlying, and the underlying’s bid-ask spread are taken for the index construction on the option level. Since the level of institutional ownership, analyst coverage, firm age, firm size, dollar open interest, and bucket volume are inversely related to informational frictions, these characteristics are inversely sorted in the indices’ construction. ***, **, * denotes statistical significance at the 1%, 5%, and 10% level.

6.2 Option mispricing

We expect to find higher levels of predictability for options that are priced incorrectly as the nonlinear ensemble manages to identify these opportunities and correctly proposes shorting over- and purchasing undervalued contracts. In the spirit of the previous section on limits-to-arbitrage, we again refrain from taking a stand on which variable constitutes option-level mispricing, but instead use a composite score. Note that we are investigating absolute levels of mispricing.

The first input to the mispricing index is iv −rv, studied by Goyal and Saretto (2009) and Carr and Wu (2009). Next, we follow Eisdorfer, Goyal, and Zhdanov (2022) to quantify option mispricing as the ratio between theoretical and observed option prices. The theoretical option price is given by the Black and Scholes (1973) pricing model, where we use each underlying stock’s realized volatility over the past quarter, estimated using high-frequency price data from the NYSE TAQ database, as our estimate for the expected volatility. For all short-term at-the-money options, we compare the log of the theoretical price with the log of the price observed in the market, that is, |Mispricing| = log(O/O) , where O denotes the theoretical and O the observed mid price. Averaging over all short-term at-the-money options, we obtain one level of mispricing per underlying stock at each point in time. Our last mispricing measure is the absolute value of the return forecast of the nonlinear ensemble made in month t.

Predictability and profitability conditional on option mispricing: R2OS
The figure shows out-of-sample R2OS as defined in Equation 4 using the nonlinear ensemble N-En for different quintiles of option mispricing. We calculate absolute option mispricing using a composite mispricing score. As inputs, we use iv −rv (Goyal and Saretto 2009; Carr and Wu 2009), the mispricing measure by Eisdorfer, Goyal, and Zhdanov (2022), as well as the absolute return prediction of the nonlinear ensemble. ***, **, * above the bars denotes statistical significance at the 0.1%, 1%, and 5% level, as defined in Equation 7.

Bivariate portfolios of option mispricing and expected returns

	Low Pred.	2	3	4	High Pred.	H-L
Option Mispricing Score
Low	-0.424***	-0.095	0.003	0.069	0.335*	0.759***
2	-0.721***	-0.141	0.003	0.141	0.509**	1.230***
3	-0.935***	-0.241*	-0.028	0.171	0.580**	1.515***
4	-1.506***	-0.500***	-0.214	0.064	0.650***	2.156***
High	-2.348***	-0.952***	-0.553***	-0.238	0.392*	2.740***
H-L	-1.924***	-0.857***	-0.555***	-0.307***	0.056	1.981***

The table shows realized returns for quintiles portfolios following the predictions by the nonlinear ensemble N-En within quintiles sorted by option mispricing. We calculate absolute option mispricing using a composite mispricing score. As inputs, we use iv −rv (Goyal and Saretto 2009; Carr and Wu 2009), the mispricing measure by Eisdorfer, Goyal, and Zhdanov (2022), as well as the absolute return prediction of the nonlinear ensemble. ***, **, * denotes statistical significance at the 1%, 5%, and 10% level.

We once again sort options into quintiles, this time by the composite mispricing score, and show the resulting predictability patterns for future returns over the next month in Figure 13.40 For low levels of mispricing, R2OS is close to zero. In contrast, for the options with the highest mispricing score, R2OS is 4.09% per month.

In Table 9, we look at the resulting realized returns of portfolios first sorted by the option mispricing score and then by N-En’s expected return forecast. While the high-minus-low return spread is significant for all levels of option mispricing, it monotonically increases with the degree of mispricing. In fact, the monthly return difference between the H-L spreads using options with the highest versus the lowest mispricing scores amounts to 1.98% per month.

Overall, these results suggest that option return predictability is largely driven by informational frictions, limits-to-arbitrage, and mispricing in the options market. However, given the connection between risk, arbitrage cost, and proxies for information frictions, we cannot rule out potential riskbased explanations. Moreover, as discussed in Section 6.3, we find that nonlinear option risk measures such as jump and volatility risk are important determinants of option returns. It is also well known that illiquid stocks tend to have high market beta and high firm-specific volatility. Moreover, illiquid stocks and options are known to exhibit skewed fat-tailed return distributions with significant volatility and jump risk premia.41 Thus, higher values of the information frictions index proposed in the paper indicate stricter limits-toarbitrage and a higher level of riskiness, so that the significantly large abnormal returns on portfolios of equity options can partly be compensation for volatility, jump, and liquidity risks in the options market.

7. Conclusion

An extensive literature examines cross-sectional determinants of stocks, bonds, currencies, mutual funds, and hedge funds. However, research on crosssectional predictors of option returns is relatively scarce and not very well understood. In this paper, we close this gap in the literature and identify variables that predict the cross-sectional differences in delta-hedged option returns. Predicting option returns is of foremost relevance for retail and institutional investors, as the importance of option markets for hedging and speculation purposes has strongly increased in recent years.

In this paper, we apply machine learning techniques to predict individual U.S. equity option returns using a set of 80 option-based and 193 stock-based characteristics during the period from 1996 to 2020. Empirically, we derive several results that enhance our knowledge of the cross-sectional pricing of equity options. First, we show that the complexity of the machine learning models matters for prediction and observe that nonlinear models outperform linear models in terms of out-of-sample R-squared. Second, our results reveal that a trading strategy based on nonlinear machine learning forecasts is highly profitable and remains statistically and economically significant even after accounting for high levels of transaction costs. Third, we find that characteristics describing the option’s location on the underlying’s implied volatility surface and nonlinear option risk measures such as jump and volatility risk are important determinants of option returns. Finally, we document that option return predictability is largely driven by informational frictions and mispricing in the options market. In line with this notion, we find that option return predictability is higher for options and underlyings with higher limitsto-arbitrage and a higher degree of mispricing.

References

Agarwal, V., and N. Y. Naik. 2004. Risks and portfolio decisions involving hedge funds. Review of Financial Studies 17:63–98.

Amihud, Y. 2002. Illiquidity and stock returns: Cross-section and time-series effects. Journal of Financial Markets 5:31–56.

An, B.-J., A. Ang, T. G. Bali, and N. Cakici. 2014. The joint cross section of stocks and options. Journal of Finance 69:2279–337.

Arik, S. Ö., and T. Pfister. 2021. Tabnet: Attentive interpretable tabular learning. Proceedings of the AAAI Conference on Artificial Intelligence 35:6679–87.

Atilgan, Y., T. G. Bali, K. O. Demirtas, and A. D. Gunaydin. 2020. Left-tail momentum: Underreaction to bad news, costly arbitrage and equity returns. Journal of Financial Economics 135:725–53.

Bakshi, G., C. Cao, and Z. Chen. 1997. Empirical performance of alternative option pricing models. Journal of Finance 52:2003–49.

Bakshi, G., and N. Kapadia. 2003. Delta-hedged gains and the negative market volatility risk premium. Review of Financial Studies 16:527–66.

Bali, T. G., F. Chabi-Yo, and S. Murray. 2022. A factor model for stock returns based on option prices. Working Paper, Georgetown University.

Bali, T. G., A. Goyal, D. Huang, F. Jiang, and Q. Wen. 2021. Predicting corporate bond returns: Merton meets machine learning. Working Paper, Georgetown University.

Bali, T. G., and A. Hovakimian. 2009. Volatility spreads and expected stock returns. Management Science 55:1797–812.

Bali, T. G., and S. Murray. 2013. Does risk-neutral skewness predict the cross-section of equity option portfolio returns? Journal of Financial and Quantitative Analysis 48:1145–71.

Baltussen, G., S. van Bekkum, and B. van der Grient. 2018. Unknown unknowns: Uncertainty about risk and stock returns. Journal of Financial and Quantitative Analysis 53:1615–51.

Bates, J. M., and C. W. Granger. 1969. The combination of forecasts. Journal of the Operational Research Society 20:451–68.

Bekaert, G., E. C. Engstrom, and N. R. Xu. 2021. The time variation in risk appetite and uncertainty. Management Science 68:3975–4004.

Bianchi, D., M. Büchner, and A. Tamoni. 2021. Bond risk premiums with machine learning. Review of Financial Studies 34:1046–89.

Black, F., and M. Scholes. 1973. The pricing of options and corporate liabilities. Journal of Political Economy 81:637–54.

Bollen, N. P., and R. E. Whaley. 2004. Does net buying pressure affect the shape of implied volatility functions? Journal of Finance 59:711–53.

Bollerslev, T., V. Todorov, and L. Xu. 2015. Tail risk premia and return predictability. Journal of Financial Economics 118:113–34.

Breiman, L. 2001. Random forests. Machine Learning 45:5–32.

Büchner, M., and B. Kelly. 2022. A factor model for option returns. Journal of Financial Economics 143:1140–61.

Buraschi, A., and J. Jackwerth. 2001. The price of a smile: Hedging and spanning in option markets. Review of Financial Studies 14:495–527.

Byun, S.-J., and D.-H. Kim. 2016. Gambling preference and individual equity option returns. Journal of Financial Economics 122:155–74.

Cao, J., and B. Han. 2013. Cross section of option returns and idiosyncratic stock volatility. Journal of Financial Economics 108:231–49.

Cao, J., A. Vasquez, X. Xiao, and X. Zhan. 2019. Volatility uncertainty and the cross-section of option returns. Working Paper, Hong Kong Polytechnic University.

Carhart, M. M. 1997. On persistence in mutual fund performance. Journal of Finance 52:57–82.

Carr, P., and L. Wu. 2009. Variance risk premiums. Review of Financial Studies 22:1311–41.

Chen, L., M. Pelger, and J. Zhu. Forthcoming. Deep learning in asset pricing. Management Science.

Christoffersen, P., M. Fournier, and K. Jacobs. 2018. The factor structure in equity options. Review of Financial Studies 31:595–637.

Christoffersen, P., R. Goyenko, K. Jacobs, and M. Karoui. 2018. Illiquidity premia in the equity options market. Review of Financial Studies 31:811–51.

Clark, T. E., and K. D. West. 2007. Approximately normal tests for equal predictive accuracy in nested models. Journal of Econometrics 138:291–311.

Cremers, M., M. Halling, and D. Weinbaum. 2015. Aggregate jump and volatility risk in the cross-section of stock returns. Journal of Finance 70:577–614.

DeMiguel, V., J. Gil-Bazo, F. J. Nogales, and A. A. P. Santos. 2021. Can machine learning help to select portfolios of mutual funds? Working Paper, London Business School.

Dennis, P., and S. Mayhew. 2002. Risk-neutral skewness: Evidence from stock options. Journal of Financial and Quantitative Analysis 37:471–93.

Dew-Becker, I., and S. Giglio. 2020. Cross-sectional uncertainty and the business cycle: Evidence from 40 years of options data. Working Paper, Northwestern University.

Dew-Becker, I., S. Giglio, and B. Kelly. 2021. Hedging macroeconomic and financial uncertainty and volatility. Journal of Financial Economics 142:23–45.

Diebold, F. X. 2015. Comparing predictive accuracy, twenty years later: A personal perspective on the use and abuse of Diebold-Mariano tests. Journal of Business & Economic Statistics 33:1–8.

Diebold, F. X., and R. S. Mariano. 1995. Comparing predictive accuracy. Journal of Business & Economic Statistics 20:134–44.

Dittmar, R. F. 2002. Nonlinear pricing kernels, kurtosis preference, and evidence from the cross section of equity returns. Journal of Finance 57:369–403.

Eisdorfer, A., A. Goyal, and A. Zhdanov. 2022. Limited attention and option prices. Working Paper, University of Connecticut.

Engelberg, J., R. D. McLean, and J. Pontiff. 2018. Anomalies and news. Journal of Finance 73:1971–2001.

Fama, E. F., and K. R. French. 1993. Common risk factors in the returns on stocks and bonds. Journal of Financial Economics 33:3–56.

. 2015. A five-factor asset pricing model. Journal of Financial Economics 116:1–22.

. 2018. Choosing factors. Journal of Financial Economics 128:234–52.

Feng, G., S. Giglio, and D. Xiu. 2020. Taming the factor zoo: A test of new factors. Journal of Finance 75:1327–70.

Filippou, I., D. Rapach, M. P. Taylor, and G. Zhou. 2020. Exchange rate prediction with machine learning and a smart carry portfolio. Working Paper, Washington University in St. Louis.

Frazzini, A., and L. H. Pedersen. 2022. Embedded leverage. Review of Asset Pricing Studies 12:1–52.

Freyberger, J., A. Neuhierl, and M. Weber. 2020. Dissecting characteristics nonparametrically. Review of Financial Studies 33:2326–77.

Friedman, J. H. 2001. Greedy function approximation: A gradient boosting machine. Annals of Statistics 29:1189–232.

Garleanu, N., and L. H. Pedersen. 2011. Margin-based asset pricing and deviations from the law of one price. Review of Financial Studies 24:1980–2022.

Garleanu, N., L. H. Pedersen, and A. M. Poteshman. 2009. Demand-based option pricing. Review of Financial Studies 22:4259–99.

Giglio, S., Y. Liao, and D. Xiu. 2021. Thousands of alpha tests. Review of Financial Studies 34:3456–96.

Goyal, A., and A. Saretto. 2009. Cross-section of option returns and volatility. Journal of Financial Economics 94:310–26.

Goyal, A., and I. Welch. 2008. A comprehensive look at the empirical performance of equity premium prediction. Review of Financial Studies 21:1455–508.

Goyenko, R., and C. Zhang. 2021. The joint cross section of option and stock returns predictability with big data and machine learning. Working Paper, McGill University.

Grammig, J., C. Hanenberg, C. Schlag, and J. Sönksen. 2020. Diverging roads: Theory-based vs. machine learning-implied stock risk premia. Working Paper, Universität Tübingen.

Green, J., J. R. M. Hand, and X. F. Zhang. 2017. The characteristics that provide independent information about average U.S. monthly stock returns. Review of Financial Studies 30:4389–436.

Grünthaler, T., F. Lorenz, and P. Meyerhof. Forthcoming. Option-based intermediary leverage. Journal of Banking & Finance.

Gu, S., B. T. Kelly, and D. Xiu. 2020. Empirical asset pricing via machine learning. Review of Financial Studies 33:2223–73.

Han, Y., A. He, D. E. Rapach, and G. Zhou. 2021. Cross-sectional out-of-sample stock return prediction with many characteristics. Working Paper, University of North Carolina at Charlotte.

Heston, S. L. 1993. A closed-form solution for options with stochastic volatility with applications to bond and currency options. Review of Financial Studies 6:327–43.

Heston, S. L., C. S. Jones, M. Khorram, S. Li, and H. Mo. Forthcoming. Option momentum. Journal of Finance.

Heston, S. L., and S. Nandi. 2000. A closed-form Garch option valuation model. Review of Financial Studies 13:585–625.

Heston, S. L., and R. Sadka. 2008. Seasonality in the cross-section of stock returns. Journal of Financial Economics 87:418–45.

Hirshleifer, D., and S. H. Teoh. 2003. Limited attention, information disclosure, and financial reporting. Journal of Accounting and Economics 36:337–86.

Hitzemann, S., M. Hofmann, M. Uhrig-Homburg, and C. Wagner. 2021. Margin requirements and equity option returns. Working Paper, University of Houston.

Hoerl, A. E., and R. W. Kennard. 1970. Ridge regression: Biased estimation for nonorthogonal problems. Technometrics 12:55–67.

Hong, H., and J. C. Stein. 1999. A unified theory of underreaction, momentum trading, and overreaction in asset markets. Journal of Finance 54:2143–84.

Hornik, K., M. Stinchcombe, and H. White. 1989. Multilayer feedforward networks are universal approximators. Neural Networks 2:359–66.

Hull, B., A. Li, and X. Qiao. 2023. Option pricing via breakeven volatility. Financial Analysts Journal 79:99– 119.

Jensen, T. I., B. T. Kelly, and L. H. Pedersen. Forthcoming. Is there a replication crisis in finance? Journal of Finance.

Jondeau, E., and M. Rockinger. 2001. Gram–Charlier densities. Journal of Economic Dynamics and Control 25:1457–83.

Kanne, S., O. Korn, and M. Uhrig-Homburg. 2022. Stock illiquidity and option returns. Journal of Financial Markets. Advance Access published July 13, 2022, 10.1016/j.finmar.2022.100765.

Kelly, B. T., D. Palhares, and S. Pruitt. Forthcoming. Modeling corporate bond returns. Journal of Finance.

Kelly, B. T., S. Pruitt, and Y. Su. 2019. Characteristics are covariances: A unified model of risk and return. Journal of Financial Economics 134:501–24.

. 2020. Instrumented principal component analysis. Working Paper, Yale University.

Keloharju, M., J. T. Linnainmaa, and P. Nyberg. 2016. Return seasonalities. Journal of Finance 71:1557–90.

Kimball, M. S. 1993. Standard risk aversion. Econometrica 61:589–611.

Kozak, S., S. Nagel, and S. Santosh. 2020. Shrinking the cross-section. Journal of Financial Economics 135:271– 92.

Krizhevsky, A., I. Sutskever, and G. E. Hinton. 2012. Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems 25:1097–105.

Lakshminarayanan, B., A. Pritzel, and C. Blundell. 2017. Simple and scalable predictive uncertainty estimation using deep ensembles. In Advances in Neural Information Processing Systems, Vol. 30, eds. Guyon, I., U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, 6405–16. Red Hook, NY: Curran Associates, Inc.

Leippold, M., Q. Wang, and W. Zhou. 2021. Machine-learning in the Chinese factor zoo. Journal of Financial Economics 145:64–82.

Lesmond, D. A., J. P. Ogden, and C. A. Trzcinka. 1999. A new estimate of transaction costs. Review of Financial Studies

Lettau, M., and M. Pelger. 2020. Factors that fit the time series and cross-section of stock returns. Review of Financial Studies 33:2274–325.

Li, B., and A. G. Rossi. 2020. Selecting mutual funds from the stocks they hold: A machine learning approach. Working Paper, Wuhan University.

Lu, Z., and S. Murray. 2019. Bear beta. Journal of Financial Economics 131:736–60.

Lundberg, S. M., and S.-I. Lee. 2017. A unified approach to interpreting model predictions. In Proceedings of the 31st International Conference on Neural Information Processing Systems, 4768–77.

Martin, I., and S. Nagel. 2022. Market efficiency in the age of big data. Journal of Financial Economics 145:154–77.

Moritz, B., and T. Zimmermann. 2016. Tree-based conditional portfolio sorts: The relation between past and future stock returns. Working Paper, LMU Munich.

Muravyev, D., and N. D. Pearson. 2020. Options trading costs are lower than you think. Review of Financial Studies 33:4973–5014.

Murray, S., H. Xiao, and Y. Xia. 2021. Charting by machines. Working Paper, Georgia State University.

Nagel, S. 2005. Short sales, institutional investors and the cross-section of stock returns. Journal of Financial Economics 78:277–309.

. 2021. Machine learning in asset pricing, 1st ed. Princeton, NJ: Princeton University Press.

Neuhierl, A., X. Tang, R. T. Varneskov, and G. Zhou. 2021. Option characteristics as cross-sectional predictors. Working Paper, Washington University in St. Louis.

Newey, W. K., and K. D. West. 1987. A simple, positive semi-definite, heteroskedasticity and autocorrelation consistent covariance matrix. Econometrica 55:703–8.

Ofek, E., M. Richardson, and R. F. Whitelaw. 2004. Limited arbitrage and short sales restrictions: Evidence from the options markets. Journal of Financial Economics 74:305–42.

Pástor, L., and R. F. Stambaugh. 2003. Liquidity risk and expected stock returns. Journal of Political Economy 111:642–85.

Pontiff, J. 2006. Costly arbitrage and the myth of idiosyncratic risk. Journal of Accounting and Economics 42:35–52.

Pratt, J. W., and R. J. Zeckhauser. 1987. Proper risk aversion. Econometrica 55:143–54.

Qu, R., A. Timmermann, and Y. Zhu. 2022. Comparing forecasting performance with panel data. Working Paper, International Monetary Fund.

Ramachandran, L. S., and J. Tayal. 2021. Mispricing, short-sale constraints, and the cross-section of option returns. Journal of Financial Economics 141:297–321.

Rapach, D. E., J. K. Strauss, and G. Zhou. 2010. Out-of-sample equity premium prediction: Combination forecasts and links to the real economy. Review of Financial Studies 23:821–62.

. 2013. International stock return predictability: What is the role of the United States? Journal of Finance 68:1633–62.

Roll, R., E. Schwartz, and A. Subrahmanyam. 2010. O/S: The relative trading activity in options and stock. Journal of Financial Economics 96:1–17.

Schlögl, E. 2013. Option pricing where the underlying assets follow a Gram/Charlier density of arbitrary order. Journal of Economic Dynamics and Control 37:611–32.

Shleifer, A., and R. W. Vishny. 1997. The limits of arbitrage. Journal of Finance 52:35–55.

Steel, M. F. 2020. Model averaging and its use in economics. Journal of Economic Literature 58:644–719.

Tian, M., and L. Wu. Forthcoming. Limits of arbitrage and primary risk taking in derivative securities. Review of Asset Pricing Studies.

Tibshirani, R. 1996. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society: Series B (Methodological) 58:267–88.

Vasquez, A. 2017. Equity volatility term structures and the cross section of option returns. Journal of Financial and Quantitative Analysis 52:2727–54.

Vasquez, A., and X. Xiao. 2021. Default risk and option returns. Working Paper, Instituto Tecnológico Autónomo de México.

Vilkov, G., and Y. Xiao. 2012. Option-implied information and predictability of extreme returns. Working Paper, Frankfurt School of Finance & Management.

Vinayak, R. K., and R. Gilad-Bachrach. 2015. Dart: Dropouts meet multiple additive regression trees. Proceedings of Machine Learning Research 38:489–97.

Wu, W., J. Chen, Z. Yang, and M. L. Tindall. 2021. A cross-sectional machine learning approach for hedge fund return prediction and selection. Management Science 67:4577–601.

Xing, Y., X. Zhang, and R. Zhao. 2010. What does the individual option volatility smirk tell us about future equity returns? Journal of Financial and Quantitative Analysis 45:641–62.

Zhan, X. E., B. Han, J. Cao, and Q. Tong. 2021. Option return predictability. Review of Financial Studies 35:1394–442.

Zhang, X. F. 2006. Information uncertainty and stock returns. Journal of Finance 61:105–37.

RFS-使用机器学习和大数据预测期权价格-英-55页-WN10