-
-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Description
Hi,
I recently noticed a potential issue with qs.reports.metrics/html when a value is provided for the benchmark parameter. Specifically, some metrics in the results seem to show inconsistencies.
To better understand this, I conducted a comparison by passing both benchmark and return together, and then using only return to call qs.reports.metrics. The "Same?" column in my analysis highlights the comparison between the "Return with Benchmark" and "Only Return" outputs.
It is worth noting that metrics such as "R^2" and "Information Ratio" are missing when using only return. Since these metrics rely on both return and benchmark return for their calculations, I would like to suggest displaying "-" for benchmark, similar to the method used for Beta and Alpha.
Additionally, metrics like "Kelly Criterion," "Expected Shortfall (cVaR)," "Payoff Ratio," "Outlier Win Ratio," "Outlier Loss Ratio," "Serenity Index," "Avg. Up Month," and "Avg. Down Month" yield differing results when a benchmark is included compared to when they are calculated individually. This discrepancy could potentially relate to the issue I previously mentioned in "The calculation result of CVAR in the HTML report appears to be questionable #467." It seems there may be variations in the logic applied during calculations, depending on whether a DataFrame or Series is used.
| Benchmark | Return with Benchmark | Only Return | Same? | |
|---|---|---|---|---|
| Start Period | 2020/2/27 | 2020/2/27 | 2020/2/27 | Y |
| End Period | 2025/11/7 | 2025/11/7 | 2025/11/7 | Y |
| Risk-Free Rate | 1.81 | 1.81 | 1.81 | Y |
| Time in Market | 99 | 99 | 99 | Y |
| Cumulative Return | 30.57 | 210.24 | 210.24 | Y |
| CAGR﹪ | 4.98 | 22.91 | 22.91 | Y |
| Sharpe | 0.26 | 1.89 | 1.89 | Y |
| Prob. Sharpe Ratio | 56.36 | 100 | 100 | Y |
| Smart Sharpe | 0.24 | 1.75 | 1.75 | Y |
| Sortino | 0.38 | 2.9 | 2.9 | Y |
| Smart Sortino | 0.35 | 2.69 | 2.69 | Y |
| Sortino/√2 | 0.27 | 2.05 | 2.05 | Y |
| Smart Sortino/√2 | 0.25 | 1.9 | 1.9 | Y |
| Omega | 1.4 | 1.4 | 1.4 | Y |
| Max Drawdown | -45.6 | -6.49 | -6.49 | Y |
| Max DD Date | 2024/9/13 | 2022/7/15 | 2022/7/15 | Y |
| Max DD Period Start | 2021/2/18 | 2022/6/13 | 2022/6/13 | Y |
| Max DD Period End | 2025/11/7 | 2022/9/9 | 2022/9/9 | Y |
| Longest DD Days | 1724 | 168 | 168 | Y |
| Volatility (ann.) | 18.38 | 10.26 | 10.26 | Y |
| R^2 | 0.23 | 0.23 | N | |
| Information Ratio | 0.06 | 0.06 | N | |
| Calmar | 0.11 | 3.53 | 3.53 | Y |
| Skew | 0.12 | -0.05 | -0.05 | Y |
| Kurtosis | 5.8 | 4.82 | 4.82 | Y |
| Expected Daily | 0.02 | 0.08 | 0.08 | Y |
| Expected Monthly | 0.38 | 1.63 | 1.63 | Y |
| Expected Yearly | 4.55 | 20.77 | 20.77 | Y |
| Kelly Criterion | -1.83 | 17.09 | 17.5 | N |
| Risk of Ruin | 0 | 0 | 0 | Y |
| Daily Value-at-Risk | -1.88 | -0.98 | -0.98 | Y |
| Expected Shortfall (cVaR) | -2.8 | -2.8 | -1.54 | N |
| Max Consecutive Wins | 10 | 9 | 9 | Y |
| Max Consecutive Losses | 7 | 9 | 9 | Y |
| Gain/Pain Ratio | 0.07 | 0.44 | 0.44 | Y |
| Gain/Pain (1M) | 0.32 | 6.78 | 6.78 | Y |
| Payoff Ratio | 0.94 | 1.06 | 1.07 | N |
| Profit Factor | 1.07 | 1.44 | 1.44 | Y |
| Common Sense Ratio | 1.22 | 1.73 | 1.73 | Y |
| CPC Index | 0.51 | 0.88 | 0.88 | Y |
| Tail Ratio | 1.14 | 1.2 | 1.2 | Y |
| Outlier Win Ratio | 2.96 | 5.1 | 3.86 | N |
| Outlier Loss Ratio | 2.92 | 5.27 | 3.71 | N |
| MTD | 0 | 0.35 | 0.35 | Y |
| 3M | 15.42 | 7.53 | 7.53 | Y |
| 6M | 24.66 | 12.18 | 12.18 | Y |
| YTD | 20.66 | 16.42 | 16.42 | Y |
| 1Y | 17.98 | 17.86 | 17.86 | Y |
| 3Y (ann.) | 6.6 | 19.36 | 19.36 | Y |
| 5Y (ann.) | -1.36 | 18.05 | 18.05 | Y |
| 10Y (ann.) | 4.98 | 22.91 | 22.91 | Y |
| All-time (ann.) | 4.98 | 22.91 | 22.91 | Y |
| Best Day | 8.48 | 3.49 | 3.49 | Y |
| Worst Day | -7.05 | -4.05 | -4.05 | Y |
| Best Month | 20.97 | 14.69 | 14.69 | Y |
| Worst Month | -7.9 | -4.71 | -4.71 | Y |
| Best Year | 43.31 | 44.05 | 44.05 | Y |
| Worst Year | -21.63 | 13.45 | 13.45 | Y |
| Avg. Drawdown | -4.02 | -1.24 | -1.24 | Y |
| Avg. Drawdown Days | 90 | 13 | 13 | Y |
| Recovery Factor | 0.79 | 17.89 | 17.89 | Y |
| Ulcer Index | 0.28 | 0.02 | 0.02 | Y |
| Serenity Index | 0.03 | 12.94 | 9.9 | N |
| Avg. Up Month | 4.53 | 3.56 | 2.49 | N |
| Avg. Down Month | -3.32 | -1.38 | -1.16 | N |
| Win Days | 50.7 | 57.31 | 57.31 | Y |
| Win Month | 51.47 | 78.26 | 78.26 | Y |
| Win Quarter | 58.33 | 91.67 | 91.67 | Y |
| Win Year | 50 | 100 | 100 | Y |
| Beta | - | 0.27 | ||
| Alpha | - | 0.19 | ||
| Correlation | - | 47.82% | ||
| Treynor Ratio | - | 780.77% |