Commit b6f6286
Fixing flaky tests (#781)
Summary:
This is a fix for another set of flaky tests: `test_multi_inp_shapley_batch_scalar_tensor_0d`, `test_multi_inp_shapley_batch_scalar_float` and `test_multi_inp_shapley_batch_scalar_tensor_1d` in `test_shapley.py`. Note that `test_multi_inp_shapley_batch_scalar_int` was not flaky to begin with.
I observed that increasing `n_samples` up to 3500 reduces flakiness by a lot. This increases the runtime from about 1.7s to 4.5s though. The tests only failed about 6/7 times out of 1000 runs with only the `n_samples` change. Hence, I also increased the `delta` to 1.2 based on the observed failures to bring that to ~0%. Let me know in case you would prefer lower `n_samples` but higher `delta` to reduce the runtime.
Thanks!
cc vivekmig
Pull Request resolved: #781
Reviewed By: NarineK, Reubend
Differential Revision: D31588284
Pulled By: vivekmig
fbshipit-source-id: d398ad5b54629f6e472309e9b66352b2e84546911 parent 6a1e261 commit b6f6286
1 file changed
+2
-1
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
372 | 372 | | |
373 | 373 | | |
374 | 374 | | |
375 | | - | |
| 375 | + | |
| 376 | + | |
376 | 377 | | |
377 | 378 | | |
378 | 379 | | |
| |||
0 commit comments