[Major] major refactoring of collectors

[vmoens]

No description provided.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3233

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 13 New Failures, 8 Unrelated Failures

As of commit 0ca9928 with merge base 8570c25 ():

NEW FAILURES - The following jobs have failed:

• Continuous Benchmark (PR) / GPU Pytest benchmark (gh)
  Process completed with exit code 139.
• Generate documentation / build-docs (3.12, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t 0172332d16b74b20f978ae274401bb8fe9500e76b639c288013eb9fc711eec1e /exec failed with exit code 2
• Habitat Tests on Linux / tests (3.9, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t 9ff42d7f4b2a01100a6c1338ac07470038d700907014212c31048ebf95c9e394 /exec failed with exit code 1
• Libs Tests on Linux / unittests-gym (3.9, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t cd2f5c358fc812dad40b921ee20d3b1dd340cc6ad84674fd85254d2993ca1f77 /exec failed with exit code 1
• SOTA Tests on Linux / tests (3.9, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t b230a8b31e1bb9529b160985e7d8f48dff22816de5d0d757a74e72428b01d5f0 /exec failed with exit code 1
• Unit-tests on Linux / tests-cpu (3.10) / linux-job (gh)
  test/test_transforms.py::TestRayModuleTransform::test_ray_module_transform_scheme_collector
• Unit-tests on Linux / tests-cpu (3.11) / linux-job (gh)
  test/test_transforms.py::TestRayModuleTransform::test_ray_module_transform_scheme_collector
• Unit-tests on Linux / tests-cpu (3.12) / linux-job (gh)
  test/test_transforms.py::TestRayModuleTransform::test_ray_module_transform_scheme_collector
• Unit-tests on Linux / tests-cpu (3.9) / linux-job (gh)
  test/test_transforms.py::TestRayModuleTransform::test_ray_module_transform_scheme_collector
• Unit-tests on Linux / tests-gpu (3.11, 12.8) / linux-job (gh)
  test/test_transforms.py::TestRayModuleTransform::test_ray_module_transform_scheme_collector
• Unit-tests on Linux / tests-olddeps (3.9, 11.6) / linux-job (gh)
  test/test_weightsync.py::TestSharedMemWeightSyncScheme::test_sharedmem_flow
• Unit-tests on Linux / tests-optdeps (3.11, 12.8) / linux-job (gh)
  test/test_transforms.py::TestRayModuleTransform::test_ray_module_transform_scheme_collector
• Unit-tests on Linux / tests-stable-gpu (3.10, 11.8) / linux-job (gh)
  test/test_transforms.py::TestRayModuleTransform::test_ray_module_transform_scheme_collector

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Continuous Benchmark / GPU Pytest benchmark (gh) (trunk failure)
  Process completed with exit code 139.
• Push Binary Nightly / build-wheel-unix (linux, ubuntu-22.04, 3.9, cp39-cp39, cpu, cpu) (gh) (trunk failure)
  Process completed with exit code 1.
• Push Binary Nightly / build-wheel-unix (macos, macos-latest, 3.9, cp39-cp39, cpu, cpu) (gh) (trunk failure)
  Process completed with exit code 1.
• Push Binary Nightly / build-wheel-windows (3.9, 3.9) (gh) (trunk failure)
  Process completed with exit code 1.
• Push Binary Nightly / test-wheel-unix (linux, ubuntu-22.04, 3.9, cp39-cp39, cpu, cpu) (gh) (trunk failure)
  Process completed with exit code 1.
• Push Binary Nightly / test-wheel-unix (macos, macos-latest, 3.9, cp39-cp39, cpu, cpu) (gh) (trunk failure)
  Process completed with exit code 1.
• Push Binary Nightly / test-wheel-windows (3.9, 3.9) (gh) (trunk failure)
  Process completed with exit code 1.
• Unit-tests on Windows / unittests-cpu (3.10, windows.4xlarge, cpu) / windows-job (gh) (trunk failure)
  test/test_cost.py::TestREDQ::test_sac_prioritized_weights

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 154. Improved: $\large\color{#35bf28}20$. Worsened: $\large\color{#d91a1a}7$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_tensor_to_bytestream_speed[pickle]	82.2181μs	80.1444μs	12.4775 KOps/s	12.3604 KOps/s	$\color{#35bf28}+0.95\%$
test_tensor_to_bytestream_speed[torch.save]	0.1391ms	0.1381ms	7.2398 KOps/s	7.1163 KOps/s	$\color{#35bf28}+1.74\%$
test_tensor_to_bytestream_speed[untyped_storage]	0.1120s	0.1118s	8.9472 Ops/s	8.7654 Ops/s	$\color{#35bf28}+2.07\%$
test_tensor_to_bytestream_speed[numpy]	2.6217μs	2.6179μs	381.9809 KOps/s	371.8140 KOps/s	$\color{#35bf28}+2.73\%$
test_tensor_to_bytestream_speed[safetensors]	40.8759μs	40.4720μs	24.7084 KOps/s	24.9032 KOps/s	$\color{#d91a1a}-0.78\%$
test_simple	0.5355s	0.5343s	1.8714 Ops/s	1.7531 Ops/s	$\textbf{\color{#35bf28}+6.75\%}$
test_transformed	1.0937s	1.0923s	0.9155 Ops/s	0.8892 Ops/s	$\color{#35bf28}+2.96\%$
test_serial	1.6172s	1.6110s	0.6207 Ops/s	0.5977 Ops/s	$\color{#35bf28}+3.85\%$
test_parallel	1.1445s	1.0683s	0.9361 Ops/s	0.9360 Ops/s	$+0.01\%$
test_step_mdp_speed[True-True-True-True-True]	0.1804ms	42.2150μs	23.6883 KOps/s	23.0700 KOps/s	$\color{#35bf28}+2.68\%$
test_step_mdp_speed[True-True-True-True-False]	53.1110μs	23.4619μs	42.6223 KOps/s	41.0799 KOps/s	$\color{#35bf28}+3.75\%$
test_step_mdp_speed[True-True-True-False-True]	57.1400μs	23.3894μs	42.7544 KOps/s	41.9987 KOps/s	$\color{#35bf28}+1.80\%$
test_step_mdp_speed[True-True-True-False-False]	36.8710μs	13.0724μs	76.4972 KOps/s	75.5360 KOps/s	$\color{#35bf28}+1.27\%$
test_step_mdp_speed[True-True-False-True-True]	83.5710μs	45.0571μs	22.1940 KOps/s	22.3482 KOps/s	$\color{#d91a1a}-0.69\%$
test_step_mdp_speed[True-True-False-True-False]	60.6010μs	26.2723μs	38.0629 KOps/s	38.2450 KOps/s	$\color{#d91a1a}-0.48\%$
test_step_mdp_speed[True-True-False-False-True]	52.2210μs	25.8293μs	38.7158 KOps/s	38.1775 KOps/s	$\color{#35bf28}+1.41\%$
test_step_mdp_speed[True-True-False-False-False]	41.5410μs	15.5550μs	64.2880 KOps/s	62.2597 KOps/s	$\color{#35bf28}+3.26\%$
test_step_mdp_speed[True-False-True-True-True]	78.7910μs	48.0932μs	20.7930 KOps/s	20.8364 KOps/s	$\color{#d91a1a}-0.21\%$
test_step_mdp_speed[True-False-True-True-False]	56.1310μs	28.7674μs	34.7616 KOps/s	34.5537 KOps/s	$\color{#35bf28}+0.60\%$
test_step_mdp_speed[True-False-True-False-True]	58.9200μs	26.0877μs	38.3323 KOps/s	38.2912 KOps/s	$\color{#35bf28}+0.11\%$
test_step_mdp_speed[True-False-True-False-False]	40.6000μs	15.4848μs	64.5796 KOps/s	62.3526 KOps/s	$\color{#35bf28}+3.57\%$
test_step_mdp_speed[True-False-False-True-True]	87.0610μs	49.4188μs	20.2352 KOps/s	19.7453 KOps/s	$\color{#35bf28}+2.48\%$
test_step_mdp_speed[True-False-False-True-False]	62.8110μs	31.4825μs	31.7637 KOps/s	31.1987 KOps/s	$\color{#35bf28}+1.81\%$
test_step_mdp_speed[True-False-False-False-True]	63.8810μs	29.0170μs	34.4626 KOps/s	34.2280 KOps/s	$\color{#35bf28}+0.69\%$
test_step_mdp_speed[True-False-False-False-False]	46.7310μs	17.9610μs	55.6760 KOps/s	54.4178 KOps/s	$\color{#35bf28}+2.31\%$
test_step_mdp_speed[False-True-True-True-True]	82.0910μs	47.5368μs	21.0363 KOps/s	21.1039 KOps/s	$\color{#d91a1a}-0.32\%$
test_step_mdp_speed[False-True-True-True-False]	76.8020μs	28.7131μs	34.8273 KOps/s	34.3952 KOps/s	$\color{#35bf28}+1.26\%$
test_step_mdp_speed[False-True-True-False-True]	2.5745ms	30.4456μs	32.8455 KOps/s	33.3126 KOps/s	$\color{#d91a1a}-1.40\%$
test_step_mdp_speed[False-True-True-False-False]	37.7610μs	17.3341μs	57.6899 KOps/s	56.8920 KOps/s	$\color{#35bf28}+1.40\%$
test_step_mdp_speed[False-True-False-True-True]	82.1910μs	50.3938μs	19.8437 KOps/s	19.5115 KOps/s	$\color{#35bf28}+1.70\%$
test_step_mdp_speed[False-True-False-True-False]	53.2010μs	31.8036μs	31.4430 KOps/s	31.3092 KOps/s	$\color{#35bf28}+0.43\%$
test_step_mdp_speed[False-True-False-False-True]	85.9310μs	32.7065μs	30.5749 KOps/s	30.6720 KOps/s	$\color{#d91a1a}-0.32\%$
test_step_mdp_speed[False-True-False-False-False]	47.6210μs	20.0390μs	49.9028 KOps/s	50.3466 KOps/s	$\color{#d91a1a}-0.88\%$
test_step_mdp_speed[False-False-True-True-True]	80.1810μs	52.7439μs	18.9595 KOps/s	18.8339 KOps/s	$\color{#35bf28}+0.67\%$
test_step_mdp_speed[False-False-True-True-False]	64.2310μs	34.1808μs	29.2562 KOps/s	29.1498 KOps/s	$\color{#35bf28}+0.36\%$
test_step_mdp_speed[False-False-True-False-True]	68.3410μs	32.1408μs	31.1131 KOps/s	31.0042 KOps/s	$\color{#35bf28}+0.35\%$
test_step_mdp_speed[False-False-True-False-False]	46.7210μs	19.6218μs	50.9636 KOps/s	48.7644 KOps/s	$\color{#35bf28}+4.51\%$
test_step_mdp_speed[False-False-False-True-True]	88.3720μs	55.1304μs	18.1388 KOps/s	18.0522 KOps/s	$\color{#35bf28}+0.48\%$
test_step_mdp_speed[False-False-False-True-False]	64.0610μs	36.4901μs	27.4047 KOps/s	27.1254 KOps/s	$\color{#35bf28}+1.03\%$
test_step_mdp_speed[False-False-False-False-True]	97.6110μs	34.6240μs	28.8817 KOps/s	28.9361 KOps/s	$\color{#d91a1a}-0.19\%$
test_step_mdp_speed[False-False-False-False-False]	50.7310μs	22.0104μs	45.4331 KOps/s	44.2022 KOps/s	$\color{#35bf28}+2.78\%$
test_values[generalized_advantage_estimate-True-True]	10.7586ms	9.9359ms	100.6450 Ops/s	95.6609 Ops/s	$\textbf{\color{#35bf28}+5.21\%}$
test_values[vec_generalized_advantage_estimate-True-True]	19.7958ms	18.4920ms	54.0775 Ops/s	55.1341 Ops/s	$\color{#d91a1a}-1.92\%$
test_values[td0_return_estimate-False-False]	0.2798ms	0.1282ms	7.8006 KOps/s	7.5715 KOps/s	$\color{#35bf28}+3.03\%$
test_values[td1_return_estimate-False-False]	26.6914ms	26.2986ms	38.0248 Ops/s	35.5582 Ops/s	$\textbf{\color{#35bf28}+6.94\%}$
test_values[vec_td1_return_estimate-False-False]	20.3944ms	18.7798ms	53.2487 Ops/s	54.5773 Ops/s	$\color{#d91a1a}-2.43\%$
test_values[td_lambda_return_estimate-True-False]	42.0706ms	39.3293ms	25.4263 Ops/s	23.7834 Ops/s	$\textbf{\color{#35bf28}+6.91\%}$
test_values[vec_td_lambda_return_estimate-True-False]	19.4255ms	18.4705ms	54.1404 Ops/s	54.6066 Ops/s	$\color{#d91a1a}-0.85\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	8.5869ms	8.5257ms	117.2924 Ops/s	110.3826 Ops/s	$\textbf{\color{#35bf28}+6.26\%}$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	2.0063ms	1.5857ms	630.6469 Ops/s	637.9961 Ops/s	$\color{#d91a1a}-1.15\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.4555ms	0.4072ms	2.4560 KOps/s	2.3348 KOps/s	$\textbf{\color{#35bf28}+5.19\%}$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	35.1222ms	34.5224ms	28.9667 Ops/s	34.0863 Ops/s	$\textbf{\color{#d91a1a}-15.02\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	2.1682ms	1.7335ms	576.8615 Ops/s	585.7348 Ops/s	$\color{#d91a1a}-1.51\%$
test_dqn_speed[False-None]	1.7598ms	1.3727ms	728.4787 Ops/s	708.9088 Ops/s	$\color{#35bf28}+2.76\%$
test_dqn_speed[False-backward]	2.0238ms	1.8837ms	530.8607 Ops/s	520.0145 Ops/s	$\color{#35bf28}+2.09\%$
test_dqn_speed[True-None]	0.8911ms	0.5192ms	1.9261 KOps/s	1.8677 KOps/s	$\color{#35bf28}+3.12\%$
test_dqn_speed[True-backward]	1.0344ms	0.9540ms	1.0483 KOps/s	943.0725 Ops/s	$\textbf{\color{#35bf28}+11.15\%}$
test_dqn_speed[reduce-overhead-None]	0.8955ms	0.5090ms	1.9646 KOps/s	1.8486 KOps/s	$\textbf{\color{#35bf28}+6.28\%}$
test_dqn_speed[reduce-overhead-backward]	0.9795ms	0.9381ms	1.0660 KOps/s	903.0989 Ops/s	$\textbf{\color{#35bf28}+18.04\%}$
test_ddpg_speed[False-None]	3.1994ms	2.8202ms	354.5903 Ops/s	353.5647 Ops/s	$\color{#35bf28}+0.29\%$
test_ddpg_speed[False-backward]	4.2526ms	4.0612ms	246.2323 Ops/s	244.1982 Ops/s	$\color{#35bf28}+0.83\%$
test_ddpg_speed[True-None]	1.5272ms	1.3502ms	740.6122 Ops/s	720.7970 Ops/s	$\color{#35bf28}+2.75\%$
test_ddpg_speed[True-backward]	2.3788ms	2.3259ms	429.9402 Ops/s	369.8551 Ops/s	$\textbf{\color{#35bf28}+16.25\%}$
test_ddpg_speed[reduce-overhead-None]	1.7342ms	1.3566ms	737.1566 Ops/s	717.8459 Ops/s	$\color{#35bf28}+2.69\%$
test_ddpg_speed[reduce-overhead-backward]	2.4925ms	2.3292ms	429.3246 Ops/s	371.4161 Ops/s	$\textbf{\color{#35bf28}+15.59\%}$
test_sac_speed[False-None]	8.3474ms	7.7682ms	128.7298 Ops/s	127.6520 Ops/s	$\color{#35bf28}+0.84\%$
test_sac_speed[False-backward]	11.3370ms	10.9320ms	91.4748 Ops/s	88.8081 Ops/s	$\color{#35bf28}+3.00\%$
test_sac_speed[True-None]	2.6043ms	2.1070ms	474.5989 Ops/s	448.8304 Ops/s	$\textbf{\color{#35bf28}+5.74\%}$
test_sac_speed[True-backward]	4.1530ms	4.0193ms	248.7988 Ops/s	225.5486 Ops/s	$\textbf{\color{#35bf28}+10.31\%}$
test_sac_speed[reduce-overhead-None]	2.1893ms	2.0881ms	478.8945 Ops/s	446.5861 Ops/s	$\textbf{\color{#35bf28}+7.23\%}$
test_sac_speed[reduce-overhead-backward]	4.1816ms	4.0091ms	249.4304 Ops/s	213.8359 Ops/s	$\textbf{\color{#35bf28}+16.65\%}$
test_redq_speed[False-None]	13.5697ms	10.3739ms	96.3954 Ops/s	92.2029 Ops/s	$\color{#35bf28}+4.55\%$
test_redq_speed[False-backward]	20.2830ms	18.0521ms	55.3951 Ops/s	55.6092 Ops/s	$\color{#d91a1a}-0.38\%$
test_redq_speed[True-None]	4.9398ms	4.5127ms	221.5983 Ops/s	223.5035 Ops/s	$\color{#d91a1a}-0.85\%$
test_redq_speed[True-backward]	10.3885ms	10.1448ms	98.5723 Ops/s	97.1260 Ops/s	$\color{#35bf28}+1.49\%$
test_redq_speed[reduce-overhead-None]	4.6648ms	4.4593ms	224.2489 Ops/s	224.9511 Ops/s	$\color{#d91a1a}-0.31\%$
test_redq_speed[reduce-overhead-backward]	13.3492ms	10.8977ms	91.7621 Ops/s	97.7088 Ops/s	$\textbf{\color{#d91a1a}-6.09\%}$
test_redq_deprec_speed[False-None]	11.6214ms	10.9599ms	91.2419 Ops/s	90.9601 Ops/s	$\color{#35bf28}+0.31\%$
test_redq_deprec_speed[False-backward]	16.4878ms	16.0233ms	62.4091 Ops/s	62.3051 Ops/s	$\color{#35bf28}+0.17\%$
test_redq_deprec_speed[True-None]	4.4362ms	3.6727ms	272.2774 Ops/s	272.7190 Ops/s	$\color{#d91a1a}-0.16\%$
test_redq_deprec_speed[True-backward]	8.0261ms	7.8214ms	127.8538 Ops/s	127.2638 Ops/s	$\color{#35bf28}+0.46\%$
test_redq_deprec_speed[reduce-overhead-None]	4.2967ms	3.6085ms	277.1200 Ops/s	272.1228 Ops/s	$\color{#35bf28}+1.84\%$
test_redq_deprec_speed[reduce-overhead-backward]	7.9935ms	7.8054ms	128.1162 Ops/s	128.4698 Ops/s	$\color{#d91a1a}-0.28\%$
test_td3_speed[False-None]	8.0015ms	7.7545ms	128.9581 Ops/s	127.4245 Ops/s	$\color{#35bf28}+1.20\%$
test_td3_speed[False-backward]	11.1015ms	10.6071ms	94.2761 Ops/s	92.9949 Ops/s	$\color{#35bf28}+1.38\%$
test_td3_speed[True-None]	2.2201ms	1.8073ms	553.3166 Ops/s	548.7960 Ops/s	$\color{#35bf28}+0.82\%$
test_td3_speed[True-backward]	3.7264ms	3.5953ms	278.1408 Ops/s	275.0883 Ops/s	$\color{#35bf28}+1.11\%$
test_td3_speed[reduce-overhead-None]	1.8416ms	1.7800ms	561.8102 Ops/s	555.5332 Ops/s	$\color{#35bf28}+1.13\%$
test_td3_speed[reduce-overhead-backward]	3.8283ms	3.6713ms	272.3837 Ops/s	272.0858 Ops/s	$\color{#35bf28}+0.11\%$
test_cql_speed[False-None]	29.0544ms	26.2860ms	38.0431 Ops/s	38.5728 Ops/s	$\color{#d91a1a}-1.37\%$
test_cql_speed[False-backward]	36.1753ms	35.2596ms	28.3611 Ops/s	28.3850 Ops/s	$\color{#d91a1a}-0.08\%$
test_cql_speed[True-None]	12.7664ms	12.4464ms	80.3444 Ops/s	78.0110 Ops/s	$\color{#35bf28}+2.99\%$
test_cql_speed[True-backward]	19.3594ms	18.7982ms	53.1966 Ops/s	53.2813 Ops/s	$\color{#d91a1a}-0.16\%$
test_cql_speed[reduce-overhead-None]	16.9146ms	12.9101ms	77.4587 Ops/s	79.4779 Ops/s	$\color{#d91a1a}-2.54\%$
test_cql_speed[reduce-overhead-backward]	19.5129ms	18.8406ms	53.0769 Ops/s	54.9932 Ops/s	$\color{#d91a1a}-3.48\%$
test_a2c_speed[False-None]	6.5411ms	5.4149ms	184.6753 Ops/s	184.5060 Ops/s	$\color{#35bf28}+0.09\%$
test_a2c_speed[False-backward]	12.3583ms	11.9434ms	83.7279 Ops/s	83.1039 Ops/s	$\color{#35bf28}+0.75\%$
test_a2c_speed[True-None]	4.0658ms	3.7309ms	268.0327 Ops/s	267.3200 Ops/s	$\color{#35bf28}+0.27\%$
test_a2c_speed[True-backward]	8.9107ms	8.7208ms	114.6680 Ops/s	114.5089 Ops/s	$\color{#35bf28}+0.14\%$
test_a2c_speed[reduce-overhead-None]	4.1315ms	3.7047ms	269.9244 Ops/s	268.8946 Ops/s	$\color{#35bf28}+0.38\%$
test_a2c_speed[reduce-overhead-backward]	9.2116ms	8.9129ms	112.1967 Ops/s	111.4856 Ops/s	$\color{#35bf28}+0.64\%$
test_ppo_speed[False-None]	6.2508ms	5.8459ms	171.0593 Ops/s	168.7476 Ops/s	$\color{#35bf28}+1.37\%$
test_ppo_speed[False-backward]	13.0882ms	12.6680ms	78.9393 Ops/s	78.1837 Ops/s	$\color{#35bf28}+0.97\%$
test_ppo_speed[True-None]	4.0338ms	3.6161ms	276.5430 Ops/s	269.2996 Ops/s	$\color{#35bf28}+2.69\%$
test_ppo_speed[True-backward]	8.8509ms	8.5702ms	116.6840 Ops/s	111.9500 Ops/s	$\color{#35bf28}+4.23\%$
test_ppo_speed[reduce-overhead-None]	4.1373ms	3.6616ms	273.1010 Ops/s	266.4154 Ops/s	$\color{#35bf28}+2.51\%$
test_ppo_speed[reduce-overhead-backward]	9.2839ms	8.8476ms	113.0256 Ops/s	110.0459 Ops/s	$\color{#35bf28}+2.71\%$
test_reinforce_speed[False-None]	4.7158ms	4.5349ms	220.5111 Ops/s	215.3841 Ops/s	$\color{#35bf28}+2.38\%$
test_reinforce_speed[False-backward]	7.7014ms	7.3515ms	136.0257 Ops/s	132.7250 Ops/s	$\color{#35bf28}+2.49\%$
test_reinforce_speed[True-None]	3.2377ms	2.8589ms	349.7797 Ops/s	339.1527 Ops/s	$\color{#35bf28}+3.13\%$
test_reinforce_speed[True-backward]	8.0629ms	7.7650ms	128.7830 Ops/s	127.2656 Ops/s	$\color{#35bf28}+1.19\%$
test_reinforce_speed[reduce-overhead-None]	2.9851ms	2.8541ms	350.3684 Ops/s	340.6555 Ops/s	$\color{#35bf28}+2.85\%$
test_reinforce_speed[reduce-overhead-backward]	8.5813ms	7.9470ms	125.8341 Ops/s	123.4722 Ops/s	$\color{#35bf28}+1.91\%$
test_iql_speed[False-None]	26.2547ms	20.3161ms	49.2220 Ops/s	50.9219 Ops/s	$\color{#d91a1a}-3.34\%$
test_iql_speed[False-backward]	36.7890ms	30.7982ms	32.4694 Ops/s	32.5826 Ops/s	$\color{#d91a1a}-0.35\%$
test_iql_speed[True-None]	11.1431ms	8.6849ms	115.1427 Ops/s	114.6691 Ops/s	$\color{#35bf28}+0.41\%$
test_iql_speed[True-backward]	17.5096ms	16.9914ms	58.8534 Ops/s	58.5258 Ops/s	$\color{#35bf28}+0.56\%$
test_iql_speed[reduce-overhead-None]	9.0335ms	8.6548ms	115.5432 Ops/s	112.6422 Ops/s	$\color{#35bf28}+2.58\%$
test_iql_speed[reduce-overhead-backward]	22.7894ms	18.1453ms	55.1107 Ops/s	57.6001 Ops/s	$\color{#d91a1a}-4.32\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	7.3865ms	5.8609ms	170.6212 Ops/s	170.0516 Ops/s	$\color{#35bf28}+0.33\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.5806ms	0.3316ms	3.0157 KOps/s	3.5211 KOps/s	$\textbf{\color{#d91a1a}-14.35\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.5988ms	0.3066ms	3.2618 KOps/s	3.3108 KOps/s	$\color{#d91a1a}-1.48\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	5.9126ms	5.5862ms	179.0113 Ops/s	177.1921 Ops/s	$\color{#35bf28}+1.03\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	2.1557ms	0.3358ms	2.9779 KOps/s	3.5780 KOps/s	$\textbf{\color{#d91a1a}-16.77\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.5814ms	0.2904ms	3.4439 KOps/s	3.7223 KOps/s	$\textbf{\color{#d91a1a}-7.48\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	1.6186ms	1.4145ms	706.9564 Ops/s	709.1444 Ops/s	$\color{#d91a1a}-0.31\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	1.6529ms	1.3198ms	757.6917 Ops/s	784.6914 Ops/s	$\color{#d91a1a}-3.44\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	5.9638ms	5.6926ms	175.6659 Ops/s	172.6872 Ops/s	$\color{#35bf28}+1.72\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.1001ms	0.4549ms	2.1984 KOps/s	1.9303 KOps/s	$\textbf{\color{#35bf28}+13.89\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.6988ms	0.4422ms	2.2615 KOps/s	2.0905 KOps/s	$\textbf{\color{#35bf28}+8.18\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	5.6821ms	5.5853ms	179.0412 Ops/s	177.2571 Ops/s	$\color{#35bf28}+1.01\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.6679ms	0.3595ms	2.7814 KOps/s	2.7521 KOps/s	$\color{#35bf28}+1.06\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.5587ms	0.3430ms	2.9151 KOps/s	3.2602 KOps/s	$\textbf{\color{#d91a1a}-10.58\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	5.7885ms	5.5335ms	180.7185 Ops/s	177.3197 Ops/s	$\color{#35bf28}+1.92\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	1.0435ms	0.3154ms	3.1701 KOps/s	2.9190 KOps/s	$\textbf{\color{#35bf28}+8.60\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.6421ms	0.3151ms	3.1732 KOps/s	3.2820 KOps/s	$\color{#d91a1a}-3.31\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	8.0404ms	5.7288ms	174.5555 Ops/s	171.8022 Ops/s	$\color{#35bf28}+1.60\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.0850ms	0.4298ms	2.3269 KOps/s	2.2578 KOps/s	$\color{#35bf28}+3.06\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.6300ms	0.4125ms	2.4243 KOps/s	2.3811 KOps/s	$\color{#35bf28}+1.81\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.4700s	14.2111ms	70.3678 Ops/s	197.9331 Ops/s	$\textbf{\color{#d91a1a}-64.45\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	3.9778ms	1.7239ms	580.0775 Ops/s	473.7152 Ops/s	$\textbf{\color{#35bf28}+22.45\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	9.1477ms	1.2556ms	796.4481 Ops/s	830.4695 Ops/s	$\color{#d91a1a}-4.10\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	6.9569ms	4.9592ms	201.6469 Ops/s	196.7023 Ops/s	$\color{#35bf28}+2.51\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	8.8732ms	2.0950ms	477.3331 Ops/s	481.7601 Ops/s	$\color{#d91a1a}-0.92\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	7.3106ms	1.2587ms	794.4734 Ops/s	800.9075 Ops/s	$\color{#d91a1a}-0.80\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.4341s	13.7739ms	72.6009 Ops/s	57.9858 Ops/s	$\textbf{\color{#35bf28}+25.20\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	12.8642ms	2.1581ms	463.3640 Ops/s	466.7218 Ops/s	$\color{#d91a1a}-0.72\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	1.2112ms	1.0190ms	981.3928 Ops/s	969.0007 Ops/s	$\color{#35bf28}+1.28\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True]	36.5006ms	33.3040ms	30.0265 Ops/s	29.2769 Ops/s	$\color{#35bf28}+2.56\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False]	19.0299ms	17.2935ms	57.8252 Ops/s	56.7691 Ops/s	$\color{#35bf28}+1.86\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True]	37.2384ms	33.8856ms	29.5110 Ops/s	28.8121 Ops/s	$\color{#35bf28}+2.43\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False]	18.9264ms	17.4674ms	57.2494 Ops/s	56.2635 Ops/s	$\color{#35bf28}+1.75\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True]	40.4958ms	35.5531ms	28.1269 Ops/s	27.6612 Ops/s	$\color{#35bf28}+1.68\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False]	20.7844ms	19.1785ms	52.1418 Ops/s	53.3278 Ops/s	$\color{#d91a1a}-2.22\%$