Update documentation, update pre-commit and linter/formatter, some minor refactorings (#70)

Author: DanielDauner

.pre-commit-config.yaml

@@ -19,12 +19,12 @@ repos:
         args: ['--no-sort-keys', "--autofix"]
 
 -   repo: https://github.com/astral-sh/ruff-pre-commit
-    rev: v0.6.2
+    rev: v0.14.6
     hooks:
-      - id: ruff
+      - id: ruff-check  # Run the linter.
         args: [--fix]
         exclude: __init__.py$
-      - id: ruff-format
+      - id: ruff-format  # Run the formatter.
         exclude: __init__.py$
 -   repo: https://github.com/kynan/nbstripout
     rev: 0.8.1

docs/README.md

@@ -0,0 +1,10 @@
+You can install relevant dependencies for editing the public documentation via:
+```sh
+pip install -e .[docs]
+```
+
+It is recommended to uses [sphinx-autobuild](https://github.com/sphinx-doc/sphinx-autobuild) (installed above) to edit and view the documentation. You can run:
+
+```sh
+sphinx-autobuild docs docs/_build/html
+```

docs/contributing.md

@@ -0,0 +1,108 @@
+## Contributing
+
+Contributions to 123D are highly encouraged! This guide will help you get started with the development process.
+
+### Getting Started
+
+#### 1. Clone the Repository
+
+```sh
+git clone [email protected]:autonomousvision/py123d.git
+cd py123d
+```
+
+#### 2. Installation
+
+```sh
+conda create -n py123d_dev python=3.12  # Optional
+conda activate py123d_dev
+pip install -e .[dev]
+pre-commit install
+```
+
+The above installation should also include linting, formatting, type-checking in the pre-commit.
+We use [`ruff`](https://docs.astral.sh/ruff/) as linter/formatter, for which you can run:
+```sh
+ruff check --fix .
+ruff format .
+```
+Type checking is not strictly enforced, but ideally added with [`pyright`](https://github.com/microsoft/pyright).
+
+
+#### 3. Managing dependencies
+
+We try to keep dependencies minimal to ensure quick and easy installations.
+However, various datasets require dependencies in order to load or preprocess the dataset.
+In this case, you can add optional dependencies to the `pyproject.toml` install file.
+You can follow examples of nuPlan or nuScenes. These optional dependencies can be install with
+
+```sh
+pip install -e .[dev,nuplan,nuscenes]
+```
+where you can combined the different optional dependencies.
+
+The optional dependencies should only be required for data pre-processing.
+When writing a dataset conversion method, you can check if the necessary dependencies are installed by calling with the `check_dependencies` function.
+
+```python
+from py123d.common.utils.dependencies import check_dependencies
+
+check_dependencies(["optional_package_a", "optional_package_b"], "optional_dataset")
+import optional_package_a
+import optional_package_b
+
+def load_camera_from_outdated_dataset(...) -> ...:
+    optional_package_a.module(...)
+    optional_package_b.module(...)
+    pass
+```
+This will notify the user if `optional_dataset` is not included in the 123D install.
+
+Also ensure that functions/modules that require optional installs are only imported when necessary, e.g:
+
+```python
+def load_camera_from_file(file_path: str, dataset: str) -> ...:
+    ...
+    if dataset == "optional_dataset":
+        from py123d.some_module import load_camera_from_outdated_dataset
+
+        return load_camera_from_outdated_dataset(...)
+    ...
+```
+
+#### 4. Other useful tools
+
+If you are using VSCode, it is recommended to install:
+- [autodocstring](https://marketplace.visualstudio.com/items?itemName=njpwerner.autodocstring) - Creating docstrings (please set `"autoDocstring.docstringFormat": "sphinx-notypes"`).
+- [Code Spell Checker](https://marketplace.visualstudio.com/items?itemName=streetsidesoftware.code-spell-checker) - A basic spell checker.
+
+Or other similar plugins depending on your preference/editor.
+
+### Documentation Requirements
+
+#### Docstrings
+- **Development:** Docstrings are encouraged but not strictly required during active development
+- **Format:** Use [Sphinx-style docstrings](https://sphinx-rtd-tutorial.readthedocs.io/en/latest/docstrings.html)
+
+
+#### Sphinx documentation
+
+All datasets should be included in the `/docs/datasets/` documentation. Please follow the documentation format of other datasets.
+
+You can install relevant dependencies for editing the public documentation via:
+```sh
+pip install -e .[docs]
+```
+
+It is recommended to uses [sphinx-autobuild](https://github.com/sphinx-doc/sphinx-autobuild) (installed above) to edit and view the documentation. You can run:
+```sh
+sphinx-autobuild docs docs/_build/html
+```
+
+### Adding new datasets
+TODO
+
+
+### Questions?
+
+If you have any questions about contributing, please open an issue or reach out to the maintainers.

docs/datasets/av2.rst

@@ -149,27 +149,27 @@ The downloaded dataset should have the following structure:
 Installation
 ~~~~~~~~~~~~
 
-No additional installation steps are required beyond the standard ``py123d`` installation.
+No additional installation steps are required beyond the standard `py123d`` installation.
 
 
 Conversion
 ~~~~~~~~~~
 
-To run the conversion, you either need to set the environment variable ``$AV2_SENSOR_ROOT`` or ``$AV2_SENSOR_ROOT``.
+To run the conversion, you either need to set the environment variable ``$AV2_DATA_ROOT`` or ``$AV2_SENSOR_ROOT``.
 You can also override the file path and run:
 
 .. code-block:: bash
 
   py123d-conversion datasets=["av2_sensor_dataset"] \
-  dataset_paths.av2_sensor_data_root=$AV2_SENSOR_ROOT # optional if env variable is set
-
+  dataset_paths.av2_data_root=$AV2_DATA_ROOT # optional if env variable is set
 
 
 
 Dataset Issues
 ~~~~~~~~~~~~~~
 
-n/a
+- **Ego Vehicle:** The vehicle parameters are partially estimated and may be subject to inaccuracies.
+
 
 
 Citation

docs/datasets/carla.rst

@@ -3,6 +3,7 @@ CARLA
 
 CARLA is an open-source simulator for autonomous driving research.
 As such CARLA data is synthetic and can be generated with varying sensor and environmental conditions.
+The following documentation is largely incomplete and merely describes the provided demo data.
 
 .. dropdown:: Quick Links
   :open:
@@ -46,7 +47,7 @@ Available Modalities
      - Depending on the collected dataset. For further information, see :class:`~py123d.datatypes.detections.BoxDetectionWrapper`.
    * - Traffic Lights
      - X
-     - TODO
+     - n/a
    * - Pinhole Cameras
      - ✓
      - Depending on the collected dataset. For further information, see :class:`~py123d.datatypes.sensors.PinholeCamera`.
@@ -90,12 +91,7 @@ Dataset Specific
 Dataset Issues
 ~~~~~~~~~~~~~~
 
-[Document any known issues, limitations, or considerations when using this dataset]
-
-* Issue 1: Description
-* Issue 2: Description
-* Issue 3: Description
-
+n/a
 
 Citation
 ~~~~~~~~

docs/datasets/index.rst

@@ -3,7 +3,8 @@ Datasets
 
 Brief overview of the datasets section...
 
-This section provides comprehensive documentation for various autonomous driving and computer vision datasets. Each dataset entry includes installation instructions, available data types, known issues, and proper citation formats.
+This section provides comprehensive documentation for various autonomous driving and computer vision datasets.
+Each dataset entry includes installation instructions, available data types, known issues, and references for further reading.
 
 .. toctree::
    :maxdepth: 1