Skip to content

Commit a37adb4

Browse files
NilsWinterNilsWinter
committed
Update root README
1 parent 2525e34 commit a37adb4

File tree

1 file changed

+145
-32
lines changed

1 file changed

+145
-32
lines changed

‎README.md‎

Lines changed: 145 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -1,42 +1,155 @@
1-
# PHOTONAI Analysis
1+
# Getting started
22

3-
- uses Apptainer (https://apptainer.org/docs/user/main/build_a_container.html)
4-
- main.py describes the run script that will be executed when running the container
5-
- main.py uses typer to create a CLI
3+
This page shows how to set up a simple PHOTONAI project using
4+
`PhotonaiProject`, run analyses, perform permutation tests, and
5+
statistically compare different analyses.
66

7-
## Example usage of PHOTONAI container
7+
## Installation
8+
9+
Install the package (and PHOTONAI) into your environment:
10+
11+
```bash
12+
pip install photonai photonai-projects
813
```
9-
apptainer run docker://photonai-analyzer-v2.3.0 --help
10-
apptainer run docker://photonai-analyzer-v2.3.0 --folder ./MYPROJECT/ --task regression --pipeline basic
14+
15+
## Basic concepts
16+
A **PhotonaiProject** manages multiple PHOTONAI analyses in a single
17+
project folder. Each analysis has its own subfolder containing:
18+
19+
- a hyperpipe constructor script (hyperpipe_constructor.py)
20+
21+
- a metadata file (hyperpipe_meta.json)
22+
23+
- a data/ folder with X.npy and y.npy
24+
25+
- (optionally) a permutations/ folder for permutation tests
26+
27+
The typical workflow is:
28+
29+
1. Create a project with PhotonaiProject.
30+
2. Add analyses (data + hyperpipe constructor).
31+
3. Run analyses to train and evaluate the models.
32+
4. Run permutation tests to obtain null distributions.
33+
5. Compare analyses statistically.
34+
35+
## Minimal example
36+
Below is a complete example using the breast cancer dataset from
37+
scikit-learn. We create three analyses using different feature sets,
38+
run them, run permutation tests, and then compare them statistically.
39+
40+
41+
```python
42+
from photonai_projects.project import PhotonaiProject
43+
from sklearn.datasets import load_breast_cancer
44+
45+
# Load example data
46+
X, y = load_breast_cancer(return_X_y=True)
47+
48+
# Split features into different sets
49+
X_1 = X[:, :3]
50+
X_2 = X[:, 3:6]
51+
52+
# Create a project
53+
project = PhotonaiProject(project_folder="example_project")
54+
55+
# ---------------------------------------------------------------------
56+
# 1) Register analyses
57+
# ---------------------------------------------------------------------
58+
for name, current_X in [
59+
("all_features", X),
60+
("first_feature_set", X_1),
61+
("second_feature_set", X_2),
62+
]:
63+
project.add(
64+
name=name,
65+
X=current_X,
66+
y=y,
67+
hyperpipe_script="path/to/hyperpipe_constructor.py",
68+
name_hyperpipe_constructor="create_hyperpipe",
69+
)
70+
71+
project.list_analyses()
72+
73+
# ---------------------------------------------------------------------
74+
# 2) Run analyses
75+
# ---------------------------------------------------------------------
76+
for name in ["all_features", "first_feature_set", "second_feature_set"]:
77+
project.run(name=name)
78+
79+
# ---------------------------------------------------------------------
80+
# 3) Run permutation tests (local example)
81+
# ---------------------------------------------------------------------
82+
# Use a small number of permutations for testing; increase for real studies.
83+
for name in ["all_features", "first_feature_set", "second_feature_set"]:
84+
project.run_permutation_test(name=name, n_perms=10, overwrite=True)
85+
86+
# ---------------------------------------------------------------------
87+
# 4) Statistical comparison of analyses
88+
# ---------------------------------------------------------------------
89+
# For the Nadeau–Bengio test you must provide n_train and n_test as used
90+
# during cross-validation. Here we give a simple example.
91+
n_samples = X.shape[0]
92+
n_train = int(0.8 * n_samples)
93+
n_test = n_samples - n_train
94+
95+
# Compare two analyses (Nadeau–Bengio corrected t-test)
96+
project.compare_analyses(
97+
first_analysis="first_feature_set",
98+
second_analysis="second_feature_set",
99+
method="nadeau-bengio",
100+
n_train=n_train,
101+
n_test=n_test,
102+
)
103+
104+
# Compare two analyses (permutation-based)
105+
project.compare_analyses(
106+
first_analysis="all_features",
107+
second_analysis="second_feature_set",
108+
method="permutation",
109+
n_perms=10,
110+
)
111+
112+
# Compare all pairs at once (optional)
113+
multi_results = project.compare_multiple_analyses(
114+
analyses=["all_features", "first_feature_set", "second_feature_set"],
115+
method="permutation",
116+
n_perms=10,
117+
)
118+
print(multi_results.head())
11119
```
12120

13-
## Example Usage of main.py
121+
## Running permutation tests on a SLURM cluster
122+
For large numbers of permutations, you can distribute them across a
123+
SLURM array:
124+
125+
```python
126+
project.prepare_slurm_permutation_test(
127+
name="second_feature_set",
128+
n_perms=1000,
129+
conda_env="my_photonai_env",
130+
memory_per_cpu=2,
131+
n_jobs=20,
132+
run_time="0-02:00:00",
133+
random_state=1,
134+
)
14135
```
15-
(photonaiAnalysis) nils@Nilss-MBP photonaiAnalysis % python main.py --help
16-
Usage: main.py [OPTIONS]
17-
18-
Run PHOTONAI Analysis
19-
20-
Options:
21-
--folder DIRECTORY Path to project folder containing features
22-
and target. [required]
23-
--task [regression|classification]
24-
Specify type of the prediction task
25-
[required]
26-
--pipeline [basic|advanced|ensemble]
27-
Specify PHOTONAI pipeline. [required]
28-
--outer-cv INTEGER Number of outer folds. [default: 10]
29-
--inner-cv INTEGER Number of inner folds. [default: 10]
30-
--scaling / --no-scaling Apply z-scaling of features? [default:
31-
scaling]
32-
--imputation / --no-imputation Apply feature imputation? [default:
33-
imputation]
34-
--help Show this message and exit.
35136

137+
This creates a slurm_job.cmd script in the analysis folder which you
138+
can submit with:
139+
140+
```bash
141+
cd example_project/second_feature_set
142+
sbatch slurm_job.cmd
36143
```
37-
![typer example](static/typer_rich_example.png)
38144

39-
# PHOTONAI Project Report
40-
Can be used to summarize results from multiple PHOTONAI analyses. Uses *datapane* to create a HTML website
41-
that visualizes the results.
145+
Each array job will call the Typer CLI entry point run_perm_job and
146+
execute a subset of permutation runs.
147+
148+
## Next steps
149+
See the Usage page for more details on:
150+
151+
- how to design your hyperpipe constructor,
152+
- how metrics and scorers are handled,
153+
- how to interpret the comparison reports.
42154

155+
See the API Reference for the full documentation of PhotonaiProject.

0 commit comments

Comments
 (0)