Can't get reasonable results from data that nerfstudio aces?

[xiumingzhang]

I've been struggling to attain reasonable results for my custom nerfstudio-like data, similar to #49, #85, etc., so I took a step back: What if I just take nerfstudio's example "poster" dataset (which nerfstudio aces) and directly run sdfstudio on it? It turned out I was unable to get reasonable results for that dataset either.

Steps to reproduce the behavior:

1. Download nerfstudio's poster dataset.
2. Run, say, neus-facto with the nerfstudio-data data parser.
3. Look at the tensorboard results.

One example full command:

method=neus-facto
ns-train \
    $method \
    --pipeline.model.sdf-field.inside-outside True \
    --pipeline.model.background-model mlp \
    --output-dir sdfstudio_train/nerfstudio_data/poster/ \
    --method-name $method \
    --experiment-name sanity-check-mlp-bg-focus-vertical \
    --vis tensorboard \
    nerfstudio-data \
    --data nerfstudio/data/nerfstudio/poster \
    --center-method focus \
    --orientation-method vertical

I tried numerous combinations of:

• --pipeline.model.background-model mlp
• --pipeline.model.sdf-field.inside-outside True
• Patching this PR: add support for "vertical" orientation and "focus" centering methods #90
• --pipeline.model.sdf-field.use-grid-feature False
• --pipeline.model.far-plane 2.0
• --pipeline.model.sdf-field.num-layers 8
• --pipeline.model.enable-collider False

Example failures:

What am I missing? Thanks in advance for thoughts/suggestions.

[niujinshuchong]

Hi, NeuS-facto is sensitive to initialization/hyper parameters. bakedsdf and bakedsdf-mlp can get something reasonable like this:

The training command is:

ns-train bakedsdf --pipeline.model.sdf-field.inside-outside False --pipeline.model.sdf-field.bias 0.2 --pipeline.model.near-plane 0.05  --vis wandb --experiment-name backedsdf nerfstudio-data --data data/nerfstudio/poster    

You could also refer to #2 (comment) to use monocular priors which often help in indoor scenes.

[xiumingzhang]

Thanks for the prompt help. I got similar results with your command on this poster scene.

One quick follow-up question: Why --pipeline.model.sdf-field.inside-outside False? The CLI help said one should set it to True for indoor scenes.

[niujinshuchong]

I thought this is a 360 inward scene so I use --pipeline.model.sdf-field.inside-outside False where the wall of the room is modelled as background. Using --pipeline.model.sdf-field.inside-outside True might also works.

[fafancier]

@xiumingzhang I got simlilar error. under same data, nerfacto can export correct pointcloud. however nues facto can't.
the pointcloud seems constitute a circle around center.
have you figure out the solution? many thx.

oh, I find that bakedsdf seems give more reason result than neusfacto