Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change the evaluation metrics to miou #62

Open
autumn-wong opened this issue Nov 24, 2024 · 3 comments
Open

Change the evaluation metrics to miou #62

autumn-wong opened this issue Nov 24, 2024 · 3 comments

Comments

@autumn-wong
Copy link

Currently there are very few occupancy networks based on sparse frameworks, thank you for your work, which has inspired me to be able to build on it to be able to continue with new ideas in my research.
I tried to change the evaluation metrics to miou to be able to easily compare the performance with other models, but I noticed that running 'python old_metrics.py' needs to save the predictions in npz format. What needs to be changed in the code to be able to use miou when executing val.py?
Can you help me in this matter?

@YANG-CY-163
Copy link
Contributor

You can rewrite the evaluate method of nuscenes_occ_dataset.py to use Metric_mIoU instead of main_rayiou; Or you can simply add the following code after getting the sem_pred with shape [200x200x16] in the evaluate method, saving predictions when executing val.py.
image

@autumn-wong
Copy link
Author

I will try it. Thanks for your quick reply!

@keke-mon
Copy link

keke-mon commented Dec 3, 2024

This is very excellent work, proposing a method with high real-time performance and innovative evaluation metrics! Regarding this issue, I tried to add the code you mentioned, but it seems that in the nuscenes_occ_dataset.py file, within the evaluate method, the shape of sem_pred is (B,N),which prints out as (1,32000). Do I need to perform any additional operations to get the shape [200x200x16]?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants