Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change the data set to train #62

Open
zbx-hue opened this issue Oct 14, 2020 · 3 comments
Open

Change the data set to train #62

zbx-hue opened this issue Oct 14, 2020 · 3 comments

Comments

@zbx-hue
Copy link

zbx-hue commented Oct 14, 2020

I deleted some training data of market-1501. Trained a new teacher network. Changed ID Classes. It's still going to report an error.
RuntimeError: CUDA error: device-side assert triggered
What else do I need to change? Thank you

@layumi
Copy link
Contributor

layumi commented Oct 14, 2020

Hi @zbx-hue
Could you show me the whole log?
Do you use fp16? If so, you may refer to #61.
Do you use multiple GPU? If so, could you try to run the code on one GPU first?

Besides, have you successfully tested and trained the model on Market-1501?

@zbx-hue
Copy link
Author

zbx-hue commented Oct 14, 2020

Hi @layumi
I try to train my data set. But something went wrong. So I did the following work to see if changing the data set would work.
I deleted ten ID folders of train, train_all and val. Trained a new teacher network. The same error occurred with my own data set.

I'm not use fp16.
Both multi-Gpus and single-Gpus have been tried. Make the same mistake.
The complete Market-1501 can be successfully trained and tested.

@zbx-hue
Copy link
Author

zbx-hue commented Oct 14, 2020

An error screenshot
An error screenshot(1)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants