You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Metadata is saved in distributed mode if there is a db worker with no flag --no-incoming. When I switched to single process mode, metadata is not saved. I did not find any setting that enables it. Is it indented? According to documentation metadata should be saved.
Here you can find my project. I am using discovery strategy. To add seeds, run python -m frontera.utils.add_seeds --config config.single --seeds-file seeds.txt. To start crawl, run scrapy crawl spider.
The text was updated successfully, but these errors were encountered:
Metadata is saved in distributed mode if there is a db worker with no flag
--no-incoming
. When I switched to single process mode, metadata is not saved. I did not find any setting that enables it. Is it indented? According to documentation metadata should be saved.Here you can find my project. I am using discovery strategy. To add seeds, run
python -m frontera.utils.add_seeds --config config.single --seeds-file seeds.txt
. To start crawl, runscrapy crawl spider
.The text was updated successfully, but these errors were encountered: