Remote webdriver not working

[nareto]

I have a docker container running selenium-chrome (the "standalone-chrome" official container) and I'm trying to get scrapy-selenium to work with it. I have this in settings.py:

DOWNLOADER_MIDDLEWARES = {
      'scrapy_selenium.SeleniumMiddleware': 800
}
SELENIUM_DRIVER_NAME = 'chrome'
SELENIUM_COMMAND_EXECUTOR='http://localhost:4444/wd/hub' 
SELENIUM_DRIVER_ARGUMENTS=['--headless']  

and I tried different combinations, but I keep getting

[scrapy.middleware] WARNING: Disabled SeleniumMiddleware: SELENIUM_DRIVER_NAME and SELENIUM_DRIVER_EXECUTABLE_PATH must be set

I don't think SeleniumRequest is working, I get further down a KeyError for 'screenshot' on response.meta (I did set screenshot=True in SeleniumRequest).

The selenium container is indeed working, it works if I test it by passing it to webdriver.Remote. Any suggestions?

[WaterKnight1998]

@nareto did you solve it?

[nareto]

No. I hadn't noticed the project is not maintained anymore. I am now trying to use scrapy-splash but having other problems. It's a pity because in my limited experience Selenium works better than Splash

Maybe I'll come back to this and not use the remote webdriver, but this makes it harder to dockerize the scraper.

[rohitsathish]

Having the same issue.

[shijialee]

Remote selenium driver doesn't work in version 0.7. For testing, replace middleware.py with the one from latest commit. Also, use SELENIUM_DRIVER_ARGUMENTS=['-headless'] for remote chrome driver.

[hyobbb]

I have the same issue so I am using selenium directly and there's no problem at all that is.. it doesn't make sense of using this package.