405 Error on reCaptcha page / errors after reloading feed

[wesleymg]

Checklist

• I am able to reproduce the bug with the latest version given here: CLICK THIS LINK.
• I made sure that there are no existing issues - open or closed - which I could contribute my information to.
• I have read the FAQ and my problem isn't listed.
• I have taken the time to fill in all the required details. I understand that the bug report will be dismissed otherwise.
• This issue contains only one bug.
• I have read and understood the contribution guidelines.

Affected version

0.27.2

Steps to reproduce the bug

1. Get to a point where a reCaptcha is required
2. Tap Solve

Expected behavior

Video should work normally, or reCaptcha should appear and be solved.

Actual behavior

Google reports a 405 error, no reCaptcha appears, and ergo cannot be solved, videos are unwatchable.

Screenshots/Screen recordings

Logs

No response

Affected Android/Custom ROM version

OxygenOS 14.0

Affected device model

OnePlus 12R

Additional information

I can intermittently watch a video, especially if I disconnect from my wifi, but after one or two, I can no longer use the app successfully.

[BaconCondensate]

Can also confirm the issue, also happens with trending tabs and not just videos. Switching networks (eg. VPN) temporarily solves the issue

My uneducated assumption is that google is doing some A/B testing in specific regions/connection

[LucaWight]

It also carries over to the regular YouTube app which also refuses to work now.

[AsyrafFile]

It also occurs bug #11139 after I restarted the the router.

[vorsn]

This is maybe not an actual recaptcha error. When you try to use YT on a browser in the same network you get 429 errors

We're sorry... ... but your computer or network may be sending automated queries. To protect our users, we can't process your request right now.

You can play sporadic videos. Updating feeds only adds 2-3 videos every x minutes. So this seems to be some kind of burst limiting. I hope you can find a workaround, looks bad.

As an user you should not update your feed anymore to avoid blocking youtube page in general.

[junelac]

It seems that YouTube started to ban IP of Newpipe users. The ban is not complete, I'm still able to watch videos on YouTube with a browser, but there are are errors message and I can't connect to my account. When using a VPN it works for a while, then I'm banned again

[aufuss]

I encountered the same Problem, first time yesterday 29.10. Switching to Browser works (Vanadium). Rebooting the phone helps (for a while).
Pixel 7a with GrapheneOS and no Google apps.

[BaconCondensate]

Seems like every alt youtube platform gets the issue. newpipe, freetube, invidious. Although the error seems different

Likely something changed in the backend

[rjdg14]

It seems that YouTube started to ban IP of Newpipe users. The ban is not complete, I'm still able to watch videos on YouTube with a browser, but there are are errors message and I can't connect to my account. When using a VPN it works for a while, then I'm banned again

It sounds to me a little like a rate limiting related error, especially since other clients are getting a 429 (too many requests) error.

Are the IP blocks others are experiencing temporary and going away after a few minutes to hours of requests from the app stopping? Also, how many requests has it taken others here within a certain period for the issue to occur?

I think YouTube has used a form of rate limiting for a while so what has likely changed?

[BaconCondensate]

It seems that YouTube started to ban IP of Newpipe users. The ban is not complete, I'm still able to watch videos on YouTube with a browser, but there are are errors message and I can't connect to my account. When using a VPN it works for a while, then I'm banned again

It sounds to me a little like a rate limiting related error, especially since other clients are getting a 429 (too many requests) error.

Are the IP blocks others are experiencing temporary and going away after a few minutes to hours of requests from the app stopping? Also, how many requests has it taken others here within a certain period for the issue to occur?

I think YouTube has used a form of rate limiting for a while so what has likely changed?

Considering accessing through the browser still works, I don't think it's rate limiting. Unless they have different limit for embeds, even the trending tab (not video) could get the error too though

[rjdg14]

Could this have anything to do with the problem perhaps (YouTube releasing a new layout)?

https://old.reddit.com/r/youtube/comments/1gev7jz/anyone_getting_429_cant_do_anything_in_android_app/lujkzxn/

I'm not sure if any of the users in that thread use third party clients, but if not then it could be a more general glitch with a new YouTube API that is causing it to trigger its rate limit block.

These are some related Reddit threads from the past day:

https://www.reddit.com/r/youtube/comments/1gf4a3y/youtube_says_im_offline_but_im_not_affecting/

https://www.reddit.com/r/youtube/comments/1gffluo/does_anyone_know_how_to_fix_error_429/

https://www.reddit.com/r/youtube/comments/1gfkqgn/what_the_actual_hell_is_going_on/

[Nachoswithpepper]

I fear that this might be another Ban-wave as previously mentioned- aimed at specific IPs. Seems both Newpipe and GrayJay are suffering, though GrayJay is able to play videos through a different stream (fails to retrieve iOS stream data, fallsback to UMP) but still returns a 429 Error when prompting refresh/retry on the Home tab, just as Newpipe fails on both Trending/Subscriptions.

[Cf0cus]

same exact issue, to add on it I checked youtube itself in the web of mobile it says "invalid response" and also if you use the youtube app it says smth error 429

[AudricV]

Please only comment if you have something useful to provide, thank you!

This is likely triggered by making too many requests to YouTube's internal API within a certain period of time. Refreshing YouTube subscriptions without fast mode is what probably triggers the limit, so it is probably a good idea to disable auto-refresh of subscriptions and avoid refreshing your feed if you have an important number of subscriptions from YouTube for now.

If you stop connecting to YouTube in any way for an unknown amount of time, you should be able to get your access back.

The response the app gets is described in this comment: #11661 (comment) (the HTML response also can be raised if you set an non Google or YouTube Origin or Referrer HTTP header on these requests (and this time you get an 403 HTTP error and not a 429 one)).

The app interprets any HTTP 429 error as a captcha request, which is very bad. This must be changed, as it also affects other services (PeerTube for instance if you reach the limit of requests within a certain amount time for your IP the instance has set).

Help is wanted to determine what amount of requests within a certain period of time triggers the error and how much time you need to wait to get back valid responses.

fails to retrieve iOS stream data, fallsback to UMP

UMP is a streaming media protocol used by YouTube and not a way to get URLs of streams.

[rjdg14]

Is it likely that subscription refreshes were also what was causing the "sign in to confirm you are not a bot" error that some users were reporting earlier this year? In my case I have never used NewPipe or any similar software for keeping track of subscriptions and never had that problem. I'm currently away from home and don't want to load any videos on my data limited mobile connection, but can confirm that the trending page and search pages within NewPipe load normally.

Is the issue known to be affecting those who do not use NewPipe for channel subscription purposes?

[terranora]

1. Activate fast mode (feed loading)
2. Turn off your Wi-Fi (or mobile network) for 15+ minutes to change your IP

[rjdg14]

There is a setting called Feed Update Threshold. It's possible this issue is occuring for those who have it set to "always update".

[ls-alh]

I'm experiencing this issue too, as of about 24 hours before this comment. My home and work networks are both affected, but mobile data is not. Maybe fingerprinting is involved? Maybe because I spend much less time on mobile networks?

Some of my relevant settings:
fetch from dedicated feed (fast mode?) - off
feed update interval - 1 hour
auto refresh - 351 channels, frequency 4 hours

It's also important to note that I'm a heavy user. I'm seeing this issue on every device connected to these networks, even after signing in. Sometimes I can load a video in the browser, sometimes not.

[CBSkarmory]

Help is wanted to determine what amount of requests within a certain period of time triggers the error and how much time you need to wait to get back valid responses.

Refreshing ~500 subscriptions (taking ~2min) (fetch from dedicated feed OFF) appears sufficient to trigger the error, though not immediately (after a few min).

Waiting 24hrs initially was sufficient to get back valid responses.

[rjdg14]

It may be that for those with, say, 500 subscriptions, YouTube is interpreting the auto refresh as though you are loading 500 channel pages in a very short period of time, which I think would trigger its probably temporary rate limit block. It's common for Google to display an "unusual traffic" captcha page if you make too many searches in a short period, and I think YouTube is very similar in this regard.

Is it possible that YouTube has simply tightened the threshold of their rate limit as a way of making scraping more difficult, and this is interfering with those who have a large number of auto-updating subscriptions in NewPipe?

Does the issue occur in the kind of scenarios where a ReCaptcha would have previously been shown? I've never had NewPipe ask for a captcha when browsing YouTube, and unlike most here do not have any subscriptions in it.

[burritoOverflow]

I've encountered this on a few occasions this week now.
It seems YouTube then blocks your IP for some time (this will apply to all clients (even "native" clients) on that network.

I've not been able to restore use of the service in any fashion outside of calling my ISP and getting a new DHCP lease.

Hard to replicate the cause given the consequences of the failure---but it seems the number of requests sent by refreshing subscriptions could be a cause.

I'm able to occasionally view videos on a web client. Many requests fail with 403 or 429.

I hadn't considered the 429 "appropriate" until I read this thread. No idea how long the "rate-limiting" lasts. I've not had it restored after waiting any duration, only via a new IP address.

EDIT: Just saw @CBSkarmory's response. I'll try waiting 24h for now.

[rjdg14]

I've tested both NewPipe and FreeTube at home today and haven't had any problems, though I don't use either for subscriptions, and I think this is probably what's causing some users to get this error. My advice would be to delete your subscriptions from the app (or at minimum disable auto refresh) and simply make a note of your favourite channels so you can remember their names and search for them manually if you want to check if they've added a new video.

In the case of FreeTube, it looks like many users are currently able to use it as normal ('d imagine the same is true of NewPipe), so I suspect the 429 error is probably affecting the minority of users of both apps who have a lot of subscriptions and who have auto refresh turned on.

[Cooked14]

I've not been able to restore use of the service in any fashion outside of calling my ISP and getting a new DHCP lease.

What's DHCP lease? I live in a sh*thole country so I'm not sure they'll even understand me if I say that. Is it the same as asking for an IP change?

[burritoOverflow]

I've not been able to restore use of the service in any fashion outside of calling my ISP and getting a new DHCP lease.

What's DHCP lease? I live in a sh*thole country so I'm not sure they'll even understand me if I say that. Is it the same as asking for an IP change?

Yes, exactly.

My network hardware has functionality to relinquish the lease, but doesn't grant a new IP without intervention from my ISP.

@rjdg14 cheers, I'll be more cautious.

Ironically enough, YT seems to 4XX sign-in attempts from "legitimate" first party clients, so I've not found a work around for this outside of new IP addr.

[sidgdev]

Using ByeDPI android app in VPN mode which circumvents DPI by creating a local VPN but doesn't change IP resolved this error on New Pipe for me and even resolved other errors in GrayJay & FreeTube as well. Turning off the app quickly brought back the same errors & again turning on errors are gone.

Believe YouTube is able differentiate requests from NewPipe & other invidious clients from browser/android client based on packet inspection but not outright rejecting requests based on IP ban.

This is a useful workaround which is working for me right now, kindly let me know if its working for others.

Link to app-

https://github.com/dovecoteescapee/ByeDPIAndroid/releases

[vehosah]

For two videos it went nice, but here we are again 429 that's all we know

[sidgdev]

Not experienced any 429 errors yet, hope you are using app in VPN instead of proxy mode.

[BaconCondensate]

Tested that method, doesn't seem to work. I don't think DPI protection will even do anything when Google isn't doing MITM because they are the endpoint

Newpipe will probably need to fake being a browser (?)

[ShareASmile]

Don't quote the whole text ,
It presents a wall of text to be scrolled down for future readers of this ticket.

I have edited comments with long text quotes.

[ThatAsshe]

Help is wanted to determine what amount of requests within a certain period of time triggers the error and how much time you need to wait to get back valid responses.

Refreshing ~500 subscriptions (taking ~2min) (fetch from dedicated feed OFF) appears sufficient to trigger the error, though not immediately (after a few min).

Waiting 24hrs initially was sufficient to get back valid responses.

Thank you for this contribution! I was looking for an answer for a few hours on why regular Youtube was giving me the error 429 and 405 on the Tubular app, and I happen to have more than 600 subscriptions oopsie 😅

[realquink]

Adding to the discussion. I use NewPipe daily about 130 subs on a phone and I use a VPN via the router. In total 3 devices use NewPipe throughout a given day on the same IP. Today I received this error for the first time.

[rabidrivas]

Freetube seems to work, It uses RSS to avoid rate limiting. Could it be fixed impementing something like that?

[ThatAsshe]

Adding to the discussion. I use NewPipe daily about 130 subs on a phone and I use a VPN via the router. In total 3 devices use NewPipe throughout a given day on the same IP. Today I received this error for the first time.

Wait exactly 24 hours and the temp. ban will be lifted, after that what I've done is to create channel groups in the 'Subscriptions' section to reduce the amount of subbed channels that'd update at once, delete the 'What's New' tab from the Main page (settings > content > content of main page) to avoid updating the list by accident or just delete your subscriptions at all until a new fix comes out!

[rabidrivas]

Adding to the discussion. I use NewPipe daily about 130 subs on a phone and I use a VPN via the router. In total 3 devices use NewPipe throughout a given day on the same IP. Today I received this error for the first time.

Wait exactly 24 hours and the temp. ban will be lifted, after that what I've done is to create channel groups in the 'Subscriptions' section to reduce the amount of subbed channels that'd update at once, delete the 'What's New' tab from the Main page (settings > content > content of main page) to avoid updating the list by accident or just delete your subscriptions at all until a new fix comes out!

Yeah, or if you are lazy, update subs with a vpn

[tauch]

I can confirm:
Refreshing the subscribed feeds (~500) instantly triggers an IP ban (that persists for approx. a day).
While the ban is active, NewPipe will prompt to solve a captcha, but that function does fail when attempted in NewPipe.
Note: The ban also affects playback via youtube-nocookie, where I get redirected to youtube while a ban is active.

Solution-Ideas:
I'd really love to see a triggerable background updater for the feeds, where the requests/min can be adjusted or even better, where NewPipe remembers which channels did not have new content for a long time, and then check those much more seldomly (reducing the necessary requests to Youtube).
A first work-around for the ban-problem could even be a simple setting to introduce a fixed minimum-delay between feeds during update (this way the update will take ages, but that could avoid getting the IP banned).

[realquink]

Wait exactly 24 hours and the temp. ban will be lifted, after that what I've done is to create channel groups in the 'Subscriptions' section to reduce the amount of subbed channels that'd update at once, delete the 'What's New' tab from the Main page (settings > content > content of main page) to avoid updating the list by accident or just delete your subscriptions at all until a new fix comes out!

Confirming this is right. Ban is lifted and I can load and play videos.

[GameOverFlowChart]

The suggested method with dpi protection worked for a while (don't ask me why that works) but seems like thats also blocked now.
YouTube works in the browser (except for comments which might be a hint), but newpipe and the official YouTube app have troubles.

[Stypox]

The rate limiting when reloading the feed is hopefully fixed by #11743, let us know if the next release (0.27.3) solves the issue for you

[Lorthiz]

I think that this should not be closed. Implemented 'fix' also slows down RSS feed updates that did not have any rate limiting problem afaik. The only way I was informed about this issue was because this 'fix' changed my feed refresh time from few seconds to few minutes. And right now having bigger amount of subscriptions makes this APP almost unusable.
For comparison FreeTube still woks great with RSS feeds.

[ShareASmile]

Implemented 'fix' also slows down RSS feed updates

Please open a new issue.