Add support for Android Auto

[haggaie]

What is it?

• Bugfix (user facing)
• Feature (user facing)
• Codebase improvement (dev facing)
• Meta improvement to the project (dev facing)

Description of the changes in your PR

• Expose media controls and playback status on Android Auto, as well as a basic browsing interface, showing playlists, history, and supporting basic searches.

Issues, bugs, and missing features

• Items (playlists and streams) are not updated in Android Auto when they are in the app.
• When closing the player from its mini view, playback continues. It should be stopped instead.
• When starting playback on Android Auto then opening the app, the mini player doesn't appear and isn't synchronized with the current state when "opened" (opening the details page of a content with autoplay disabled, pressing the play button plays the content on the details page)
• Remote playlists are limited to their first page.
• when starting the app then connecting to an Desktop Head Unit. The DHU was blocked on the loading step happening before showing playback controls normally, the app was playing the audio in the meantime then the app crashed.
• Non square thumbnails seems to forced to the square format in the miniplayer commands if you enable the setting of the Desktop Head Unit.
• Integration with voice search - voice search from within the app works, but search from the google assistant (e.g., "play on NewPipe") only supports applications from the Google Play Store.

Future enhancements

• Show progress bars for played items and for the one currently playing, if applicable (see https://developer.android.com/training/cars/media?hl=en#browse-progress-bar for the latter).
• Add more content to Android Auto.

Screenshots/Screen Record

Fixes the following issue(s)

• Fixes [Feature Request] Compatibility with Android Auto #1758

APK testing

The APK can be found by going to the "Checks" tab below the title. On the left pane, click on "CI", scroll down to "artifacts" and click "app" to download the zip file which contains the debug APK of this PR.

Due diligence

• I read the contribution guidelines.

[sonarqubecloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

0.0% Coverage
0.0% Duplication

[Stypox]

Interesting! Is it possible that using MediaBrowserServiceCompat along with setSessionToken might solve some issues related to the notification? Also, from the docs of MediaBrowserServiceCompat, it enables "applications to browse media content provided by an application". Do you intend to implement anything related to browsing content, or do you want to pretend there is no browasable content for simplicity and just implement media controls? Also, since the service is started whenever external applications need to browse media, isn't it possible that the player starts at random?

[haggaie]

Interesting! Is it possible that using MediaBrowserServiceCompat along with setSessionToken might solve some issues related to the notification?

I'm not familiar with these issues, but I can try testing them if they are reproduced with this patch. Can you point me in their direction?

Also, from the docs of MediaBrowserServiceCompat, it enables "applications to browse media content provided by an application". Do you intend to implement anything related to browsing content, or do you want to pretend there is no browasable content for simplicity and just implement media controls?

I wanted to start with media controls, because it seems useful in its own. I can set a play queue in advance and control it from the car. Later I would also like to add see browsing. I think that for Auto it should be limited - maybe things like saved playlists and search history? There's also an API for search queries I am hoping would allow integration with voice search.

Also, since the service is started whenever external applications need to browse media, isn't it possible that the player starts at random?

Yes, I think the documentation had something to say about it - that the service shouldn't start playing when it starts. It should only start playing when you get a callback from the MediaBrowserServiceCompat. But the service can tell how if it gets started by another UI or by a media browser, can't it?

I had a related question: at first I thought it would be better to create a separate service. I then saw that it is easier to use the existing player service, but do you think it is the right approach?

Another small problem I noticed with the pull request in its current state, is that the Android Auto's rendering of the media session notification shows the progress circle around the pause button in gray, rather than showing the current position. This is weird, because the notification on my phone does show a seekable progress bar.

[Stypox]

Can you point me in their direction?
the media session notification shows the progress circle around the pause button in gray, rather than showing the current position

You actually answered my question: the issue with the notification I was referring to is the one related to no seek bar showing up. And apparently no, setSessionToken does not fix the issue magically.

I wanted to start with media controls, because it seems useful in its own.

Yeah that's fine, you can take care of the rest later if you want. Integration with voice search would also be really nice.

But the service can tell how if it gets started by another UI or by a media browser, can't it?

Yes it can, but atm I think that part is broken, because there are user reports of the player popping up from nowhere or crashing at random when connecting or disconnecting bluetooth.

I had a related question: at first I thought it would be better to create a separate service. I then saw that it is easier to use the existing player service, but do you think it is the right approach?

Here are a few considerations:

• player code is already pretty messy, but maybe the PlayerService is not too messy
• when you do setSessionToken, you need an instance of the MediaBrowserServiceCompat, and it might be difficult to get hold of it if it's an external service (I guess it would require having two services running at the same time?)

So I think all in all it's best to keep it in PlayerService, but try to put as few code there as possible in order not to clutter the class (i.e. prefer creating utility classes).

[haggaie]

I'm trying to implement some simple browsing, but I got into lifecycle issues. The MediaBrowserServiceCompat is supposed to have a call to setSessionToken in onCreate (and expect it to be called only once), and I can solve this by creating the MediaSession in MediaSessionPlayerUi's constructor. However, if I close the notification, it causes the player service to clean up, but if I understand correctly, PlayerService remains alive because of the Android Auto media browser. Then if I try to play anything again, the service crashes because it has already destroyed its Player.

How can I handle this? Perhaps closing the notification shouldn't close the entire player, and instead, stop and close the notification UI?

[Stypox]

Perhaps closing the notification shouldn't close the entire player, and instead, stop and close the notification UI?

Closing the notification should definitely release the player. Afterwards any incoming intent to start the player should create a new player, while other intents should be treated separately (and NOT start the player). But it's a big mess. If you can't find out a way to do this, maybe go with the option of creating a separate service.

[haggaie]

Closing the notification should definitely release the player. Afterwards any incoming intent to start the player should create a new player, while other intents should be treated separately (and NOT start the player).

Makes sense, I tried doing something along these lines, though I don't differentiate between the intents yet. The code also still creates the player when the service is created, but I can change it as you wrote.

I also tried adding some browsing capabilities, but for some reason it only worked on the desktop HUD and not with my car. I'll have to check that out.

[goyalyashpal]

Android Auto's rendering of the media session notification shows the progress circle around the pause button in gray, rather than showing the current position.
- #9592 (comment)

• if i recall correct, then i don't think that android auto's rendering shows the seek bar in any player
  (and for good reasons - u don't want to even give a chance to driver to fiddle with seekbars)
• Have you tried using other players on android auto to verify? like youtube or via bluetooth?

---

OT: google also bought the "waze navigation" map app you are using, but they just don't show it anywhere as then, users will start abandoning even waze.

https://edition.cnn.com/2022/12/08/tech/google-waze-maps-merge/index.html
In February 2021, Waze’s former top executive Noam Bardin said it struggled to grow within Google and that Waze could have “probably grown faster and much more efficiently had we stayed independent.”

google and others are fiercely anti-competitive and purchase any company doing better than them. Remeber, Triple EEEs is their motto.

[haggaie]

• if i recall correct, then i don't think that android auto's rendering shows the seek bar in any player
  (and for good reasons - u don't want to even give a chance to driver to fiddle with seekbars)
• Have you tried using other players on android auto to verify? like youtube or via bluetooth?

Yes, that's what I've seen too. In Spotify for example the progress is shown as a circle that can't be manipulated, rather than a seek bar. Still, the circle shows playback progress with two colors. I'll try capturing a screenshot sometime.

OT: google also bought the "waze navigation" map app

I know, that's too bad because I liked Waze. Up until now, Google haven't merged it, even if they haven't grown it much. I think it was pretty good at predicting traffic, which is something that requires a popular service, so I might end up with Google Maps if they merge them.

[goyalyashpal]

the circle shows playback progress with two colors

oh yeah, that thin two coloring of the circle's border stroke, right? i can imagine that.

---

OT:

I know, that's too bad because I liked Waze.

same, i found this too while searching for alternates to google maps some years ago.
Anyways, I heard several other companies have come together to form a geolocation alliance or smth like that... oh yeah "Overture Maps Foundation" - article by TechCrunch, referred to by Nick (from TLE) in news video for '22 Dec W3

[haggaie]

Hi, I've updated the pull request and tried to keep the MediaSession and its connector in the new helper object so that they outlive the player.

I also tried making the media Ids more URI-like, inspired by VLC.

By the way, I also tried implementing the APIs for voice search (I think it should just be onPlayFromSearch/onPrepareFromSearch), but it didn't work. As far as I understand, Google Assistant only works with apps from the Google Play Store. I can't even invoke "open NewPipe", and this shouldn't require any code.

[aedwards00]

Apologies for my unfamiliarity with your development process, but is this likely to be included in the next release?

It's a feature I'm very much interested in and looks to the untrained eye like it's ready to be merged?

[AudricV]

Thank you very much for this feature!

Please apply the following changes and rebase your branch on top of the dev one, so we can test the changes on the latest release. I am trusting you for your usage and your implementation of the feature.

However, there is an issue with this PR: the single ExoPlayer library is being discontinued, as the library has now its place in the Media3 one.

This change removes the media session extension, as media session management is directly done by Media3, so this code would have to be rewritten soon if and when we migrate to Media3 the app.

Even if this migration may happen soon, I don't think we will migrate to Media3 in the next release and I think we can merge this pull request, especially because of the feature it gives to the app.

Finally, I see several changes in this PR related to the player service, especially on command interception. Wouldn't your changes fix the player crash issues we currently have when using a media command on a Bluetooth device and maybe the player ANRs?

[haggaie]

Thanks for your feedback. I'm trying to rebase, but I came across a build issue with the updated dev branch. I'm getting the following error:

> Could not resolve all files for configuration ':app:debugRuntimeClasspath'.
   > Could not find com.github.TeamNewPipe.NewPipeExtractor:extractor:v0.22.6.
     Searched in the following locations:
       - https://dl.google.com/dl/android/maven2/com/github/TeamNewPipe/NewPipeExtractor/extractor/v0.22.6/extractor-v0.22.6.pom
       - https://repo.maven.apache.org/maven2/com/github/TeamNewPipe/NewPipeExtractor/extractor/v0.22.6/extractor-v0.22.6.pom
       - https://jitpack.io/com/github/TeamNewPipe/NewPipeExtractor/extractor/v0.22.6/extractor-v0.22.6.pom
       - https://repo.clojars.org/com/github/TeamNewPipe/NewPipeExtractor/extractor/v0.22.6/extractor-v0.22.6.pom
     Required by:
         project :app > com.github.TeamNewPipe:NewPipeExtractor:v0.22.6
   > Could not find com.github.TeamNewPipe.NewPipeExtractor:NewPipeExtractor:v0.22.6.
     Searched in the following locations:
       - https://dl.google.com/dl/android/maven2/com/github/TeamNewPipe/NewPipeExtractor/NewPipeExtractor/v0.22.6/NewPipeExtractor-v0.22.6.pom
       - https://repo.maven.apache.org/maven2/com/github/TeamNewPipe/NewPipeExtractor/NewPipeExtractor/v0.22.6/NewPipeExtractor-v0.22.6.pom
       - https://jitpack.io/com/github/TeamNewPipe/NewPipeExtractor/NewPipeExtractor/v0.22.6/NewPipeExtractor-v0.22.6.pom
       - https://repo.clojars.org/com/github/TeamNewPipe/NewPipeExtractor/NewPipeExtractor/v0.22.6/NewPipeExtractor-v0.22.6.pom
     Required by:
         project :app > com.github.TeamNewPipe:NewPipeExtractor:v0.22.6
   > Could not find com.github.TeamNewPipe.NewPipeExtractor:timeago-parser:v0.22.6.
     Searched in the following locations:
       - https://dl.google.com/dl/android/maven2/com/github/TeamNewPipe/NewPipeExtractor/timeago-parser/v0.22.6/timeago-parser-v0.22.6.pom
       - https://repo.maven.apache.org/maven2/com/github/TeamNewPipe/NewPipeExtractor/timeago-parser/v0.22.6/timeago-parser-v0.22.6.pom
       - https://jitpack.io/com/github/TeamNewPipe/NewPipeExtractor/timeago-parser/v0.22.6/timeago-parser-v0.22.6.pom
       - https://repo.clojars.org/com/github/TeamNewPipe/NewPipeExtractor/timeago-parser/v0.22.6/timeago-parser-v0.22.6.pom
     Required by:
         project :app > com.github.TeamNewPipe:NewPipeExtractor:v0.22.6

Do you know what causes this?

Regarding moving to media3, I understand it can make most of this code obsolete. Once you are done with the move to media3, I could try to test its support for android auto and see when changes are needed.

[AudricV]

Do you know what causes this?

The recurrent Jitpack artifact Not found issue we have with NewPipe Extractor, update temporarily the extractor on your side to test the changes. We will try to switch soon from Jitpack to something else for NewPipe Extractor and our nanojson fork.

[haggaie]

Finally, I see several changes in this PR related to the player service, especially on command interception. Wouldn't your changes fix the player crash issues we currently have when using a media command on a Bluetooth device and maybe the player ANRs?

Is there an issue about this I can read? Maybe I could try and reproduce this problem and see if my patches help.

[AudricV]

#9095, #9390 for both issues and probably more which may be not listed as duplicates of them.

[haggaie]

#9095, #9390 for both issues and probably more which may be not listed as duplicates of them.

I tried reproducing #9095, and I believe it still happens with this PR. I tried using my car's console without Android Auto, only functioning as a Bluetooth headset. If I play a video on NewPipe, close it, and press "Play" on my car's console, the application crashes.

I got the following in the crash log:

java.lang.RuntimeException: Unable to start service org.schabi.newpipe.player.PlayerService@27e84bf with Intent { act=android.intent.action.MEDIA_BUTTON flg=0x10000010 cmp=org.schabi.newpipe.debug.androidauto/org.schabi.newpipe.player.PlayerService (has extras) }: java.lang.NullPointerException: Attempt to invoke virtual method 'org.schabi.newpipe.player.playqueue.PlayQueue org.schabi.newpipe.player.Player.getPlayQueue()' on a null object reference
    at android.app.ActivityThread.handleServiceArgs(ActivityThread.java:4670)
    at android.app.ActivityThread.-$$Nest$mhandleServiceArgs(Unknown Source:0)
    at android.app.ActivityThread$H.handleMessage(ActivityThread.java:2176)
    at android.os.Handler.dispatchMessage(Handler.java:106)
    at android.os.Looper.loopOnce(Looper.java:201)
    at android.os.Looper.loop(Looper.java:288)
    at android.app.ActivityThread.main(ActivityThread.java:7884)
    at java.lang.reflect.Method.invoke(Native Method)
    at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:548)
    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:936)
  Caused by: java.lang.NullPointerException: Attempt to invoke virtual method 'org.schabi.newpipe.player.playqueue.PlayQueue org.schabi.newpipe.player.Player.getPlayQueue()' on a null object reference
    at org.schabi.newpipe.player.PlayerService.onStartCommand(PlayerService.java:90)
    at android.app.ActivityThread.handleServiceArgs(ActivityThread.java:4652)
    ... 9 more

So I think closing the player must have left the service running but with the player object null. I haven't tested the dev branch for the same error, so this crash may be due to my changes and not the original one reported in #9095.

[asentes2]

Hello,
i use apk in CI, artifacts
i have Honor 8X with android 10 and pioneer autoradio with carplay and android auto
it doesnt work for me when i set android auto to developper mode i see newpipe-androidauto in android auto launcher
i see the newpipe icon and newpipe widget with buttons grayed but i cant change song pause or anything (and i dont see the song name)
its not working in widget and in "big screen" when i click on newpipe icon, its black

Thx for your help

[notbasa]

Ialso tried the build from CI. does not work with Android 13 and Peugeot e2008. The Newpipe icon is visible in the launcher, but it does not even display the UI framework in the screenshot by asentes2 above. Please let me know if there's logs I could pull to help out. I'd really like to get this working. AFAIK this would be the first Android Auto capable player with You Tube music out there.

[haggaie]

@asentes2, @notbasa, I can also see the problem with the current version of the PR on the desktop head unit. I'll try to look into it.

[ready2code31]

@asentes2, @notbasa, I can also see the problem with the current version of the PR on the desktop head unit. I'll try to look into it.

Thx for your answer, hope you find solutions.

[asentes2]

Hi, Does this work to change provider with adb on non root device ?

[ready2code31]

i just tested it to make newpipe-androidauto as google play app with adb, app say its installed from google play
but controls not working better in android auto

[notbasa]

Hi, Does this work to change provider with adb on non root device ? Le sam. 14 déc. 2024 à 11:53, notbasa @.***> a écrit :

Sorry, forgot to mention that it does require root making it even more niche case. Rooting a daily driver phone leads to issues with banking apps etc. I have old phone separately just for AA use. Not really a valid use case for average users I would assume, although Newpipe users are not exactly mainstream users.

[notbasa]

i just tested it to make newpipe-androidauto as google play app with adb, app say its installed from google play but controls not working better in android auto

"> + disposePrepareOrPlayCommands()

•    playbackError(R.string.content_not_supported, PlaybackStateCompat.ERROR_CODE_NOT_SUPPORTED)
  

As far as I understand, Google Assistant uses these APIs when it acts as a controller of a media application (e.g., following a command like "play music on NewPipe". Unfortunately, I believe only Play Store apps can be controlled by Google Assistant, so there wasn't much point in implementing them."

To me that looks like the necessary API end points have not been implemented in NewPipe, so just changing the app source does nothing. The code would also need to have support for it, so that Google Assistant could be used to control it via voice.

[haggaie]

@asentes2, @notbasa, I believe I fixed the issue. It seems that the onBind fix was removed at some point from the PR. Once the update version builds in CI, could you test again?

@snaik20, @Stypox as far as I understand, the custom IBinder that PlayerService has doesn't forward the MediaBrowserService messages to the superclass's implementation, so to fix it I had to check the intent and decide on which IBinder to return according to the intent. If there's a better or common way of implementing this, I'd be happy to learn.

[haggaie]

Regarding changing the app to be listed as if it's from Google Play store, I think it's worth checking, though I don't know if that would be enough for all of Google Assistant's functionality to work. Anyway, I would suggest addressing this in a separate PR, if the maintainers agree.

[sonarqubecloud]

Quality Gate passed

Issues
8 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[ready2code31]

Regarding changing the app to be listed as if it's from Google Play store, I think it's worth checking, though I don't know if that would be enough for all of Google Assistant's functionality to work. Anyway, I would suggest addressing this in a separate PR, if the maintainers agree.

OMG ITS WORKING
Thx a lot, what a fast fix
Now i can view newpipe playlist on android auto and controls work fine (play/pause/next/previous)

[notbasa]

I can confirm that it now works on my Android 14 & Peugeot e2008 combination as well. Great work and thank you all.
Basic player controls work and you can also do search for songs (missing for InnerTune which is the other AA enabled GMusic player I've found).
Some way to browse content and add songs to the play queue would be great additional improvement.

[haggaie]

I can confirm that it now works on my Android 14 & Peugeot e2008 combination as well. Great work and thank you all. Basic player controls work and you can also do search for songs (missing for InnerTune which is the other AA enabled GMusic player I've found). Some way to browse content and add songs to the play queue would be great additional improvement.

Thanks @notbasa for testing. When you say suggest a way to browse content, you mean in addition to the saved playlists and history? Something like the home screen of the app?

[notbasa]

Thanks @notbasa for testing. When you say suggest a way to browse content, you mean in addition to the saved playlists and history? Something like the home screen of the app?

Yes. Currently the main screen is just empty. It would be great if if could show the same/similar stuff like the Youtube Music app, so recommended lists & personal library contents. (Not really sure what the right terms are). However showing just the same content as on the Newpipe main screen does not seem to make much sense, as at least for me it shows mostly just latest YT content and not sure if there's a setting somewhere to make it focus on music content.
All in all, I think it sounds like a new feature ticket rather than something that should be included here?
It would be nice to hear what YT Music premium uses the AA screen space for. Here's some screenshots to clarify: https://groovyandroid.com/youtube-music-for-android-auto-overview-and-screenshot-tour-15815/

[haggaie]

Yes. Currently the main screen is just empty. It would be great if if could show the same/similar stuff like the Youtube Music app, so recommended lists & personal library contents. (Not really sure what the right terms are). However showing just the same content as on the Newpipe main screen does not seem to make much sense, as at least for me it shows mostly just latest YT content and not sure if there's a setting somewhere to make it focus on music content. All in all, I think it sounds like a new feature ticket rather than something that should be included here? It would be nice to hear what YT Music premium uses the AA screen space for. Here's some screenshots to clarify: https://groovyandroid.com/youtube-music-for-android-auto-overview-and-screenshot-tour-15815/

That makes sense, though I agree it sounds like a separate feature. The current version of this PR does show the saved playlists, though that would be empty for a new install (and the CI version would probably be considered as such). I don't think NewPipe normally plays local music files, right? So that part wouldn't really fit, but it does make sense to show some suggestions based on what is currently playing or has been played recently, maybe similarly to the auto-play feature.

[notbasa]

I don't think NewPipe normally plays local music files, right? So that part wouldn't really fit, but it does make sense to show some suggestions based on what is currently playing or has been played recently, maybe similarly to the auto-play feature.

Fully agree. Just to clarify by personal library files I was referring to music uploaded to YouTube Music (or moved from Google Music when it closed) like YT Music does, not local files on the phone. Anyway agree it is out of scope for this ticket.
Already as it is Newpipe is now the best YT capable music player for people with Android auto.