Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speech Command Tutorial #18

Open
khanhlvg opened this issue Aug 28, 2020 · 14 comments
Open

Speech Command Tutorial #18

khanhlvg opened this issue Aug 28, 2020 · 14 comments

Comments

@khanhlvg
Copy link

@farmaker47 thank you for agreeing to help with the speech command tutorial!

Background

  • Speech command sample is already available on GitHub but there's no good documentation explaining how to use it.
  • We'd like you to help with creating and end-to-end tutorial on how to train the TFLite model and deploy it to a mobile app.

Our ask

  • Turn these scripts into a single Colab notebook that demonstrate how to train a speech command TFLite model. You can refer to this tutorial for idea.
  • In the end of the notebook, add an instruction on how to integrate the model into the speech_command Android (and iOS, if possible) app.
@farmaker47
Copy link
Contributor

I started working on it!
Thanks

@farmaker47
Copy link
Contributor

Hi Khanh,

@khanhlvg
There is a broken link inside project... at this file line 20.

Has this tutorial moved to new location?

Thanks

@khanhlvg
Copy link
Author

khanhlvg commented Sep 3, 2020

I just learnt that there's an engineer in TFLite team is now working on rebuilding the speech command sample using TF2 and that may overlap with the tutorial that you're working on.

Do you mind pausing the effort for now? Sorry for the back and forth...

There is a broken link inside project... at this file line 20.

I think it was taken down from TF.org because it's a TF 1 tutorial. I think it's the same one with this document on GitHub.
https://github.com/tensorflow/docs/blob/master/site/en/r1/tutorials/sequences/audio_recognition.md

@farmaker47
Copy link
Contributor

Yes no problem,

I saw that code was with TF version 1.14 and I was wondering if everything was going to compile.
It is a wise decision to pause the effort for creating the colab notebook for now...but nothing will stop me from converting Speech commands app to Kotlin... :) :) :)

Thanks

@khanhlvg
Copy link
Author

khanhlvg commented Sep 3, 2020

but nothing will stop me from converting Speech commands app to Kotlin... :) :) :)

That's nice! There won't be overlapping on that so please proceed :) Once the training code is converted to TF2, we can connect it with the new Kotlin app that you are going to build.

@farmaker47
Copy link
Contributor

Hi Khanh,

I want to ask if I can use the same colors, TensorFlow logos and layouts inside Speech commands Kotlin app. Is it a problem?

@farmaker47
Copy link
Contributor

Update.
App is almost ready.
Kotlin, MVVM, DI, Coroutines.

Working already:

  • When user says one of the screen words the specific boxes are highlighted.
  • UI is exactly the same

Not working yet:

  • Minus and plus button for threads

Also I want to ask if I have to unhide selection for NNAPI delegate and make it work. This option is hidden and apparently never completed. Do you think I have to show it now?

Thanks

@khanhlvg
Copy link
Author

khanhlvg commented Sep 8, 2020

I want to ask if I can use the same colors, TensorFlow logos and layouts inside Speech commands Kotlin app. Is it a problem?

Sure, feel free to use the TF colors.

Also I want to ask if I have to unhide selection for NNAPI delegate and make it work. This option is hidden and apparently never completed. Do you think I have to show it now?

If it wasn't shown in the current app UI and not currently used then I think it's fine to keep it that way. Delegate support would be nice to have but I haven't tested if the model is supported by any of the current delegates.

@farmaker47
Copy link
Contributor

farmaker47 commented Sep 8, 2020

I have already finished app.
Now minus and plus button select number of threads and this is kept upon rotation.
Also application is working during configuration changes.

I have created a landscape layout file:
Screenshot_2020-09-08-11-03-27-816_com george speech_commands_kotlin

I will work the READ.ME file for some more info for the users.

Please build to confirm that everything is working and get back with some suggestions of improvement.

Thank you

@farmaker47
Copy link
Contributor

farmaker47 commented Sep 8, 2020

I played with some delegates

It seems that GPU is not supported as there is no improvement of speed.

But I think NnApiDelegate is supported as it give results....lower inference time but it runs with no problems.
CPU time 100ms with 1 thread
---------- 60ms with 2 threads
----------- no difference with more threads

NnApiDelegate time 350ms with 1 thread
----------------------- no difference with more

@khanhlvg
Copy link
Author

khanhlvg commented Sep 9, 2020

It's very likely that NNAPI CPU implementation is slower, especially if you're using an older phone because NNAPI implementation is included in Android while TFLite CPU implementation can be updated over time.

Given that the delegates don't provide any performance advantage, let's exclude it from the UI then.

@sayakpaul
Copy link
Contributor

@farmaker47 are you still working on this project?

@farmaker47
Copy link
Contributor

No I am not Sayak . I have just finished the android application.

@sayakpaul
Copy link
Contributor

I think we could still include the application as a demo app within our README.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants