Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add custom accessibility annotations to AnnotatedStringResource #1402

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

francoisadam
Copy link

📋 Changes

  • Added string annotation handling for all TtsSpan types inside the AnnotatedStringResource
  • Convert strings with these annotations to more human readable equivalent
  • Added a new tab in the catalog app to preview and check new accessibility improvements
  • Added coreLibDesugaring to Spark to handle date parsing (will have to check for size impacts)

🤔 Context

For most content descriptions, TalkBack won't read properly the words unless we do some formatting.
Before Compose, there was the option to use TtsSpan to add metadate to text and help TalkBack properly read the words.
But there is only one annotation available for Compose as of right now (Verbatim), so we need to do it manually for now.

✅ Checklist

  • Add custom Tts annotations
  • Add a new tab in the catalog app to display and test these annotations
  • Add unit tests once these new annotations are approved
  • Add other annotations for specific strings like zipcode for example
  • Put these accessibility annotation handling in a different extension to make sure this is only done for contentDescriptions and not for user-visible texts
  • Make it i18n compatible ?

📸 Screenshots

Comment on lines +304 to +310
annotatedString = annotatedString.buildWithAccessibilitySpan(it, start, end)
}
append(annotatedString)
getSpans(0, length, Any::class.java).forEach {
val start = getSpanStart(it)
val end = getSpanEnd(it)
buildWithSpanStyle(it, start, end, density, colors, typography)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure to fully understand why we duplicated this part of the code. Can't we merge the tts handling with the already existing buildWithSpan which is already parsing the entire string/annotations once?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's the frustrating part, TtsSpan in spannable uses metadata to add info for Tts to better read the words but this is not possible yet in compose and AnnotatedStrings (well only Verbatim is possible for now)

So the only way I found was to actually reformat the string itself to make it more readable by Tts.

The first part with buildWithAccessibilitySpan updates the strings before using append, and the second part with buildWithSpanStyle only adds styling to the annotated string (and so needs to be done after the append)

But if you know how to extend the TtsAnnotation from Compose, or a better way to do this I'm all ears 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants