Skip to content

Platform Acceptance Test for iOS

Jump to bottom
rc-swag edited this page Nov 16, 2023 · 9 revisions

Warning

These acceptance tests are now deprecated as the source for formal testing. Formal testing is done against regression tests stored in a test suite managed by the test team. Developers can update acceptance tests here on the wiki, however, if the developer recommends a test needs to be added to the formal tests contact the test team.

Keyman for iPhone and iPad Acceptance Test Procedures

  1. These test procedures are to be run before moving from alpha to beta, or beta to stable, or before PRs are merged into stable branches.
  2. Copy these tests into a new issue (for tier transitions) or a comment on the PR.
  3. When annotating results, use > at the start of a new line under the checkbox to note the result. (For an example, see #3973).

User Testing

Gather Assets for Testing

  • Record device's iOS version
  • Ensure the device is properly configured for testing and is registered with TestFlight.

At this time, Keyman for iOS does not support external keyboard input.

Setup Steps

  1. Uninstall previous version of Keyman on the device (on a simulator, "Erase All Content and Settings..." under Device menu)
  2. Install test build IPA (on a simulator, "iOS Simulator Image")
  3. Disable internet connectivity of the device
  4. Start Keyman and select Add a keyboard for your language
  5. Verify error message "Could not reach Keyman server!" or "The Internet connection appears to be offline." is displayed
  6. Click OK to dismiss. Close Keyman app
  7. On device, enable internet connectivity
  8. Start Keyman
  9. On initial Get Started menu, select "Enable Keyman as system-wide keyboard" and do any necessary system steps to enable Keyman

SUITE_INSPECT: Visual Inspection of master branch

  • TEST_BUILD_STATUS: Build Status Verify latest CI build of master is successful
  • TEST_VERIFY_HISTORY: Verify history.md contains all the current changes
  • TEST_VERIFY_UNIT_TESTS: Verify that all unit tests for iOS are currently passing.
    • May be integrated with the first step above once our usual CI processes include the iOS unit tests.

SUITE_MENU_FUNCTIONALITY: Menu Functionality Tests, in-app

  • GROUP_IPHONE_PORTRAIT: portrait mode in iPhone
  • GROUP_IPHONE_LANDSCAPE: landscape mode in iPhone
  • GROUP_IPAD_PORTRAIT: portrait mode in iPad
  • GROUP_IPAD_LANDSCAPE: landscape mode in iPad

Test Cases

  • TEST_GET_STARTED: Get Started

    • Verify touching Add a keyboard for your language pulls up the keyboard search
      • This should also be accessible as Settings > Installed Languages > +.
    • Verify touching Enable Keyman as system-wide keyboard pulls up Keyman's menu within iOS settings. (11.0+ devices)
      • If older versions of iOS are supported, ensure that the help page for doing it manually is properly linked.
    • Verify touching More info brings up an accurate version and the main table-of-contents help page.
      • This should also be accessible as Info from the main menu dropdown (on phones) or as the circled-i button (on tablets).
    • Set Don't show again to checked, close and re-open Keyman, and verify Get Started menu does not appear

  • TEST_ADD_NEW_KEYBOARD: Add New Keyboard This menu is accessed via Get Started menu or Settings menu

    • On the device, enable internet connectivity
    • Verify a new keyboard can be downloaded and selected, say "US Basic"
    • Return to the text input screen and confirm that it operates correctly.
    • Without changing selected language, confirm that it can be cleanly deleted via the Settings menu.
    • Confirm that another keyboard is correctly selected afterward.

  • TEST_ADD_NEW_KEYBOARD_FROM_FILE:

    • On the device, go to keyman.com on Safari
    • Search for "US Basic"
    • scroll down to keyboard details and tap to download the keyboard package (i.e. .kmp file)
    • open Files app and navigate to Download folder to find "basic_kbdus.kmp" and tap on it to start the installation processes
    • Keyman app should be pulled up
    • follow the screen to finish the keyboard installation
    • verity that the keyboard is correctly installed and working as expected

  • TEST_SWITCH_KEYBOARD

    • press on the globe key
    • the keyboard menu should be shown
    • press on a keyboard to switch to, say "EuroLatin (SIL)"
    • confirm that "EuroLatin (SIL)" is now selected and ready to be used
    • swich back to "US Basic" and confirm it does as expected

  • TEST_SHARE: Share

    • Type and verify text can be copied and shared to external app

  • TEST_KEYMAN_BROWSER: Keyman Browser ###

    • On default page, click on text field and set system keyboard to Keyman
    • Verify can type with Keyman as system keyboard
    • Verify that predictive suggestions are properly applied when selected (using sil_euro_latin + English)
    • Close and reopen Keyman app

  • TEST_TEXT_SIZE: Text Size

    • Verify text can be rescaled from Text Size 9 to 72

  • TEST_CLEAR_TEXT: Clear Text

    • Verify text can be cleared

  • TEST_INFO: Info

    • Verify that the Keyman iOS version is displayed; record the version
    • Verify that the Info menu options matches that of the More Info... option from the Get Started menu option.
    • Verify that the Keyman iOS version matches the version string seen at (Home Screen) > [iOS] Settings > Keyman.

  • TEST_SETTINGS: Settings

    • Installed Languages ... Add Language

    • Select an installed language ... Add (another) Keyboard

      • Should automatically perform a filtered search based upon that language's BCP-47 code, i.e. for "English" it should be seen as shown in the screenshot below.

    • Select an installed language ... Dictionary

      • Turn on/off 'Enable Predictions' and validate each
      • Turn on/off 'Enable Corrections' and validate each

    • Validate that when both predictions and corrections are off, banner is not visible

    • Add/Remove dictionaries - validate

    • If multiple dictionaries are available, test swapping between them

SUITE_IN_APP_PHONE_ONLY:

  • GROUP_IPHONE_PORTRAIT: portrait mode in iPhone
  • GROUP_IPHONE_LANDSCAPE: landscape mode in iPhone

Test Cases

  • TEST_US_BASIC: English: US Basic in-app
    • Verify that the US Basic keyboard can be downloaded through Settings > Installed Languages > English > +
      • This should automatically bring up search results for English keyboards.
      • When the package-installer launches, English should be the one pre-selected language.
    • Install the package for both English and Tagalog by selecting both of them in the installation processes.
    • Set English: US Basic as the active in-app keyboard.
    • Verify that predictive text suggestions appear.
    • Set Tagalog: US Basic as the active in-app keyboard.
    • Verify that predictive text suggestions do not appear.
    • Verify that long-press 123 key works, presenting options for a currency layer and a symbol layer.
    • Select the currency layer, then press the £ key.
    • Verify that the keyboard outputs correctly and returns to the default layer.
    • Go to the symbol layer and press the © key.
    • Verify that the keyboard outputs correctly and remains on the symbol layer.
    • Revert to the default layer and ensure basic key inputs work.
    • Long-press e on the default layer and select a subkey.
    • Verify that the selected key produces the correct output.
    • From the Settings > Languages > English > US Basic menu, follow the "Help link" and ensure it displays appropriate help.
      • (Should work both online and offline)
    • From the Settings > Languages > English > US Basic menu, scan the QR code with a phone and test that it links to the current version of that keyboard's public download page on keyman.com.
    • Returning to the app, uninstall the US Basic keyboard via Settings for Tagalog.
    • Set English: US Basic as the active in-app keyboard.
    • Verify that the keyboard appears correctly and ensure basic key inputs work.
      • This is to ensure that uninstalling for one language doesn't adversely affect installations of the same resource for another language.
    • Uninstall the US Basic keyboard via Settings for English.

SUITE_IN_APP_KEYBOARD:

  • GROUP_IPHONE_PORTRAIT: portrait mode in iPhone
  • GROUP_IPHONE_LANDSCAPE: landscape mode in iPhone
  • GROUP_IPAD_PORTRAIT: portrait mode in iPad
  • GROUP_IPAD_LANDSCAPE: landscape mode in iPad

Test Cases

  • TEST_EUROLATIN: English (Eurolatin) in-app
    • In portrait orientation, verify that the OSK appears and fills the width the bottom of the screen
    • Verify long-press q key works
    • Verify long-press k key works
      • Verify that any predictive suggestions generated feel 'reasonable' (based upon 'k', the selected subkey, or one of its neighbors)
    • Verify uppercase layer can be selected via SHIFT
    • Verify number layer can be selected via 123
    • Verify long-press 1 key works
    • Verify long-press 0 key works
    • Verify backspace, space, and enter keys work

SUITE_SYSTEM_KEYBOARD:

  • GROUP_IPHONE_PORTRAIT: portrait mode in iPhone
  • GROUP_IPHONE_LANDSCAPE: landscape mode in iPhone
  • GROUP_IPAD_PORTRAIT: portrait mode in iPad
  • GROUP_IPAD_LANDSCAPE: landscape mode in iPad

Test the following after setting Keyman as a system keyboard.

Test Cases

  • TEST_BASIC_USAGE: ensure that the keyboards functions as expected on all layers. Please test on multiple keyboards, i.e. EuroLatin (SIL), Cameroon QWERTY keyboard and Khmer Angkor.
  • TEST_LONG_PRESS: ensure that longpress on all applicable layers are working as intended.
    • Note that khmer_angkor lacks longpress keys on tablet form-factor devices, though the other two listed above still have them.
  • TEST_GLOBE_KEY: ensure that a short press on the globe key switch to the next keyboard and a longpress on it open the keyboard menu where you can select a keyboard of your choice

SUITE_SWITCH_LOCALE

  • GROUP_IPHONE_PORTRAIT: portrait mode in iPhone
  • GROUP_IPHONE_LANDSCAPE: landscape mode in iPhone
  • GROUP_IPAD_PORTRAIT: portrait mode in iPad
  • GROUP_IPAD_LANDSCAPE: landscape mode in iPad

Test Cases

  • TEST_KM_LOCALE
    • from System Preferences, switch the UI language to Khmer by adding it in Language & Region using +
    • drag Khmer to the top of the list to set it as a primary UI language
    • check the dropdown menu to see it in Khmer
    • verify the UI language switch to Khmer
  • TEST_AM_LOCALE
    • from System Preferences, switch the UI language to Amharic by adding it in Language & Region using +
    • drag Amharic to the top of the list to set it as a primary UI language
    • check the dropdown menu to see it in Amharic
    • verify the UI language switch to Amharic

SUITE_PREDICTIVE_TEXT_AND_AUTO_CORRECTION: Predictive Text and Auto Correction

  • GROUP_IPHONE_PORTRAIT: portrait mode in iPhone
  • GROUP_IPHONE_LANDSCAPE: landscape mode in iPhone
  • GROUP_IPAD_PORTRAIT: portrait mode in iPad
  • GROUP_IPAD_LANDSCAPE: landscape mode in iPad

Test Cases

  • TEST_BROKEN_SUGGESTION: Verify that the following reproduction is no longer affected by this bug.

    1. In Keyman for iOS, type Prob using the default keyboard (EuroLatin).
    2. Select Problem from banner.
    3. Press Backspace, then select "Prob" (undoing the last suggestion).
    4. Then press Spacebar.
    5. You should see Prob , as if no suggestion were auto-applied from the spacebar.

  • TEST_QUOTE_O: Using the default English keyboard for your test environment, type ' followed immediately by o.

    • If most of the displayed suggestions do not begin with an 'o', FAIL this test.
      • Do not fail if "I" is one of them - it's a super-common word, and 'o' does neighbor 'i'.
    • Apply any of the displayed suggestions. If the ' mark is replaced, *FAIL_ this test.
    • (nts: This test and the other two below are adopted from https://github.com/keymanapp/keyman/pull/7205.)

  • TEST_QUOTE_OP: Using the default English keyboard for your test environment, type ' followed immediately by o, then p.

    • If most of the displayed suggestions do not begin with an 'op', FAIL this test.
      • Do not fail if "OP" is suggested, despite its capitalization.
    • Apply any of the displayed suggestions. If the ' mark is replaced, FAIL this test.

  • TEST_OP: Using the default English keyboard for your test environment, type you, add a space, then type o, then p.

    • If most of the displayed suggestions do not begin with an op, FAIL this test.
      • Do not fail if "OP" is suggested, despite its capitalization.
    • Apply any of the displayed suggestions. If unwanted effects occur, FAIL this test.
      • No prior whitespace should be deleted and the suggestion should be cleanly applied.
      • Example: applying open to you op should result in you open for the full context.

  • TEST_NW_Q: Using the default (sil_euro_latin) keyboard, press the far upper-left of the 'q' (or 'Q') key.

  • TEST_E_P: Using the default (sil_euro_latin) keyboard, press the far upper-right of the 'p' (or 'P') key.

    • All suggestions should begin with the corresponding letter and case.

  • TEST_GUILLEMET: Guillemets («») are used to quote a speech in some language, i.e. Khmer and Jarai. We would like to ensure that the quillemet won't get delete when a predictive text is in action.

    • In Keyman for iOS app with Jarai keyboard active, go to the shift layer and press « on the first row.
    • Now press ស (the second key from the left on the third row) and pick the first option on the suggestion banner.
    • FAIL this test if the guillemet disappears.
Clone this wiki locally