From c1ca64d7e2b379a3e6f31eb341102215327ee0b1 Mon Sep 17 00:00:00 2001 From: Nick Gerleman Date: Sat, 27 Feb 2021 07:30:55 -0800 Subject: [PATCH 1/6] Keyboard-friendly List Virtualization Behavior VirtualizedList virtualizes away items based on proximity to the lists viewport, without consideration for other features such as focus or selection. This strategy limits the ability to correctly implement some common keyboard interactions. This especially impacts desktop platforms (react-native-web, react-native-windows, react-native-macos), but also impacts the tablet with keyboard form-factor on iPadOS/Android. This proposal outlines these issues, proposing a set of behavior changes and new APIs to address them. It does not address concrete implementation. --- .../0006-Keyboard-Friendly-Virtualization.md | 186 ++++++++++++++++++ 1 file changed, 186 insertions(+) create mode 100644 proposals/0006-Keyboard-Friendly-Virtualization.md diff --git a/proposals/0006-Keyboard-Friendly-Virtualization.md b/proposals/0006-Keyboard-Friendly-Virtualization.md new file mode 100644 index 00000000..2e0ed321 --- /dev/null +++ b/proposals/0006-Keyboard-Friendly-Virtualization.md @@ -0,0 +1,186 @@ +--- +title: "RFC0006: Keyboard-friendly List Virtualization Behavior" +author: + - Nick Gerleman +date: February 2021 +--- + +# RFC0006: Keyboard-friendly List Virtualization Behavior + +## Summary + +VirtualizedList virtualizes away items based on proximity to the lists viewport, +without consideration for other features such as focus or selection. This +strategy limits the ability to correctly implement some common keyboard +interactions. This especially impacts desktop platforms (react-native-web, +react-native-windows, react-native-macos), but also impacts the tablet with +keyboard form-factor on iPadOS/Android. + +This proposal outlines these issues, proposing a set of behavior changes and new +APIs to address them. It does not address concrete implementation. + +## Basic example + +```jsx + +``` + +## Motivation + +### Keyboarding issues with VirtualizedList (and FlatList/SectionList) + +Keyboard users expect to be able to navigate between between views inside of a +list via tab-loop or arrow keys. A component with focus may move out of +visibility by panning or scrolling a parent surface. The focused component may +be virtualized away after leaving visibility. + +Removing components in the focus chain breaks some common keyboard scenarios: +1. Moving to a previous/next selected item via arrow-key +1. Tabbing focus to a user-definable next component +1. Imperatively moving focus to a user-defined component + +## Detailed design + +### Realization windows + +Focus may be synchronously moved by the native UI framework. This forces us to +prevent virtualization (keep realized) any components which focus may be moved +to. We may see additional navigation after the initial focus change, unblocked +on the rendering of new components. This necessitates a chain of additional +realized components. + +Common keyboarding scenarios should "just work" without having to change +settings. Three of these scenarios include: + +1. Navigating from one child component of a rendered item to another +1. Arrowing between focused items +1. Shifting to first/last focused item via home/end keys + +`VirtualizedList` has enough internal knowledge to transparently support these +scenarios. This can be accomplished by adding three **"realization windows"**: +1. **"focused"** Realize cells around a cell with active focus. Can be +tracked by listening to bubbling onFocus in the default `CellRendererComponent`. +1. **"home"** The area at the beginning of the list (for home key). +1. **"end"** The area at the end of the list (for end key). + +The above scenarios are not exhaustive. E.g. we may want to keep a *selected* +item realized that is not *focused*. These scenarios may require additional APIs +allowing external users greater control over realization behavior. These could +potentially be exposed in the future as dynamic/user-controllable realization +windows, or separate high-level APIs. + +### Endpoint-specific default behaviors + +VirtualizedList should be keyboard-friendly by default on desktop platforms as +they are oriented towards usage of a physical keyboard. This additional +realization is not free however, increasing memory usage to retain additional +cells. This tradeoff may not make sense for mobile. + +This leads to desired behavior of: +- **Keep FlatList focus-safe on desktop:** Common realization windows for +needed for keyboard navigation are enabled by default. +- **Keep FlatList memory usage steady on mobile:** New realization windows +are opt-in where not pay to play. + +The full matrix of behaviors is described in the *API Reference* below. In +general, **desktop users should realize ** + +Endpoint detection is generally performed via `Platform` APIs, such as +`Platform.isTV()` or `Platform.isPad()`. This change would add a +`Platform.isDesktop()` endpoint query to allow desktop-specific behavior. + +### New VirtualizedList props + +Properties should be added to VirtualizedList (and FlatList/SectionList) to +allow optimizing for app-specific needs. This can be exposed as a +`RealizationWindowConfig`, describing the size and enablement of predefined +realization windows. This API could be expanded in the future to allow +dynamically controlled realization behaviors. + + +### API reference + +```tsx +/** + * Allows enabling or disabling an area for realization, optionally overriding + * its window size. + */ +export type RealizationWindow = + | boolean + | [boolean, {windowSize?: number}]; + +/** + * Determines what areas of the list to keep realized. Enabling realization may + * be required for correct behavior when focus is out of viewport, but may also + * increase memory usage. + */ +export type RealizationWindowConfig = { + /** + * Keeps the area around the currently focused item realized. + * - Enabled by default on desktop/tv (where focus-loop is important) + * - Enabled by default on mobile, where this can only use more memory when + * physical-keyboard is used. + * + * Requires the default (CellRenderComponent)[ + * https://reactnative.dev/docs/virtualizedlist#cellrenderercomponent] to be + * used + * + * Defaults to the (VirtualizedList windowSize)[ + * https://reactnative.dev/docs/virtualizedlist#windowsize] + */ + focused?: RealizationWindow; + + /** + * Keeps the beginning area of the list realized. + * - Enabled by default on desktop (for home-key navigation) + * - Disabled by default on mobile/TV + * + * Defaults to **half** the (VirtualizedList windowSize)[ + * https://reactnative.dev/docs/virtualizedlist#windowsize] + */ + home?: RealizationWindow; + + /** + * Keeps the end area of the list realized. + * - Enabled by default on desktop (for end-key navigation) + * - Disabled by default on mobile/TV + * + * Defaults to **half** the (VirtualizedList windowSize)[ + * https://reactnative.dev/docs/virtualizedlist#windowsize] + */ + end?: RealizationWindow; + + // Potential to add additional props for different (or dynamic) realization + // areas +} + +/** + * VirtualizedList (and FlatList/SectionList) component properties + */ +export type VirtualizedListProps = { + ... + + /** + * Enables realization of different areas of the VirtualizedList + */ + keepAreasRealized?: RealizationWindowConfig; +} +``` + +## Drawbacks + +Apart from implementation complexity and risk, the greatest drawback to +additional realization is memory consumption. This motivates configurability to +mitigate impact. + +## Alternatives + +The main alternative to improving keyboarding support of existing components on +desktop platforms would be to add additional virtualization components unique to +them. From 1cbee9dccc2a26391ac584e3787e4f5873b7b58c Mon Sep 17 00:00:00 2001 From: Nick Gerleman Date: Sat, 27 Feb 2021 07:44:03 -0800 Subject: [PATCH 2/6] Fix some typos --- proposals/0006-Keyboard-Friendly-Virtualization.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/proposals/0006-Keyboard-Friendly-Virtualization.md b/proposals/0006-Keyboard-Friendly-Virtualization.md index 2e0ed321..3d7b046e 100644 --- a/proposals/0006-Keyboard-Friendly-Virtualization.md +++ b/proposals/0006-Keyboard-Friendly-Virtualization.md @@ -1,7 +1,6 @@ --- title: "RFC0006: Keyboard-friendly List Virtualization Behavior" -author: - - Nick Gerleman +author: Nick Gerleman date: February 2021 --- @@ -23,7 +22,7 @@ APIs to address them. It does not address concrete implementation. ```jsx Date: Sun, 28 Feb 2021 22:40:04 -0800 Subject: [PATCH 3/6] Revisions --- .../0006-Keyboard-Friendly-Virtualization.md | 58 +++++++++---------- 1 file changed, 26 insertions(+), 32 deletions(-) diff --git a/proposals/0006-Keyboard-Friendly-Virtualization.md b/proposals/0006-Keyboard-Friendly-Virtualization.md index 3d7b046e..fbc91984 100644 --- a/proposals/0006-Keyboard-Friendly-Virtualization.md +++ b/proposals/0006-Keyboard-Friendly-Virtualization.md @@ -9,7 +9,7 @@ date: February 2021 ## Summary VirtualizedList virtualizes away items based on proximity to the lists viewport, -without consideration for other features such as focus or selection. This +without consideration for other inputs such as focus or selection. This strategy limits the ability to correctly implement some common keyboard interactions. This especially impacts desktop platforms (react-native-web, react-native-windows, react-native-macos), but also impacts the tablet with @@ -32,34 +32,30 @@ APIs to address them. It does not address concrete implementation. ## Motivation -### Keyboarding issues with VirtualizedList (and FlatList/SectionList) - Keyboard users expect to be able to navigate between between views inside of a list via tab-loop or arrow keys. A component with focus may move out of visibility by panning or scrolling a parent surface. The focused component may be virtualized away after leaving visibility. -Removing components in the focus chain breaks some common keyboard scenarios: -1. Moving to a previous/next selected item via arrow-key -1. Tabbing focus to a user-definable next component -1. Imperatively moving focus to a user-defined component +The removal of this native component disrupts state needed to move focus/selection. ## Detailed design ### Realization windows -Focus may be synchronously moved by the native UI framework. This forces us to +Focus may be moved by the native UI framework without waiting on a new component to be rendered. This forces us to prevent virtualization (keep realized) any components which focus may be moved to. We may see additional navigation after the initial focus change, unblocked on the rendering of new components. This necessitates a chain of additional realized components. -Common keyboarding scenarios should "just work" without having to change -settings. Three of these scenarios include: +Common keyboarding scenarios should ideally work by default, without having to change +settings. Some of these scenarios include 1. Navigating from one child component of a rendered item to another 1. Arrowing between focused items -1. Shifting to first/last focused item via home/end keys +1. Shifting focus from currently focused item via page up/down +1. Shifting focus to first/last item via home/end keys `VirtualizedList` has enough internal knowledge to transparently support these scenarios. This can be accomplished by adding three **"realization windows"**: @@ -68,6 +64,9 @@ tracked by listening to bubbling onFocus in the default `CellRendererComponent`. 1. **"home"** The area at the beginning of the list (for home key). 1. **"end"** The area at the end of the list (for end key). +> Note that Apple devices have "fn + left" or "fn + right" which behave similarly +> to home/end, but do not move selection in apps like Finder. + The above scenarios are not exhaustive. E.g. we may want to keep a *selected* item realized that is not *focused*. These scenarios may require additional APIs allowing external users greater control over realization behavior. These could @@ -82,13 +81,12 @@ realization is not free however, increasing memory usage to retain additional cells. This tradeoff may not make sense for mobile. This leads to desired behavior of: -- **Keep FlatList focus-safe on desktop:** Common realization windows for -needed for keyboard navigation are enabled by default. +- **Keep FlatList focus-safe on desktop:** Realization windows needed for +correct keyboard navigation should be enabled by default. - **Keep FlatList memory usage steady on mobile:** New realization windows are opt-in where not pay to play. -The full matrix of behaviors is described in the *API Reference* below. In -general, **desktop users should realize ** +The full matrix of behaviors is described in the *API Reference* below. Endpoint detection is generally performed via `Platform` APIs, such as `Platform.isTV()` or `Platform.isPad()`. This change would add a @@ -122,9 +120,7 @@ export type RealizationWindow = export type RealizationWindowConfig = { /** * Keeps the area around the currently focused item realized. - * - Enabled by default on desktop/tv (where focus-loop is important) - * - Enabled by default on mobile, where this can only use more memory when - * physical-keyboard is used. + * - Enabled by default on all platforms * * Requires the default (CellRenderComponent)[ * https://reactnative.dev/docs/virtualizedlist#cellrenderercomponent] to be @@ -136,21 +132,25 @@ export type RealizationWindowConfig = { focused?: RealizationWindow; /** - * Keeps the beginning area of the list realized. - * - Enabled by default on desktop (for home-key navigation) - * - Disabled by default on mobile/TV + * Keeps the beginning area of the list realized. Useful to allow instantly + * shifting focus to the first item (e.g. via Home key). + * - Disabled by default on all platforms + * - Should be enabled by default on desktop if FlatList begins transparently + * supporting home/end focus. * - * Defaults to **half** the (VirtualizedList windowSize)[ + * Defaults to **1/4th** the (VirtualizedList windowSize)[ * https://reactnative.dev/docs/virtualizedlist#windowsize] */ home?: RealizationWindow; /** - * Keeps the end area of the list realized. - * - Enabled by default on desktop (for end-key navigation) - * - Disabled by default on mobile/TV + * Keeps the end area of the list realized. Useful to allow instantly + * shifting focus to the first item (e.g. via End key). + * - Disabled by default on all platforms + * - Should be enabled by default on desktop if FlatList begins transparently + * supporting home/end focus. * - * Defaults to **half** the (VirtualizedList windowSize)[ + * Defaults to **1/4th** the (VirtualizedList windowSize)[ * https://reactnative.dev/docs/virtualizedlist#windowsize] */ end?: RealizationWindow; @@ -177,9 +177,3 @@ export type VirtualizedListProps = { Apart from implementation complexity and risk, the greatest drawback to additional realization is memory consumption. This motivates configurability to mitigate impact. - -## Alternatives - -The main alternative to improving keyboarding support of existing components on -desktop platforms would be to add additional virtualization components unique to -them. From e78bff3a2bdf4c7b3b1004e77289d2cc286c257b Mon Sep 17 00:00:00 2001 From: Nick Gerleman Date: Sun, 28 Feb 2021 23:11:55 -0800 Subject: [PATCH 4/6] Support custom CellRendererComponent --- proposals/0006-Keyboard-Friendly-Virtualization.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/proposals/0006-Keyboard-Friendly-Virtualization.md b/proposals/0006-Keyboard-Friendly-Virtualization.md index fbc91984..73ee1c01 100644 --- a/proposals/0006-Keyboard-Friendly-Virtualization.md +++ b/proposals/0006-Keyboard-Friendly-Virtualization.md @@ -122,9 +122,9 @@ export type RealizationWindowConfig = { * Keeps the area around the currently focused item realized. * - Enabled by default on all platforms * - * Requires the default (CellRenderComponent)[ - * https://reactnative.dev/docs/virtualizedlist#cellrenderercomponent] to be - * used + * To function with a non-default (CellRendererComponent)[ + * https://reactnative.dev/docs/virtualizedlist#cellrenderercomponent], + * `onFocus` and `onBlur` props must be supported. * * Defaults to the (VirtualizedList windowSize)[ * https://reactnative.dev/docs/virtualizedlist#windowsize] From cdf77f0ed45caf58ea22847b5d0da1bc5124014e Mon Sep 17 00:00:00 2001 From: Nick Gerleman Date: Tue, 9 Mar 2021 17:55:49 -0800 Subject: [PATCH 5/6] Updates --- .../0006-Keyboard-Friendly-Virtualization.md | 55 +++++++++++-------- 1 file changed, 33 insertions(+), 22 deletions(-) diff --git a/proposals/0006-Keyboard-Friendly-Virtualization.md b/proposals/0006-Keyboard-Friendly-Virtualization.md index 73ee1c01..8a30282d 100644 --- a/proposals/0006-Keyboard-Friendly-Virtualization.md +++ b/proposals/0006-Keyboard-Friendly-Virtualization.md @@ -49,8 +49,8 @@ to. We may see additional navigation after the initial focus change, unblocked on the rendering of new components. This necessitates a chain of additional realized components. -Common keyboarding scenarios should ideally work by default, without having to change -settings. Some of these scenarios include +Common keyboarding scenarios should ideally work by default, without having to +change settings. Some of these scenarios include 1. Navigating from one child component of a rendered item to another 1. Arrowing between focused items @@ -59,19 +59,33 @@ settings. Some of these scenarios include `VirtualizedList` has enough internal knowledge to transparently support these scenarios. This can be accomplished by adding three **"realization windows"**: -1. **"focused"** Realize cells around a cell with active focus. Can be -tracked by listening to bubbling onFocus in the default `CellRendererComponent`. +1. **"focused"** Realize cells around a cell with most recent focus. Can be +tracked by listening to bubbling onFocusCapture in the default +`CellRendererComponent`. 1. **"home"** The area at the beginning of the list (for home key). + - Note that part of the top may already reamain realized, since the initial + render is retained. This may not be the case with non-default + `initialItemIndex` however, and the height may not correspond to layout + window. 1. **"end"** The area at the end of the list (for end key). > Note that Apple devices have "fn + left" or "fn + right" which behave similarly > to home/end, but do not move selection in apps like Finder. -The above scenarios are not exhaustive. E.g. we may want to keep a *selected* -item realized that is not *focused*. These scenarios may require additional APIs -allowing external users greater control over realization behavior. These could -potentially be exposed in the future as dynamic/user-controllable realization -windows, or separate high-level APIs. +The above scenarios are not exhaustive. It may be required to add additional +APIs allowing external users greater control over realization behavior. These +ould potentially be exposed in the future as dynamic/user-controllable +realization windows, or separate high-level APIs. + +### Batch rendering behavior + +After rendering a predefined number of initial items, `VirtualizedList` renders +new items in batches. Batches may not render the full extent of items to +realize, and the number of items to render per-batch is configurable. + +Cells in realization windows should be automatically rendered, even if not yet +visible. This can be done during batch renders, giving priority to first render +the window around visible content. ### Endpoint-specific default behaviors @@ -113,9 +127,9 @@ export type RealizationWindow = | [boolean, {windowSize?: number}]; /** - * Determines what areas of the list to keep realized. Enabling realization may - * be required for correct behavior when focus is out of viewport, but may also - * increase memory usage. + * Determines what areas of the list to keep realized. Realizing extra windows + * may be required for correct behavior when focus is out of viewport, but may + * also increase memory usage. */ export type RealizationWindowConfig = { /** @@ -124,22 +138,20 @@ export type RealizationWindowConfig = { * * To function with a non-default (CellRendererComponent)[ * https://reactnative.dev/docs/virtualizedlist#cellrenderercomponent], - * `onFocus` and `onBlur` props must be supported. + * `onFocusCapture` and `onBlurCapture` props must be supported. * - * Defaults to the (VirtualizedList windowSize)[ - * https://reactnative.dev/docs/virtualizedlist#windowsize] + * Defaults to keeping the screen centered around the focused cell, along + * with a screen above and below realized. */ focused?: RealizationWindow; /** * Keeps the beginning area of the list realized. Useful to allow instantly * shifting focus to the first item (e.g. via Home key). - * - Disabled by default on all platforms - * - Should be enabled by default on desktop if FlatList begins transparently - * supporting home/end focus. + * - Disabled by default on all platforms (Should be enabled by default on + * desktop if FlatList begins transparently supporting home/end focus) * - * Defaults to **1/4th** the (VirtualizedList windowSize)[ - * https://reactnative.dev/docs/virtualizedlist#windowsize] + * Defaults to keeping a single screen at the beginning of the list realized */ home?: RealizationWindow; @@ -150,8 +162,7 @@ export type RealizationWindowConfig = { * - Should be enabled by default on desktop if FlatList begins transparently * supporting home/end focus. * - * Defaults to **1/4th** the (VirtualizedList windowSize)[ - * https://reactnative.dev/docs/virtualizedlist#windowsize] + * Defaults to keeping a single screen at the end of the list realized */ end?: RealizationWindow; From 4b055d05352e0389bf2561bb0d5f6a387a37a788 Mon Sep 17 00:00:00 2001 From: Nick Gerleman Date: Wed, 10 Mar 2021 10:11:38 -0800 Subject: [PATCH 6/6] no onBlurCapture needed --- proposals/0006-Keyboard-Friendly-Virtualization.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/0006-Keyboard-Friendly-Virtualization.md b/proposals/0006-Keyboard-Friendly-Virtualization.md index 8a30282d..4e2ef5c5 100644 --- a/proposals/0006-Keyboard-Friendly-Virtualization.md +++ b/proposals/0006-Keyboard-Friendly-Virtualization.md @@ -138,7 +138,7 @@ export type RealizationWindowConfig = { * * To function with a non-default (CellRendererComponent)[ * https://reactnative.dev/docs/virtualizedlist#cellrenderercomponent], - * `onFocusCapture` and `onBlurCapture` props must be supported. + * the `onFocusCapture` prop must be supported. * * Defaults to keeping the screen centered around the focused cell, along * with a screen above and below realized.