Support capturing "long screenshots" using API 31+ ScrollCapture APIs #2076
Description
Problem
When a screen or view has a lazy scrollable component, such as a Compose LazyColumn, it may be desirable to capture a screenshot of the entire lazy contents. However, because it is a lazy layout, the size of that content cannot be known before rendering it (e.g. measuring it with unconstrained height will throw).
Nothing about this problem or proposed solution is inherent to Compose, although since it's so easy to use lazy lists in Compose that it tends to come up quite a lot there.
Proposal
API level 31 introduced a new platform feature called "scrolling screenshots" for taking screenshots of apps that have lots of scrollable content. Paparazzi could use this same API to capture scrollable content fully without having to guess at its size. This would work for both Views and Compose.
The platform APIs allow finding scroll containers and asking them to render an arbitrary amount of their contents into a separate set of buffers, one viewport-or-smaller-sized-region at a time. Support for this feature was added to Compose lazy lists in 1.7.
Implementation
The Compose support for this includes a copy of the system implementation that drives the API entirely from a normal instrumented UI tests (source). Paparazzi could use a similar implementation to generate screenshots.
API
I haven't thought much about the Paparazzi API. It could be as simple as another set of methods on Paparazzi:
fun scrollingScreenshot(
view: View,
name: String? = null,
orientation: Orientation = Vertical
)
fun scrollingScreenshot(
name: String? = null,
orientation: Orientation = Vertical,
content: @Composable () -> Unit
)The configuration for these would look different than normal screenshots. The size of the screenshot in the requested orientation would be determined by the content, so only the cross-axis dimension would be taken from the device config. GIFs would not be supported. Long screenshots can take multiple full device frames to render, so animations would need to be disabled.
The API allows a bunch of candidate scroll containers to be reported, and the system uses heuristics to choose the best one to capture. The compose testing helper has similar logic, which Paparazzi might also need. However, since Paparazzi tests can be of individual components, there probably doesn't need to be a very complicated mechanism for customizing that search behavior, since the test can just be written around a specific component.
Blockers
The scroll capture APIs don't work on Canvas, they require working directly with Surface to synchronize and be able to send images across processes efficiently. Both the platform and compose testing drivers of this API use the platform API ImageReader to get a Surface they can read images back from. layoutlib does not seem to currently support ImageReader. I hope it is possible for layoutlib to implement support for ImageReader, but I don't think it's something that can be done in Paparazzi.
ImageReader is high-level though, can we drop to something lower? I don't think so. Paparazzi uses a special layoutlib implementation of HardwareRenderer to capture views. All the code it uses to work with surfaces is private.