[camera] add video stabilization

[ruicraveiro]

Adds support for video stabilization to camera_platform_interface, camera_avfoundation, camera_android_camerax and camera packages.

The video stabilization modes are defined in the new VideoStabilizationMode enum defined in camera_platform_interface:

/// The possible video stabilization modes that can be capturing video.
enum VideoStabilizationMode {
  /// Video stabilization is disabled.
  off,

  /// Basic video stabilization is enabled.
  /// Maps to CONTROL_VIDEO_STABILIZATION_MODE_ON on Android
  /// and throws CameraException on iOS.
  on,

  /// Standard video stabilization is enabled.
  /// Maps to CONTROL_VIDEO_STABILIZATION_MODE_PREVIEW_STABILIZATION on Android
  /// (camera_android_camerax) and to AVCaptureVideoStabilizationModeStandard
  /// on iOS.
  standard,

  /// Cinematic video stabilization is enabled.
  /// Maps to CONTROL_VIDEO_STABILIZATION_MODE_PREVIEW_STABILIZATION on Android
  /// (camera_android_camerax) and to AVCaptureVideoStabilizationModeCinematic
  /// on iOS.
  cinematic,

  /// Extended cinematic video stabilization is enabled.
  /// Maps to AVCaptureVideoStabilizationModeCinematicExtended on iOS and
  /// throws CameraException on Android.
  cinematicExtended,
}

There is some subjectivity on the way with which I mapped the modes to both platforms, and here's a document that compares the several modes: https://docs.google.com/spreadsheets/d/1TLOLZHR5AcyPlr-y75aN-DbR0ssZLJjpV_OAJkRC1FI/edit?usp=sharing, which you can comment on.

List which issues are fixed by this PR. You must list at least one issue.
Partially implements flutter/flutter#89525

Pre-launch Checklist

• I read the Contributor Guide and followed the process outlined there for submitting PRs.
• I read the Tree Hygiene page, which explains my responsibilities.
• I read and followed the relevant style guides and ran the auto-formatter. (Unlike the flutter/flutter repo, the flutter/packages repo does use dart format.)
• I signed the CLA.
• The title of the PR starts with the name of the package surrounded by square brackets, e.g. [shared_preferences]
• I linked to at least one issue that this PR fixes in the description above.
• I updated pubspec.yaml with an appropriate new version according to the pub versioning philosophy, or this PR is exempt from version changes.
• I updated CHANGELOG.md to add a description of the change, following repository CHANGELOG style, or this PR is exempt from CHANGELOG changes.
• I updated/added relevant documentation (doc comments with ///).
• I added new tests to check the change I am making, or this PR is test-exempt.
• All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel on Discord.

[google-cla]

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

[hellohuanlin]

only reviewed iOS part

[ruicraveiro]

Hi @hellohuanlin,

I have improved camera_avfoundation's implementation based on your comments. Instead of adding a new commit, I forced pushed a squash commit of those changes and another commit on top of it that adds the dependency_overrides to the pubspecs.

Assuming there will be more improvments I need to make, is it OK I keep adding commits until everything is fixed, and only then do a final squash merge, excluding the dependency_overrides commit, and force that squash commit?

Thanks!

[ruicraveiro]

Hi @stuartmorgan, I have just pushed an update to this branch with the following changes:

• At the platform levels, camera_android_camerax and camera_avfoundation, if setVideoStabilizationMode() is called with a mode that is unavailable, then that method with throw an ArgumentError. I also figured a cleaner way to map between VideoStabilizationMode and the platform's specific representation of the mode. Now I am mapping both ways using a single method, which in both platforms is called _getSupportedVideoStabilizationModeMap(). I also fixed a couple of nits picked up by @hellohuanlin.

• Because it is a common feature with a common logic, I implemented the fallback logic in CameraController, in the camera package, for setVideoStabilizationMode() method, as specified in my message on the 20th December. When allowFallback is true, then CameraController will only call the platform's implementation with a supported video stabilization mode and will throw ArgumentException if none is available (or if more than off is asked for and only off is available). When allowFallback is false, then CameraController passes the mode as is to the platform's implementation.

I have some broken tests, but I don't think they're related to anything I did, except merging the main branch onto this one. On my machine, all Dart unit tests related to this package are OK. Native tests are indeed failing, but they're also failing when I run them on the latest version of the main branch (plus I'm getting failed Dart tests for other packages on both this branch and main). Except for the very tiny nits (bool -> BOOL and a couple of empty lines removed), all changes were made to Dart code, not native code.

[leooruiz]

Hello @ruicraveiro,

Thank you for your work on this stabilization PR. I tested the update on my sports recording app and encountered a compilation error on iOS. Below is the error message from the console.

Error log:

Launching lib/main.dart on iPhone 11 in release mode...
Automatically signing iOS for device deployment using specified development team in Xcode project: H9UAZA7PV4
Xcode build done.                                           59.5s
Failed to build iOS app
Could not build the precompiled application for the device.
ARC Semantic Issue (Xcode): No visible @interface for 'NSObject<FLTCaptureDeviceFormat>' declares the selector 'isVideoStabilizationModeSupported:'
/Users/leo_ruiz/repositories/xports/flutter_packages/packages/camera/camera_avfoundation/ios/camera_avfoundation/Sources/camera_avfoundation/FLTCam.m:1246:36

ARC Semantic Issue (Xcode): No visible @interface for 'NSObject<FLTCaptureDeviceFormat>' declares the selector 'isVideoStabilizationModeSupported:'
/Users/leo_ruiz/repositories/xports/flutter_packages/packages/camera/camera_avfoundation/ios/camera_avfoundation/Sources/camera_avfoundation/FLTCam.m:1262:38
2

Error running application on iPhone 11.

Exited (1).

Before compiling, I ran flutter clean and flutter pub get.

Environment details:

• Flutter: 3.27.1 (stable channel)
• Dart: 3.6.0
• macOS: Version 15.2 (M1 Pro)
• Xcode: 16.2
• Device: iPhone 11 running iOS 18.3.2

Flutter doctor output:

❯ flutter doctor -v
[✓] Flutter (Channel stable, 3.27.1, on macOS 15.2 24C101 darwin-arm64, locale en-BR)
    • Flutter version 3.27.1 on channel stable at /Users/leo_ruiz/.asdf/installs/flutter/3.27.1
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 17025dd882 (3 months ago), 2024-12-17 03:23:09 +0900
    • Engine revision cb4b5fff73
    • Dart version 3.6.0
    • DevTools version 2.40.2

[✓] Android toolchain - develop for Android devices (Android SDK version 35.0.0)
    • Android SDK at /Users/leo_ruiz/Library/Android/sdk
    • Platform android-35, build-tools 35.0.0
    • Java binary at: /Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-17.0.6b802.4-9586694)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 16.2)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 16C5032a
    • CocoaPods version 1.16.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2022.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-17.0.6b802.4-9586694)

[✓] VS Code (version 1.98.2)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.106.0

[✓] Connected device (5 available)
    • SM A055M (mobile)               • R9XX60289MX               • android-arm64  • Android 14 (API 34)
    • iPhone 11 (mobile)     • 00008030-0014059A1ED0402E • ios            • iOS 18.3.2 22D82
    • macOS (desktop)                 • macos                     • darwin-arm64   • macOS 15.2 24C101 darwin-arm64
    • Mac Designed for iPad (desktop) • mac-designed-for-ipad     • darwin         • macOS 15.2 24C101 darwin-arm64
    • Chrome (web)                    • chrome                    • web-javascript • Google Chrome 134.0.6998.166
    ! Error: Browsing on the local area network for iPad. Ensure the device is unlocked and attached with a cable or associated with the same local area network as this Mac.
      The device must be opted into Developer Mode to connect wirelessly. (code -27)

[✓] Network resources
    • All expected network resources are available.

• No issues found!

If you need any further information or additional testing on my side, please let me know. Thanks again for your support and the excellent work on this PR!

[stuartmorgan-g]

I have some broken tests, but I don't think they're related to anything I did, except merging the main branch onto this one. On my machine, all Dart unit tests related to this package are OK. Native tests are indeed failing, but they're also failing when I run them on the latest version of the main branch

I'm not sure why you would be getting failures on main; we run the tests continuously in CI. But the compile failures in the native tests here are related to the changes in the PR; likely the PR just needs to be updated to add the methods you are calling to the wrappers that were recently added as part of refactoring of the camera implementation to increase testability and allow for a migration to Swift (where we can't use OCMock).

[stuartmorgan-g]

The app-level fallback logic looks good (modulo questions about how off is managed in those APIs), mostly just minor comments before this can be split into sub-PRs.

[stuartmorgan-g]

The comment should not be calling out specific platforms; the restriction should have the same behavior on all platforms.

[ruicraveiro]

I think you're right and the comment is only creating confusion. Will remove it.

[stuartmorgan-g]

This is only true for allowFallback=false.

[ruicraveiro]

I will also remove it, especially since after that comment, I document both allowFallback cases.

[stuartmorgan-g]

This makes it sound like getSupportedVideoStabilizationModes can return off, which I believe we agreed should not happen. If it can't, then we can avoid confusion by saying "... or getSupportedVideoStabilizationModes() returns an empty list".

[ruicraveiro]

Three separate cases:

• When the device has no camera stabilization modes. I floated the idea that we could return off, but I agreed with you not to.

• When the camera does support some video stabilization mode other than off. Off was always part of the available options, so much so that is one of the options in both tables in my latest specification proposal, in the comment I made on the 20th December. Furthermore, it is one of the options returned by both Android and iOS.

• When the only option is 'off'. This is the scenario represented by the third column in those specification tables. Basically what it means is that even if allowFallback is true, if the only available mode is off and something else is specified, then it is an error.

Here's the table again:

requested mode	no supported mode ([])	only off supported	max level1 supported	max level2 supported	max level3 supported
off	throw ArgumentError()	off	off	off	off
level1	throw ArgumentError()	throw ArgumentError()	level1	level1	level1
level2	throw ArgumentError()	throw ArgumentError()	level1	level2	level2
level3	throw ArgumentError()	throw ArgumentError()	level1	level2	level 3

Let me know if you still have a different understanding. For me it is very clear that if level1, 2 or 3 are available, there is no reason to exclude off. It is also very clear, and I agreed with you, that we shouldn't introduce off artificially if the device doesn't return any available mode. What is really hard to define is the behaviour of when off is the only mode, even more so because I think that this is a theoretical discussion as I don't expect the case of where the platform returns only off to ever happen. None of the devices I tested showed that behaviour, as they would either report no supported stabilization mode or at least 2 supported modes, one of them always being off.

So, maybe, because it isn't really something expected to happen, I can simplify the comment and ignore that case. Let me know what you think.

[stuartmorgan-g]

Three separate cases:

• When the device has no camera stabilization modes. [...]
  [...]
• When the only option is 'off'. [...]

I'm not following. How are these separate cases? What is the conceptual difference here?

[stuartmorgan-g]

Nit: capitalize as a sentence.

[stuartmorgan-g]

Nit: mode

[ruicraveiro]

What are you referring to? The comment ... and something else ... (where it would be some other mode)? Is it the 'sm' variable, which stands for 'supportedMode'? Or something else?

[stuartmorgan-g]

Sorry, the variable sm.

[stuartmorgan-g]

You'll need to run the repo tooling's format command on the package to auto-format everything.

[stuartmorgan-g]

Same naming nit here: supportsVideoStabilizationMode:

[stuartmorgan-g]

The changes to this file should be reverted; if there are sill third-party implementations using this (hopefully not), they should get the default behavior of nothing being supported, rather than throwing unsupported platform channel method exceptions.

[stuartmorgan-g]

It will be Future<VideoStabilizationMode> setTargetVideoStabilizationMode(VideoStabilizationMode mode) and it will return, not the requested mode, but the actual mode that was selected. The returned value will be the same as the one that will be returned by Controller.value.videoStabilizationMode.

Sounds good; returning the actual value from a best-effort-request method is definitely useful, and something more of our APIs should do in general.

What happened to this design? It was a good suggestion, but it's not in the updated implementation.

[stuartmorgan-g]

This needs to return an empty list, so that the default behavior for implementations that don't support stabilization is gracefully reporting that, rather than throwing.

[FabioCFonseca]

Hi @ruicraveiro,

I tested this PR on Android, and it worked fine; the crop indicates stabilization is being applied.

I’m just unsure about the differences among VideoStabilizationMode levels 1, 2, and 3, as I couldn’t visually distinguish any variations...

Enviroment:

Device: Motorola Edge 20 - Android 13

Doctor summary (to see all details, run flutter doctor -v):
[✓] Flutter (Channel stable, 3.27.1, on macOS 14.4.1 23E224 darwin-arm64, locale en-BR)
[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
[✓] Xcode - develop for iOS and macOS (Xcode 15.4)
[✓] Chrome - develop for the web
[✓] Android Studio (version 2023.3)
[✓] VS Code (version 1.97.0)
[✓] Connected device (4 available)
[✓] Network resources

[stuartmorgan-g]

I tested this PR on Android, and it worked fine; the crop indicates stabilization is being applied.

I’m just unsure about the differences among VideoStabilizationMode levels 1, 2, and 3, as I couldn’t visually distinguish any variations...

Android only supports level 1.

[romainfd]

Hi, thank you and congratulations @ruicraveiro (and @stuartmorgan-g) for the amazing work on this! I am very interested in this feature, is there any way for me to help you progress on it?

[bparrishMines]

@ruicraveiro It looks like the camera_android_camerax ProxyApi update broke your changes to that plugin. If you are still working on this, let me know if you need help updating that portion or I can also quickly update the API wrapper for you.

[harryterkelsen]

Please make the change Stuart requested of not throwing UnimplementedError in the platform interface and having each platform gracefully indicate whether or not they support video stabilization.

[gyenesvi]

Hi @ruicraveiro, is this still ongoing work or is it stalled? Any ideas how much work is required / when this could become available in Flutter? Would be nice to understand if this feature is worth waiting for, as video recording in its current form is not very useful for my purposes. Thanks!

[stuartmorgan-g]

Thank you for your contribution. Since there are outstanding comments but the PR hasn’t been updated in several months, I’m going to close it so that our PR queue reflects active PRs. Please don't hesitate to submit a new PR if you have the time to address the review comments. Thanks!

[stuartmorgan-g]

@romainfd, @gyenesvi Anyone is welcome to make their own forks of PRs that are no longer being worked on, and open new PRs to complete the work.

[romainfd]

Great, thank you for clarifying things @stuartmorgan-g! I just contacted @gyenesvi to make sure we don't both work on it at the same time but, unless he wants to do it before, I should be able to give it a try late August or in September.

[ruicraveiro]

@romainfd No, I didn't mean to work on this; I was just curious about the status/progress of this feature, whether there's anything blocking it or there's a chance that it's going to happen anytime soon. My understanding is that this feature is available on the native platforms in some form, so it's mostly a matter of enabling this setting from the flutter side. From the comments it seemed that the bulk of it had already been implemented, and the past year was spent with discussing naming and default value issues. Is that right? Or are there more significant bits missing?

This feature seems like a pretty basic/necessary addition to the package, and flutter developers have been complaining about its non-existence for several years, as this puts flutter based camera apps to a big disadvantage compared to native apps. In my case, the video I can record with my flutter app using this package looks like videos from a decade ago compared to videos my iphone (12) is actually capable of recording, which kind of invalidates the whole point of my app. So I'd like to know if this whole path is viable for my app.

Do you guys know why @ruicraveiro stopped working on this / responding to requests after the initial commits?

Hi, I didn't exactly quit the PR, but I haven't found the time to try yet again to get it approved. However, since @stuartmorgan-g closed the PR, that determines that I am done with it. I would be grateful, though, if someone has the patience and energy to see this through.

[stuartmorgan-g]

However, since @stuartmorgan-g closed the PR, that determines that I am done with it.

I closed is as part of our standard PR triage process; if we didn't close PRs that were inactive for months we would have have hundreds of abandoned PRs to go through on a regular basis, which isn't a good use of time, and would make it much harder to make sure active PRs are not falling through the cracks.

If you want to pick this back up again later and would rather not open a new PR referencing this one, please feel free to leave a comment here and I'd be happy to re-open it.

[gyenesvi]

Hi, I didn't exactly quit the PR, but I haven't found the time to try yet again to get it approved. However, since @stuartmorgan-g closed the PR, that determines that I am done with it. I would be grateful, though, if someone has the patience and energy to see this through.

Thanks for the heads up anyways, I was kind of afraid that this would be the case, though it seemed to have been close to completion. Let's hope @romainfd has the knowledge and time to pick this up and finish it.