Skip to content

update video player to media3-exoplayer and improve caching #403

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jun 24, 2025
Merged

update video player to media3-exoplayer and improve caching #403

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
cc5658f
update to media3-exoplayer 1.7.1
michaelknoch Jun 23, 2025
90d48af
cleanup
michaelknoch Jun 23, 2025
36ec9a2
improve caching
michaelknoch Jun 24, 2025
98490c5
cleanup
michaelknoch Jun 24, 2025
bdef046
cleanup
michaelknoch Jun 24, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion build.gradle
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,11 @@ android {

dependencies {
implementation "org.jetbrains.kotlin:kotlin-stdlib:$kotlin_version"
implementation "com.google.android.exoplayer:exoplayer:2.13.3"
implementation "androidx.media3:media3-exoplayer:1.7.1"
implementation "androidx.media3:media3-ui:1.7.1"
implementation "androidx.media3:media3-database:1.7.1"
implementation "androidx.media3:media3-datasource-okhttp:1.7.1"
implementation "com.squareup.okhttp3:okhttp:4.12.0"
}

repositories {
Expand Down
258 changes: 140 additions & 118 deletions src/main/java/org/uikit/VideoJNI.kt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,48 +2,38 @@ package org.uikit

import android.content.Context
import android.net.Uri
import android.util.Log
import android.widget.RelativeLayout
import com.google.android.exoplayer2.*
import com.google.android.exoplayer2.database.ExoDatabaseProvider
import com.google.android.exoplayer2.source.ProgressiveMediaSource
import com.google.android.exoplayer2.trackselection.DefaultTrackSelector
import com.google.android.exoplayer2.ui.PlayerView
import com.google.android.exoplayer2.upstream.*
import com.google.android.exoplayer2.upstream.cache.CacheDataSink
import com.google.android.exoplayer2.upstream.cache.CacheDataSource
import com.google.android.exoplayer2.upstream.cache.LeastRecentlyUsedCacheEvictor
import com.google.android.exoplayer2.upstream.cache.SimpleCache
import android.util.Log
import androidx.media3.common.MediaItem
import androidx.media3.common.PlaybackException
import androidx.media3.common.PlaybackParameters
import androidx.media3.common.Player
import androidx.media3.database.ExoDatabaseProvider
import androidx.media3.datasource.DataSource
import androidx.media3.datasource.FileDataSource
import androidx.media3.datasource.cache.CacheDataSink
import androidx.media3.datasource.cache.CacheDataSource
import androidx.media3.datasource.cache.LeastRecentlyUsedCacheEvictor
import androidx.media3.datasource.cache.SimpleCache
import androidx.media3.datasource.okhttp.OkHttpDataSource
import androidx.media3.exoplayer.ExoPlayer
import androidx.media3.exoplayer.SeekParameters
import androidx.media3.exoplayer.source.ProgressiveMediaSource
import androidx.media3.exoplayer.trackselection.DefaultTrackSelector
import androidx.media3.exoplayer.upstream.DefaultBandwidthMeter
import androidx.media3.ui.PlayerView
import org.libsdl.app.SDLActivity
import okhttp3.Cache as OkHttpCache
import okhttp3.OkHttpClient
import okhttp3.Protocol
import java.io.File
import java.util.concurrent.TimeUnit
import kotlin.math.absoluteValue


@Suppress("unused")
class AVURLAsset(parent: SDLActivity, url: String) {
internal val videoSource: ProgressiveMediaSource
private val context: Context = parent.context

init {
val mediaItem = MediaItem.Builder().setUri(Uri.parse(url)).build()

// ExtractorMediaSource works for regular media files such as mp4, webm, mkv
val cacheDataSourceFactory = CacheDataSourceFactory(
context,
512 * 1024 * 1024,
64 * 1024 * 1024
)

videoSource = ProgressiveMediaSource
.Factory(cacheDataSourceFactory)
.createMediaSource(mediaItem)
}
}

@Suppress("unused")
class AVPlayer(parent: SDLActivity, asset: AVURLAsset) {
internal val exoPlayer: SimpleExoPlayer
private var listener: Player.EventListener
internal val exoPlayer: ExoPlayer
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok I'm confused. there's media3 which contains ExoPlayer now? I thought they just renamed it 😵

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Jetpack Media3’s ExoPlayer module is still ExoPlayer, it's just repackaged under the AndroidX Media3 umbrella (whatever androidx is, lol)

private val listener: Player.Listener

external fun nativeOnVideoReady()
external fun nativeOnVideoEnded()
Expand All @@ -54,90 +44,83 @@ class AVPlayer(parent: SDLActivity, asset: AVURLAsset) {
val bandwidthMeter = DefaultBandwidthMeter.Builder(parent.context).build()
val trackSelector = DefaultTrackSelector(parent.context)

exoPlayer = SimpleExoPlayer.Builder(parent.context)
.setBandwidthMeter(bandwidthMeter)
.setTrackSelector(trackSelector)
.build()
exoPlayer.prepare()
exoPlayer.setMediaSource(asset.videoSource)
exoPlayer = ExoPlayer.Builder(parent.context)
.setBandwidthMeter(bandwidthMeter)
.setTrackSelector(trackSelector)
.build().apply {
setMediaSource(asset.mediaSource)
prepare()
}

listener = object: Player.EventListener {
override fun onPlayerStateChanged(playWhenReady: Boolean, playbackState: Int) {
when (playbackState) {
listener = object : Player.Listener {
override fun onPlaybackStateChanged(state: Int) {
when (state) {
Player.STATE_READY -> nativeOnVideoReady()
Player.STATE_ENDED -> nativeOnVideoEnded()
Player.STATE_BUFFERING -> nativeOnVideoBuffering()
else -> {}
Player.STATE_ENDED -> nativeOnVideoEnded()
}
}

override fun onSeekProcessed() {
isSeeking = false
if (desiredSeekPosition != getCurrentTimeInMilliseconds()) {
seekToTimeInMilliseconds(desiredSeekPosition)
override fun onPositionDiscontinuity(
oldPosition: Player.PositionInfo,
newPosition: Player.PositionInfo,
reason: Int
) {
if (reason == Player.DISCONTINUITY_REASON_SEEK) {
isSeeking = false
if (desiredSeekPosition != exoPlayer.currentPosition) {
seekToTimeInMilliseconds(desiredSeekPosition)
}
}
}

override fun onPlayerError(error: ExoPlaybackException) {
Log.e("SDL", "ExoPlaybackException occurred")
val message = error.message ?: "N/A"
nativeOnVideoError(error.type, message)
override fun onPlayerError(error: PlaybackException) {
Log.e("SDL", "PlaybackException: ${'$'}{error.errorCodeName}")
nativeOnVideoError(error.errorCode, error.message ?: "N/A")
}
}

exoPlayer.addListener(listener)
}

fun play() {
// ExoPlayer API to play the video
exoPlayer.playWhenReady = true
}

fun pause() {
// ExoPlayer API to pause the video
exoPlayer.playWhenReady = false
}

fun setVolume(newVolume: Double) {
exoPlayer.volume = newVolume.toFloat()
}

fun getCurrentTimeInMilliseconds(): Long {
return exoPlayer.currentPosition
fun getCurrentTimeInMilliseconds(): Long = exoPlayer.currentPosition
fun getPlaybackRate(): Float = exoPlayer.playbackParameters.speed
fun setPlaybackRate(rate: Float) {
exoPlayer.setPlaybackParameters(PlaybackParameters(rate, 1.0f))
}


private var isSeeking = false
private var desiredSeekPosition: Long = 0
private var lastSeekedToTime: Long = 0

private fun seekToTimeInMilliseconds(timeInMilliseconds: Long) {
desiredSeekPosition = timeInMilliseconds
private fun seekToTimeInMilliseconds(timeMs: Long) {
desiredSeekPosition = timeMs

// This *should* mean we don't always scroll to the last position provided.
// In practice we always seem to be at the position we want anyway:
if (isSeeking) { return }

if (isSeeking) return

val delta = (desiredSeekPosition - lastSeekedToTime).absoluteValue

// Seeking to the exact millisecond is very processor intensive (and SLOW!)
// Only do put that effort in if we're scrubbing very slowly over a short time period:
val syncParameters = if ((desiredSeekPosition - lastSeekedToTime).absoluteValue < 250) {
SeekParameters.EXACT
} else {
SeekParameters.CLOSEST_SYNC
}

val seekParameters = if (delta < 150) SeekParameters.EXACT else SeekParameters.CLOSEST_SYNC
isSeeking = true
exoPlayer.setSeekParameters(syncParameters)
exoPlayer.seekTo(timeInMilliseconds)
lastSeekedToTime = timeInMilliseconds
}

fun getPlaybackRate(): Float {
return exoPlayer.playbackParameters.speed
}

fun setPlaybackRate(rate: Float) {
exoPlayer.setPlaybackParameters(PlaybackParameters(rate, 1.0F))
exoPlayer.setSeekParameters(seekParameters)
exoPlayer.seekTo(timeMs)
lastSeekedToTime = timeMs
}

fun cleanup() {
Expand All @@ -148,65 +131,104 @@ class AVPlayer(parent: SDLActivity, asset: AVURLAsset) {

@Suppress("unused")
class AVPlayerLayer(private val parent: SDLActivity, player: AVPlayer) {
private var exoPlayerLayout: PlayerView
private val exoPlayerView: PlayerView = PlayerView(parent.context).apply {
useController = false
tag = "ExoPlayer"
this.player = player.exoPlayer
}

init {
val context = parent.context

exoPlayerLayout = PlayerView(context)
exoPlayerLayout.player = player.exoPlayer
exoPlayerLayout.useController = false
exoPlayerLayout.tag = "ExoPlayer"
parent.addView(exoPlayerLayout, 0)
parent.addView(exoPlayerView, 0)
}

fun setFrame(x: Int, y: Int, width: Int, height: Int) {
val layoutParams = RelativeLayout.LayoutParams(width, height)
layoutParams.setMargins(x, y, 0, 0)
exoPlayerLayout.layoutParams = layoutParams
exoPlayerView.layoutParams = RelativeLayout.LayoutParams(width, height).also {
it.setMargins(x, y, 0, 0)
}
}

fun setResizeMode(resizeMode: Int) {
exoPlayerLayout.resizeMode = resizeMode
exoPlayerView.resizeMode = resizeMode
}

fun removeFromParent() {
Log.v("SDL", "Removing video from parent layout")
parent.removeViewInLayout(exoPlayerLayout)
}
fun removeFromParent() = parent.removeViewInLayout(exoPlayerView)
}

/**
* Data-source factory that reuses the singleton cache and HTTP client.
*/
internal class CacheDataSourceFactory(private val maxFileSize: Long): DataSource.Factory {
private val upstreamFactory = OkHttpDataSource.Factory(Media3Singleton.okHttpClient)
private val simpleCache = Media3Singleton.simpleCache

override fun createDataSource(): DataSource =
CacheDataSource(
simpleCache,
upstreamFactory.createDataSource(),
FileDataSource(),
CacheDataSink(simpleCache, maxFileSize),
CacheDataSource.FLAG_BLOCK_ON_CACHE or CacheDataSource.FLAG_IGNORE_CACHE_ON_ERROR,
null
)
}


/**
* Singleton holder for shared OkHttpClient (HTTP/2) and ExoPlayer disk cache.
*/
object Media3Singleton {
private var initialized = false

lateinit var okHttpClient: OkHttpClient
private set

///// Caching data source
// Thank you https://stackoverflow.com/a/45488510/3086440
lateinit var simpleCache: SimpleCache
private set

private class CacheDataSourceFactory(private val context: Context, private val maxCacheSize: Long, private val maxFileSize: Long) : DataSource.Factory {
/**
* Initialize once with application context and cache sizes.
*/
fun init(context: Context, httpCacheSize: Long, mediaCacheSize: Long) {
if (initialized) return
initialized = true

private val defaultDatasourceFactory: DefaultDataSourceFactory
// 1. Shared OkHttpClient with HTTP/2 support and disk cache
val httpCacheDir = File(context.cacheDir, "http_http2_cache")
val okCache = OkHttpCache(httpCacheDir, httpCacheSize)
okHttpClient = OkHttpClient.Builder()
.cache(okCache)
.protocols(listOf(Protocol.HTTP_2, Protocol.HTTP_1_1))
.build()

// The cache survives the application lifetime, otherwise the cache keys can get confused
companion object {
var simpleCache: SimpleCache? = null
// 2. Shared ExoPlayer disk cache (LRU evictor)
val mediaCacheDir = File(context.cacheDir, "media")
val evictor = LeastRecentlyUsedCacheEvictor(mediaCacheSize)
simpleCache = SimpleCache(mediaCacheDir, evictor, ExoDatabaseProvider(context))
}
}

@Suppress("unused")
class AVURLAsset(parent: SDLActivity, url: String) {
internal val mediaSource: ProgressiveMediaSource
private val context: Context = parent.context

init {
val bandwidthMeter = DefaultBandwidthMeter.Builder(context).build()
defaultDatasourceFactory = DefaultDataSourceFactory(this.context,
bandwidthMeter, DefaultHttpDataSource.Factory())
}
Media3Singleton.init(
context = context,

override fun createDataSource(): DataSource {
if (simpleCache == null) {
val cacheEvictor = LeastRecentlyUsedCacheEvictor(maxCacheSize)
val dataBaseProvider = ExoDatabaseProvider(context)
val file = File(context.cacheDir, "media")
simpleCache = SimpleCache(file, cacheEvictor, dataBaseProvider)
}
// the http cache holds HTTP responses/validators (ETags, headers, small bodies),
// so we get fast 304s and header compression.
httpCacheSize = 20L * 1024 * 1024,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you know what the difference is here? why are they different sizes?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • httpCacheSize (20 MiB) is for OkHttp’s disk cache of HTTP responses/validators (ETags, headers, small bodies), so we get fast 304s and header compression—but we don’t store full video ranges there.

  • mediaCacheSize (512 MiB) is ExoPlayer’s SimpleCache, which actually holds the raw MP4 bytes

I just restructured the initialization and size definitions a bit to make it easier to follow

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok sounds good to me! thanks. might be worth leaving a code comment with some of that info


return CacheDataSource(simpleCache!!, defaultDatasourceFactory.createDataSource(),
FileDataSource(), CacheDataSink(simpleCache!!, maxFileSize),
CacheDataSource.FLAG_BLOCK_ON_CACHE or CacheDataSource.FLAG_IGNORE_CACHE_ON_ERROR, null)
// this cache actually holds the raw MP4 bytes
mediaCacheSize = 512L * 1024 * 1024
)

val mediaItem = MediaItem.fromUri(Uri.parse(url))
val cacheFactory = CacheDataSourceFactory(
maxFileSize = 256L * 1024 * 1024
)
mediaSource = ProgressiveMediaSource.Factory(cacheFactory)
.createMediaSource(mediaItem)
}
}
}