lumen-ai/com.subhashh.lumen.ai/MediaFrames

MediaFrames

The stills Smart Search looks at. A photo: Android's cached thumbnail. A video: one frame a second up to two minutes (exact moments, not just keyframes), then MaxFrames spread evenly, so a long video costs about what a two-minute one does. Frames that look like the last one kept are skipped before the model ever sees them: a talking head or a still shot doesn't fill the index with copies.

Functions

forEach

suspend fun forEach(context: Context, item: MediaRef, px: Int = Px, skipSame: Boolean = true, perSecond: Int = 1, onFrame: suspend (atMs: Long, frame: Bitmap) -> Boolean)

Calls onFrame with each frame's time (0 for a photo) and pixels, at most px on their long side; stops if it returns false. skipSame: leave out frames that look like the last one (Smart Search, faces); the text pass wants them all, since a new caption over the same shot barely changes a small thumbnail, and it skips unchanged lines itself.

forReading

fun forReading(context: Context, item: MediaRef, maxSide: Int): Bitmap

A photo at up to maxSide on its long side, for reading its text: thumbnails are too small for chat-sized letters. Software pixels, first frame of an animation.

thumbnail

fun thumbnail(context: Context, item: MediaRef): Bitmap

A photo's one still: Android's cached thumbnail.

times

fun times(durationMs: Long, perSecond: Int = 1): List<Long>

Frame times in ms: every second for short videos, else evenly spread mid-segment points.

videoFrameAt

fun videoFrameAt(context: Context, item: MediaRef, atMs: Long, maxSide: Int): Bitmap?

The video frame showing at atMs (the exact one, not the nearest keyframe), at most maxSide on its long side. Plain videos and the host's own (LumenAiHost.videoSource) alike; HDR comes out tone mapped, which reads fine.