lumen-ai/com.subhashh.lumen.ai/OnnxEmbedder

OnnxEmbedder

class OnnxEmbedder(dir: File, spec: OnnxSpec, threads: Int, gpu: Boolean = false) : Embedder(source)

A CLIP-style model (two encoders, one shared space). Text always runs on ONNX Runtime's CPU provider. The image encoder does too, unless OnnxSpec.visualFile is a LiteRT .tflite: then it runs on the GPU when gpu is set (only after GpuCheck passed on this phone) and on LiteRT's CPU path otherwise. dir holds the files named in spec.

Constructors

constructor(dir: File, spec: OnnxSpec, threads: Int, gpu: Boolean = false)

Types

object Companion

Properties

open override val dim: Int

Functions

open override fun close()

open override fun embedImage(bitmap: Bitmap): FloatArray

L2-normalized, so a dot product is the cosine similarity.

open override fun embedText(query: String): FloatArray

fun pixels(original: Bitmap): FloatArray

Resize (crop or squash), then CHW floats normalized with the model's mean and std.

Loads the text model and runs it once: its first run sets up memory and is slow.