Our CTO Johan Risch has begun a series of posts that provide a look "under the hood" of Stream Analyze's solution for Edge AI. Our runtime is incredibly powerful for product owners, engineering managers and the data scientists working to build and optimize AI for the Edge. See his first installment here:
_____________________________________________________________________________________________________
In 2016 a manufacturer asked me if we could run neural nets. Ten years and one detour later, here's the answer: any supported ONNX model becomes a query. Stream Analyze is shipping this to our customers this month.
Back then I was barely two years out of university. I built Stream Analyze Neural Network runtime (SA.NN) to answer that question. Half a year later came the next one: can you import TensorFlow models? So I wrote a TensorFlow to SA.NN translator. It worked for many models. Then ONNX was announced in late 2017. It was no sure bet it would become the standard, so we stayed with TensorFlow.
Fast forward to 2026. A few very good conversations about running models in new environments, regardless of where they were trained, got me thinking about ONNX again.
The hard part was never the translator. It was having a target worth translating to: a patent-pending way of expressing linear algebra over dense arrays inside a query language, running on everything from microcontrollers to GPUs. That took years.
Once it existed, and with ONNX shipping an extensive conformance test suite, the translator became exactly the kind of problem LLMs are good at. I pointed Fable at it. We now pass 96%+ of the ONNX Runtime tests.
And it's not a toy. To be clear about what we do: we don't optimize model architectures or quantize models. Plenty of good companies do that.
We take the model you already have and make it one part of a persistent analytical runtime, alongside live data, signal processing, state, logic and other models.
The model isn't the application. It's an operator inside the application.
And because the whole thing is expressed as a query, you can change that analytical behavior independently of the surrounding firmware and software.
In 2017 I bet on TensorFlow and ONNX won, so am I making the same mistake again? I don't think so. ONNX has become one of the industry's standard interchange formats.
And this time the bet is smaller: if the industry moves on, to ExecuTorch, LiteRT or something we haven't seen yet, we write another translator into the same target.
That's the whole point of the target. What would you bet on?
(That query language can also be swapped for Python. More on that later.) Check out a tour of key components on our Under the Hood page.
#StreamAnalyze #edgeAnalytics #edgeAI #ONNX