Put the thinking next to the sensor, and know where to stop

A sensor turns the world into numbers. The easy habit is to send those numbers somewhere else to be understood. For some devices that is the right answer. For many it is not, and the first reason is energy.

A device that runs on a small battery, or on power it harvests, has a budget. Sending data out spends it. Every raw sample streamed over a radio is energy not available for sensing. If the device computes a small answer where the data is born and sends only that answer, the bill changes. I hold this as a design principle, not as a measurement. The balance depends on the radio, the chip and the model, and it should be worked out for each device.

Here is what that looks like in practice, in three pieces of my group’s work, listed at the end.

Electrochemical sensors in real samples are crowded. In cerebrospinal fluid, uric acid and ascorbic acid respond alongside the neurotransmitter you want. In our work on embedded machine learning for dopamine detection [1], a small model running on a custom potentiostat learned to discriminate that interference, with an overall accuracy of 98.1% in floating point and 96.01% after being squeezed to 8-bit numbers. It used 4.55% of the potentiostat’s memory. The squeeze cost about two points of accuracy and bought a model small enough to live on the device. That trade of memory against accuracy is the real design problem, and we found it can be tuned.

Our work on cuffless blood pressure estimation from an optical pulse signal [2] asks a question that most deep-learning models for this skip: will it fit on the hardware it is meant to run on? We trained a compact student model from larger teacher models, a method called knowledge distillation. The student has 20.7 thousand parameters, with minimal penalty on average error, and the smallest acceptable version fits the budget of an FPGA. On a microcontroller, deployment remained limited by memory when the whole window of signal was processed. I keep that last sentence on purpose. Sometimes the hardware answer is not yet.

The third piece is the chip itself. In conventional computers the memory and the processor are separate, and the traffic between them is a bottleneck. Two reviews I co-authored start from that problem. One covers hardware for neural networks built from memristors [3]. The other covers searching for network designs that suit in-memory computing [4]. Computing inside the memory is one way to cut the traffic. These are reviews, and the first is plain that practical challenges remain.

So when does the processing belong on the device? When the answer has to be fast, private or cheap in energy. When the data is too heavy or too sensitive to send, and the decision is small enough to make where it is collected.

And when does it not? I can give four reasons that I would stand behind.

The model changes often. A model baked into a chip falls behind. If it needs retraining on new data every few weeks, the cloud is the better home, or the device must be built to accept updates.

The device cannot tell when it is wrong. A small model on a small device can be confidently mistaken, with nothing beside it to check against. This is the hard problem I wrote about in Trust is a level, not a label. If a device cannot flag its own low-confidence readings, it should not be making the decision alone.

The link is cheap and the data is small. If power and connectivity are plentiful and the data volume is low, sending it out is simpler and good enough. Not every sensor needs a neural network.

The decision needs context from many devices. One wearable cannot know what a thousand patients look like. It can do the first pass on its own signal and send a summary to a place that can see the whole.

My rule of thumb, which is a proposal and not a standard, is this: process on the device what must be fast, private or frugal, and send out what needs context, updating or a second opinion. On either side of that line, the device should say how sure it is.

I want to be careful about what I have shown. The three examples above support the design argument. They are not a head-to-head measurement of on-device and cloud energy on the same device, and I have not claimed one. That comparison, done properly for a given sensor, is the test I would trust.

If you are deciding where the intelligence should live in your own device, start by writing down the energy cost of sending one second of data out, and one second of computing on it. Then decide.