Local AI is more interesting than I expected

Running models locally

The interesting part isn’t necessarily replacing cloud models. It’s understanding where local inference makes sense.

For example, a simple Python function might look like this:

def analyse_activity(activity):
    return {
        "distance": activity["distance"],
        "heart_rate": activity["average_hr"]
    }

There are obvious constraints around memory and model size, but techniques such as quantisation and mixture of experts make the picture much more interesting.