ESP32S3 cluster running 1.58-bit (BitNet) Language model

(github.com)

59 points | by nkko 7 hours ago

6 comments

  • tdhz77 2 hours ago
    Soon ai in every lightbulb running Kubernetes
    • tombert 29 minutes ago
      You know, I've always liked Futurama but I always kind of thought it was silly that literally everything has an AI and a personality.

      But, you know, I actually think that there might be a logic to it. Economies of scale might mean that almost-literally every computer you buy in the year 3000 has some kind of AI-assistance chip in there, and sure maybe it will have full AI with a personality spitting out one-liners.

  • nonasking_ 46 minutes ago
    Thanks for sharing. It's fascinating to see a 0.5B LLM being split across seven ESP32s like this.
  • NDlurker 1 hour ago
    I'm curious how this would handle grammar checking on a basic word processor. Or maybe generate worlds for small text based games. I have no idea what the capabilities are of a cluster like this.
  • matthewfcarlson 1 hour ago
    I’m actually working on a small project that’s exactly this! Less quant so it’s only 150M parameters but this is amazing.
  • cameron_b 4 hours ago
    It is a bit of a bummer to see that the degree of 'compression' makes it a fancy llm noise-maker. It is still charming.
  • sjakati98 3 hours ago
    Gemma 4 when?
    • nkozyra 47 minutes ago
      We're gonna need a bigger ESP.