I'm making LLMs work on small and old devices

 (github.com)

24 points | by mrashad74 1 hour ago

3 comments

  • ForHackernews 3 minutes ago
    This is great. I've recently been playing with repurposing some old thin clients as homelab Linux machines

    Not my site, but same idea: https://www.parkytowers.me.uk/thin/

    All this old/obsolete gear is plenty capable when used with the right software.

  • dava96 33 minutes ago
    This is really cool man. I like this on going trend of resurrecting old tech with AI and this is yet another avenue to explore.

    I'd be interested in seeing some older flag ship mobile devices

    [-]
    • mrashad74 19 minutes ago
      I have a few phones and tablets laying around, will try to play with..
  • mrashad74 1 hour ago
    Recently I started this open source project at my AI company, where I'm trying to make LLMs working

    Now Reached 16B models running on iPad Air 1st gen and xiaomi tv box, both have less than 1GB of usable memory, and reached some decent speeds given the hardware.

    I'm open sourcing the effort done, kernels developed and results for everyone to re-produce and push more, also the modified models are on hugging face : https://huggingface.co/hyperionaidev

    I want a more challenging device, what do you think?

    [-]
    • post-it 19 minutes ago
      > I want a more challenging device, what do you think?

      One of those vapes with a screen on it.

      [-]