• 1 Post
  • 1 Comment
Joined 1 year ago
cake
Cake day: June 13th, 2023

help-circle
  • Thank you for the discussion folks, I’ll try out Llama.cpp and report back.

    I also saw the Neural Engine Stuff hasn’t been merged into main kernel yet but it’s available as a separate out of tree patch. Hopefully merging that will help with more model support? (Pure guesswork)

    I also saw that PCL stuff isn’t ready yet and u/marcan42 said it’s WIP that also might be helpful in getting better model support because read somewhere that metal isn’t going to be a part of the Asahi kernel ever(?)

    I’m no expert at any of this but hopefully we’ll be able to run some sort of GPTs locally someday.

    Good luck.