locally on my Mac Studio M1 with 64GB of RAM.
9 tokens per second is not fast, but acceptable.
A 15-second wait for the first token output is very good.
Subscribe to:
Post Comments (Atom)
apt quotation..
“A man should be able to change a diaper, plan an invasion, butcher a hog, conn a ship, design a building, write a sonnet, balance accounts, build a wall, set a bone, comfort the dying, take orders, give orders, cooperate, act alone, solve equations, analyze a new problem, pitch manure, program a computer, cook a tasty meal, fight efficiently, die gallantly. Specialization is for insects.” by Robert A. Heinlein (author, aeronautical engineer, and naval officer)
No comments:
Post a Comment