Apple's MLX framework can run the 816-billion-parameter Muse Spark Open model across Mac Studio computers. Apple says a four-Mac Studio cluster generates more than 36 tokens per second, using MLX's distributed inference and Thunderbolt RDMA support.

machinelearning.apple.comMLX runs 816-billion-parameter model across four Mac Studios