From Newsgroup: comp.sys.mac.system


Apple have recently, and quietly, released a Thunderbolt software
upgrade in OS 26.2 called RDMA, to allow very rapid communication
amongst a cluster of 4 Mac Studios - virtually addressing memory in its partner computers and v-v.

This is a very expensive setup (shy of US$40K) for Joe Budget, but for a
small AI firm, or university w/o the bigger budgets, it's attainable.

And certainly sips power for the performance delivered.

The following tech youtubers try it out with different approaches to
their testing.

https://youtu.be/x4_RsUxRjKU (Jeff Geerling's channel)

and

https://youtu.be/4l4UWZGxvoc (Jakku - a little "enthusiastic" - but he
goes through more AI configs and comparisons)

In both these cases it appears Apple have lent them the kits to try out
for a couple months.

The "glue" s/w in both cases (exo) is still not perfectly baked but
seems to be improving on a near daily basis. (Open source - not clear
what their collab with Apple is but Jeff Geerling points out that some
secrecy issues taint this "open source" project - possibly due to
working with Apple). Also the variety of LLM's available so far is
limited (so far) for reasons I don't recall.

The models are massive with 10's of billions of parameters and crowding
the 1T parameter space.

An Indian tech writer has a nice spin on all this for countries w/o the
cash to play in Big-AI or other issues preventing some classes of data
from leaving India for overseas servers. (Personal, medical, military, government, etc....) https://cxotoday.com/news-analysis/did-apple-just-quietly-give-startups-a-way-to-run-trillion-parameter-ai-models-without-touching-the-cloud/

I'm not a "fan" of AI, but it's prudent to learn about it, play with it
a little and understand its abilities. I do use it - cautiously - to translate long text into French (and most models can adjust style and
tone for regional (say France v. Québec), formal, informal, technical,
etc. Then of course I read the French v. carefully - mistakes and mis-translations do occur (and such lead to context changes as well - if
not flat out borking the meaning of the text)).

If you're into the "big math" involved in LLM's, I'd suggest YouTube
channels:
3Blue1Brown and Welch Labs. Outstanding presenters and graphics.
--
"It would be a measureless disaster if Russian barbarism overlaid
the culture and independence of the ancient States of Europe."
Winston Churchill
--- Synchronet 3.21a-Linux NewsLink 1.2