====== Hardware ======
{{ https://www.amd.com/content/dam/amd/en/images/blogs/migrated-blogs/753606-0.jpg?800|Unified vs traditional memory architecture}}
* [[youtube>nwIZ5VI3Eus|Apple M3 Ultra vs RTX 5090 – The Final Battle]] -- equivalent on small models, but M3 Ultra is better on big models like [[https://openrouter.ai/deepseek/deepseek-r1-distill-llama-70b:free|DeepSeek R1 Distill Llama 70B]]
* [[youtube>bAao58hXo9w&t=1193s|Comparison of M3 Ultra & RTX 5090 32GB & RTX Pro 6000 96GB]] has an outcome that it is better to build RTX Pro 6000 system rather than Apple M3 Ultra for $10000 (price of M3 Ultra wit 512GB of RAM)
* [[youtube>QJqKqxQR36Y|Wire eight DGX Sparks into a 1TB “VRAM” cluster]] -- one would need a 4400Gbps switch and [[aliexpress>1005008648510408|QSFP56 cables (~€65)]].
* [[youtube>wCBLMXgk3No|Running LLM on AMD "Strix Halo" AI Ryzen MAX+ 395 (HP Z2 G1a Mini Workstation)]] -- Linux tuning and software configuration
* [[youtube>c3v6R0IbnYs|Building a Framework Desktop cluster with AMD's Ryzen AI Max+ 395]]
* [[youtube>GBR6pHZ68Ho|M4 Mac Mini cluster]] connected via Thunderbolt.
* [[youtube>PhJnZnQuuT0|Comparison of Jetson Thor, DGX Spark, and Apple Silicon M4 Pro]] all having unified 128GB of RAM with 273GB/s memory bandwidth.
* [[youtube>QbtScohcdwI|Stress-testing the NVIDIA DGX Spark, Dell Pro Max GB10, ASUS Ascent GX10, and MSI Edge Expert]] all running the same Grace Blackwell (GB10) chip and 128GB of unified RAM.
===== AMD chipset =====
^ Name ^ Price ^ CPU ^ GPU ^ Memory ^ Storage ^ Display ports ^ Extension slots ^ Connectivity ^ Wireless ^ Dimensions ^ Max TFLOPS ^ Tokens/sec (120B Q4) ^ Tokens / € ^ Tokens / W ^ Clustering ^
| [[https://frame.work/nl/en/desktop?tab=specs|Framework Desktop]] | [[https://frame.work/nl/en/products/desktop-diy-amd-aimax300/configuration/new|€3591]] | AMD Ryzen™ AI Max+ 395, 3.0GHz, 16 cores | AMD Radeon™ 8060S, 2.9GHz, 20 compute units | 128GB LPDDR5x, 8000 MT/s | 1TB, 2× NVMe PCIe 4.0 ×4 M.2 2280, max 8TB each | 1× HDMI v2.1, 2× DisplayPort v1.4 (8K@60Hz) | 1× PCIe 4.0 ×4 slot (max 50GbE) | RJ45 5Gbit (Realtek RTL8126), 2× USB4-C, 2× USB-A 3.2 Gen 2 | WiFi 7 (AMD RZ717) | 123.7x123.1x54.6mm | ~50–60 FP16 | 12–15 | 0.0055–0.0075 | 0.10–0.14 | :HELP: Either Ethernet (bad), or adding [[https://www.ebay.com/sch/i.html?_nkw=qsfp28+pci&_from=R40&_sop=15|50GbE QSFP28 PCIe adapter]] + cables |
| [[https://www.notebookcheck.net/Bosgame-M5-AI-Mini-Desktop-Pricing-and-release-date-set-for-new-AMD-Ryzen-AI-Max-395-mini-PC.1024582.0.html|Bosgame M5 AI]] | [[https://www.bosgamepc.com/products/bosgame-m5-ai-mini-desktop-ryzen-ai-max-395?sku=18073504178329773749034095|€2080]] (pre-order $1699) | AMD Ryzen™ AI Max+ 395, 3.0GHz, 16 cores | AMD Radeon™ 8060S, 2.9GHz, 20 compute units | 128GB LPDDR5x, 8000 MT/s | 2TB, 2× NVMe PCIe 4.0 ×4 M.2 2280, max 8TB each | 1× HDMI v2.1, 1× DisplayPort v1.4 (8K@60Hz) | | RJ45 2.5Gbit, 2× USB4-C, 3× USB-A 3.2 Gen 2, 2× USB-A 2.0, SD card | WiFi 7, Bluetooth 5.2 (in M.2 2230 key-E PCIe 3.0 slot) | :HELP: | ~50–60 FP16 | 12–15 | 0.0055–0.0075 | 0.10–0.14 | :DEL: |
| [[https://www.geekompc.com/geekom-a9-mega-ai-mini-pc/|GEEKOM A9 Mega AI Mini PC]] | $3200 / [[https://www.amazon.nl/-/en/GEEKOM-A9-Mega-Improved-Pre-installed/dp/B0GL7KF3LC|€3500]] ([[https://www.kickstarter.com/projects/1906688106/geekom-a9-mega-the-most-powerful-mini-pc-on-earth|kickstarter $1899]]) | AMD Ryzen™ AI Max+ 395, 3.0GHz, 16 cores | AMD Radeon™ 8060S, 2.9GHz, 20 compute units | 128GB LPDDR5x, 8000 MT/s | 2TB, 2× NVMe PCIe 4.0 ×4 M.2 2280, max 8TB each | 2× HDMI v2.1 (8K@60Hz) | | 2× RJ45 2.5Gbit, 2× USB4-C (support Display Port 2.1), 2× USB-C, 3× USB-A 3.2 Gen 2, SD card | WiFi 7, Bluetooth 5.4 (in M.2 2230 key-E PCIe 3.0 slot) | 171x171x71mm | ~50–60 FP16 | 12–15 | 0.0055–0.0075 | 0.10–0.14 | :DEL: |
| [[https://de.gmktec.com/en/products/gmktec-evo-x2-amd-ryzen%E2%84%A2-ai-max-395-mini-pc-1|GMKtec EVO-X2]] | €3000 | AMD Ryzen™ AI Max+ 395, 3.0GHz, 16 cores | AMD Radeon™ 8060S, 2.9GHz, 20 compute units | 128GB LPDDR5x, 8000 MT/s | 2TB, 2× NVMe PCIe 4.0 ×4 M.2 2280, max 8TB each | 1× HDMI v2.1, 1× DisplayPort v1.4 (8K@60Hz) | | RJ45 2.5Gbit, 2× USB4-C, 3× USB-A 3.2 Gen 2, 2× USB-A 2.0, SD card | WiFi 7, Bluetooth 5.4 (in M.2 2230 key-E PCIe 3.0 slot) | 193x185.8x77mm | ~50–60 FP16 | 12–15 | 0.0055–0.0075 | 0.10–0.14 | :DEL: |
| [[https://www.bee-link.com/products/beelink-gtr9-pro-amd-ryzen-ai-max-395?variant=47842426224882|Beelink GTR9 Pro]] | €3000 | AMD Ryzen™ AI Max+ 395, 3.0GHz, 16 cores | AMD Radeon™ 8060S, 2.9GHz, 20 compute units | 128GB LPDDR5x, 8000 MT/s | 2TB, 2× NVMe PCIe 4.0 ×4 M.2 2280, max 8TB each | 1× HDMI v2.1 (8K@60Hz) | | 2× RJ45 40Gbit, 3× USB4-C, 2× USB-A 3.2 Gen 2, 2× USB-A 2.0, SD card | WiFi 7 (MT7925), Bluetooth 5.4 (in M.2 2230 key-E PCIe 3.0 slot) | 180x180x90.8mm | ~50–60 FP16 | 12–15 | 0.0055–0.0075 | 0.10–0.14 | :HELP: Maybe 2×40Gbit = 10MB/s |
| [[https://liliputing.com/fevm-fa-ex9-mini-pc-with-amd-strix-halo-coming-soon/|FEVM FA-EX9]] | [[aliexpress>1005010109359032|€2950]] | AMD Ryzen™ AI Max+ 395, 3.0GHz, 16 cores | AMD Radeon™ 8060S, 2.9GHz, 20 compute units | 128GB LPDDR5x, 8000 MT/s | 2TB, NVMe PCIe 4.0 ×4 + x1 M.2 2280, max 8TB each | 1× HDMI v2.1, 1× DisplayPort v1.4 (8K@60Hz) | Oculink 64Gb/s | RJ45 2.5Gbit, 2× USB4-C, 3× USB-A 3.2 Gen 2, 2× USB-A 2.0, SD card | WiFi 7 (MT7925), Bluetooth 5.3 (in M.2 2230 key-E PCIe 3.0 slot) | 192x190x55mm | ~50–60 FP16 | 12–15 | 0.0055–0.0075 | 0.10–0.14 | Oculink |
| [[https://www.minisforum.com/products/ms-s1-max|MS-S1 MAX]] | [[https://minisforumpc.eu/products/minisforum-ms-s1-max-mini-pc|€3120]] | AMD Ryzen™ AI Max+ 395, 3.0GHz, 16 cores | AMD Radeon™ 8060S, 2.9GHz, 20 compute units | 128GB LPDDR5x, 8000 MT/s | 2TB, NVMe PCIe 4.0 ×4 + x1 M.2 2280, max 8TB each | 1× HDMI v2.1 (8K@60Hz) | 1× PCIe 4.0 ×4 slot | 2× RJ45 10Gbit (Realtek RTL8127), 2× USB4-C (40Gb/s, Display Port 2.0, PD out 15W), 2× USB4-C v2 (80Gb/s, Display Port 2.0, PD out 15W), 3× USB-A 3.2 Gen 2, 2× USB-A 2.0, SD card | WiFi 7 (MT7925), Bluetooth 5.4 (in M.2 2230 key-E PCIe 3.0 slot) | 192x190x55mm | ~50–60 FP16 | 12–15 | 0.0055–0.0075 | 0.10–0.14 | See above using [[https://www.ebay.com/itm/144116491578|MCX416A-CCAT Mellanox ConnectX-4 2× 50Gbit/s QSFP28 ($146)]] |
* [[https://liliputing.com/more-ryzen-ai-max-395-mini-pcs-with-128gb-are-now-available-if-you-can-afford-one/|More Ryzen AI Max+ 395 mini PCs with 128GB are now available… Check other models]]
* [[https://www.notebookcheck.net/MSI-AI-Edge-mini-PC-based-on-AMD-Ryzen-AI-Max-395-and-128-GB-LPDDR5X-RAM-launching-soon.1199933.0.html|MSI AI Edge mini-PC based on AMD Ryzen AI Max+ 395 and 128 GB LPDDR5X RAM launching soon]]
===== Future =====
NVidia chipset:
* [[https://www.theverge.com/news/631835/nvidia-blackwell-ultra-ai-chip-gb300|Rubin has 50 petaflops of FP4, up from 20 petaflops in Blackwell]]
===== Old-school but interesting =====
* [[youtube>G5DBks2IsYI&t=854s|Nvidia Tesla V100 в работе и играх]] -- сравнение Tesla V100 с 3090, 4090, 5090 по TFLOPS, [[youtube>Bg_50B81-6E|Что может серверная Tesla V100 в играх?]] -- установка самых последних драйверов на NVidia Tesla V100 PG503.
* [[youtube>xyKEQjUzfAk|Cheap GMKtec Mini PC with 96GB RAM runs a 70B LLM in Intel Core Ultra CPU]] using [[github>intel/ipex-llm|Intel® LLM Library]] (discontinued), but checkout [[youtube>Cmsx01H-0xY|Geekom A9 Max with AMD Ryzen AI Max runs 120B LLM]]
====== Software ======
* [[https://lmstudio.ai/|LM Studio]] -- run AI models, locally and privately
* [[https://localllm.in/blog/interactive-vram-calculator|VRAM Calculator for Local Open Source LLMs]] or try [[github>alexziskind1/llm-inference-calculator|LLM Inference Hardware Calculator]]
* [[youtube>qmAbco38pXA|"speculative decoding" is a technique to dramatically boost Large Language Model speed]] using a smaller model drafts ahead, while a larger model verifies
===== OpenClaw =====
==== Install OpenClaw ====
* Add new ''openclaw'' user: ''useradd -m -u 987 -s /bin/bash openclaw && mkdir -m 700 /home/openclaw && chown openclaw:openclaw /home/openclaw''
* Make sure that ''gateway.auth.token'' in ''/home/openclaw/.openclaw/openclaw.json'' matches the one in ''docker-compose.yml'' (''openclaw-gateway.environment.OPENCLAW_GATEWAY_TOKEN'').
* Start/stop the container: ''docker compose up -d openclaw-gateway'' / ''docker compose down''
* To approve device (OpenClaw UI) request:
# docker compose exec openclaw-gateway bash
$ /app/openclaw.mjs devices list
Direct scope access failed; using local fallback.
Pending (1)
┌──────────────────────────────────────┬───────────────────────────────────────────────────┬──────────┬───────────────┬──────────┬────────┐
│ Request │ Device │ Role │ IP │ Age │ Flags │
├──────────────────────────────────────┼───────────────────────────────────────────────────┼──────────┼───────────────┼──────────┼────────┤
│ 4ec7a9d0-f5f2-4316-9c07-f9e4fa96620a │ f74fdd392ee3bca006e344a86554ef76b473d7e2bd8502fde │ operator │ │ just now │ │
└──────────────────────────────────────┴───────────────────────────────────────────────────┴──────────┴───────────────┴──────────┴────────┘
$ /app/openclaw.mjs devices approve 4ec7a9d0-f5f2-4316-9c07-f9e4fa96620a
Direct scope access failed; using local fallback.
Approved f74fdd392ee3bca006e344a86554ef76b473d7e2bd8502fde (4ec7a9d0-f5f2-4316-9c07-f9e4fa96620a)
* If you get the following error ''Control UI requires gateway.controlUi.allowedOrigins (set explicit origins), or set gateway.controlUi.dangerouslyAllowHostHeaderOriginFallback=true to use Host-header origin fallback
mode'' then add the following to ''.openclaw/openclaw.json'':
"gateway": {
"controlUi": {
"allowedOrigins": [
"http://localhost"
]
}
}
and force that host in Apache configuration:
RequestHeader set Origin "http://localhost:18789"
* To workaround OpenClaw error "Proxy headers detected from untrusted address. Connection will not be treated as local." add the following to Apache configuration:
ProxyAddHeaders Off
* To force OpenClaw to bind to localhost instead of *, make sure that `OPENCLAW_GATEWAY_BIND` is not set ([github>openclaw/openclaw/blob/3e72c0352dde84a0bcb3aabafa99c2d4b12d1c46/docker-compose.yml#L34C10-L37|remove lines that configure ''--bind'' and ''--port'']]) and configure those via ''.openclaw/openclaw.json''
"gateway": {
"port": 18789,
"mode": "local",
"bind": "loopback"
}
Then check logs:
openclaw-gateway-1 | 2026-04-05T17:55:40.225+00:00 [canvas] host mounted at http://127.0.0.1:18789/__openclaw__/canvas/ (root /home/node/.openclaw/canvas)
openclaw-gateway-1 | 2026-04-05T17:55:40.296+00:00 [gateway] listening on ws://127.0.0.1:18789, ws://[::1]:18789 (PID 7)
* To add extras to original docker image do:
# cat - > Dockerfile
FROM alpine/openclaw:2026.4.2
USER root
RUN apt-get update -q \
&& apt-get install -y -q --no-install-recommends \
python3-bs4 \
python3-dateutil \
python3-lxml \
python3-requests \
python3-yaml \
&& rm -rf /var/lib/apt/lists/* /var/cache/apt/archives/*
USER openclaw
# docker build -t openclaw-custom:2026.4.2 .
then use ''openclaw-custom:2026.4.2'' in ''docker-compose.yaml''.
=== Install LLama local model ===
* Add to ''docker-compose.yml'':
services:
ollama:
image: ollama/ollama:latest
user: "987:1001"
network_mode: host
volumes:
- /home/openclaw/.ollama:/.ollama
* Pull LLM model (~3GB) and run it:
# docker compose up -d ollama
# docker compose exec ollama ollama pull llama3.1:8b-instruct-q4_K_M
# docker compose exec ollama ollama run llama3.1:8b-instruct-q4_K_M
* Configure OpenClaw to use it so that configuration in ''.openclaw/openclaw.json'' looks like this:
"models": {
"providers": {
"vllm": {
"baseUrl": "http://127.0.0.1:11434/v1",
"apiKey": "dummy-api-key",
"api": "openai-completions",
"models": [
{
"id": "ollama/llama3.1:8b-instruct-q4_K_M",
"name": "ollama/llama3.1:8b-instruct-q4_K_M",
"reasoning": false,
...
=== Install browser support ===
* Add extra directory mappings:
volumes:
- /home/openclaw/.openclaw:/home/node/.openclaw
- /home/openclaw/.openclaw/.agent-browser:/home/node/.agent-browser
- /home/openclaw/.openclaw/.cache:/home/node/.cache
* Run:
~/.openclaw$ ./bin/agent-browser install
* Test:
~/.openclaw$ ./bin/agent-browser open google.com
* To configure add this to ''.openclaw/openclaw.json'':
~/.openclaw$ ./bin/agent-browser open google.com
==== Using OpenClaw ====
* To re-run onboarding, do ''%%docker compose run --rm openclaw-cli onboard%%'' or like below.
* To switch the model, run ''%%docker compose exec openclaw-gateway /app/openclaw.mjs config%%''
* To list all model, run
# docker compose exec openclaw-gateway /app/openclaw.mjs models list
Model Input Ctx Local Auth Tags
openrouter/stepfun/step-3.5-flash:free text 250k no yes default,configured,alias:OpenRouter
openai/gpt-5.1-codex text+image 391k no yes fallback#1,configured,alias:GPT
* When you get a paring request from a bot:
OpenClaw: access not configured.
Your Telegram user id: 30528162
Pairing code: Z5RQ92MX
you need to run
# docker compose exec openclaw-gateway /app/openclaw.mjs pairing approve telegram Z5RQ92MX
Approved telegram sender 30528162.