So I am wondering if anyone has had success with AI, which AI, free or paid, and maybe if it is some AI that you are hosting yourself (and, if so, is it linux based?)?
I host locally using llama.cpp on a i9 64 GB ram Dell 5280 workstation and two
RTX 3060 GPUs with 12 GB each. llama.cpp utilizes both GPUs to give me a combined 24 GB of vram. I created a new door called Quantasia that uses both cloud and local LLMs, and I like the response times of llama.cpp that Quantasi
achieves when hitting a local bot based on it.
Is llama the same thing as Ollama? I have heard of some folks hosting that themselves. I don't currently have any machines that have GPUs but suspect they might be required to host my own? Did that machine come with the GPUs installed?
Thanks for answering, and in advance for the 20-questions. ;)
Mike
* SLMR 2.1a * Those who live by the sword... kill those who don't.
---
■ BgNet 1.0ß12 ≈ moe's tavern * 1-502-875-8938 * moetiki.ddns.net:27
Web-based telnet client
IRC
Email & news access