What kind of computer do employees need to run an AI locally at a reasonable speed?
Preferably directly from people who have gained experience with it. It should also be a decent AI that can partially automate office tasks, allowing employees to focus on more meaningful things, such as devoting more time to customers.
Alternatively, an API would also be fine, as long as the provider complies with the GDPR and all the other important things.
In principle, I would do all this in 4 parts limbs.
Overview:
As is generally known, the core component, the graphics card. Here you can look for generation and storage. The generation is usually more important, as newer maps also add new technologies, such as efficiency and speed. The memory only indicates how large the model can be in the sense of parameters.
Normal computer:
GPU: I would use the 20th series as the lowest generation. Of course, older generations also go in principle, but they are not so extensive from compatibility. From the 30th series, this will be possible to use the new NVIDIA RAG system. This allows you to feed your model with data like companies.
Apple devices:
Apple’s new chips are really unbelievable about AI. With the Unified Memory you can really load large models. I’d start here with 32 GB from the pro.
The model:
The Meta Llama models are for us gold. The 8B variants are probably what they use in the office.
Thank you very much. I probably won’t get through Apple devices at the management. If I understood your answer correctly, we’d be fine with an RTX 3060, wouldn’t we?
My understanding of graphics cards is not yet so good; I use AI a lot about the API, and there is much easier. However, it is of course not suitable for a company in terms of data protection.
That’s right. With the map you are very well placed.
Good luck!