Exploring Local AI: First Steps with DeepSeek
In my ongoing exploration of AI technology, I've taken a significant step forward by beginning to run models locally. My first venture into this space involved testing DeepSeek R1 14b via Ollama on my RTX 3090 12GB video card. While the experience has been enlightening, it's also revealed some interesting challenges and opportunities for future improvement.
Initial Experience
For my first task I unplugged the network cable and asked it to summarize my motivation and experience running a model unplugged. It wasn't that great but here is what it produced:
Surprisingly, the process has turned out to be somewhat easier than anticipated. However, hardware limitations have emerged as significant challenges. GPU RAM was the first hurdle, as expected, and finding high-end video cards with 24GB or more of VRAM has been unexpectedly difficult due to limited availability.
Performance Insights
Currently, my setup isn't powerful enough for meaningful work—it's about 1/100th as effective as models like Claude/Cline. Since I already run everything locally, upgrading to a more robust GPU is necessary at some point to bridge this performance gap.
Looking Forward
The experience with DeepSeek R1 14b has made it clear that while local AI deployment is becoming more accessible, there's still a significant hardware barrier to achieving production-level performance. I'll need a better video card to run more parameters and achieve faster processing times. However, I'm optimistic about the future—as models continue to improve and hardware becomes more readily available, I expect this setup to become increasingly useful for practical applications.