Explorations into Ollama⌗

My FIRST MYXT BLOG POST!
No AI or LLMs were used in the creation of this post. MTM original.

YouTube Videos Send Me Down Buffalo Trails⌗

As a Senior Strategist for a future-focused design agency - and Head of Product for an exciting new telepresence startup, many of my customers and team members are curious about the new 2024 AI wave.
From Generative Art displays, to Production, to LLMs, AI is definitely at top of mind for creating experiences.
My journey to building skillsets in AI Product Management - it sometimes feels like “how do I inject AI into whatever product I have” - whereas, several product managers are working on truly AI-native applications.
I am actively exploring both ends of this spectrum. Insert a southwestern pun about traveling the open plains here
Whether it is how to enhance our 900lbs projects - or think of truly unique experiences via AI.
For example - PeerVsn (the telepresence startup) has roadmapped the addition of multi-lingual translated live closed captions.
This would involve simultaneously directing our streams captured in studio towards a 3rd Party Closed Caption app that can send CC data to our app in real-time at ultra-low latency.
To better understand the opportunities for experiences like PeerVsn, I need to stay informed on underlying core technologies - like the AI models, how they are trained, algorithm design, and how developers interface with them.
Also, I’m passionate about ethical algorithm training (using ethically sourced data for model training), and my own personal data privacy.
I was inspired by NetworkChuck’s recent video to delve into self-hosting.

While I was SORELY tempted to buy a $3000 new computer immediately to secure an NVIDIA 4000 series GPU that would surely do anything I need (and give me an excuse to game more with work as an excuse) - I decided that a more realistic exploration of AI innovation would be - how to run models locally with a LESSER GPU.
I decided to experiment with my personal iMac computer.

My iMac Specifications:⌗

According to ChatGPU - the i7 CPU along with dedicated graphics w/ 4GB of VRAM - ensure that I can run the smaller models that are 7B or lower.
My experiment results!! Let’s see.
I ran llama3.2 model from Meta first - kind of a default they recommend running first. Cool stuff.

**Screenshot from Warp, my preferred terminal.

Ollama3.2 ran ran but2tery smooth. Off to a great start!