🧠 Simple AI Assistant Docker App that Remembers

This project is a production-ready AI chat app built with Docker Model Runner, Streamlit, and LangChain. It lets you talk to a local LLM running via Docker, or switch seamlessly to a large cloud-based model (like those on OpenRouter), all while remembering your entire conversation history. Each model will get the full chat context, even in moments when it wasn't participating in the conversation.

🎥 Video Tutorial

To build this application step by step, please watch my YouTube tutorial below. It will explain all the app processes, and the logic behind them.

⭐ Features

1. Run local open-source LLMs with Docker Model Runner 🤖
2. Clean Streamlit chat interface with message history 💻
3. Seamless switch between local and cloud models 🕹️
4. Context-passing for memory-aware responses 💡
5. Fully containerized with Docker Compose 🐋

📸 Screenshot

⚡️ Quick Start

1️⃣ Prerequisites

• Docker Desktop installed and updated to current version.
• Docker Model Runner enabled in Docker Desktop (see official docs)

2️⃣ Clone This Repo

git clone https://github.com/MariyaSha/simple_AI_assistant.git
simple_AI_assistant

3️⃣ Create a .env File

Create a file named .env in the project root:

LOCAL_BASE_URL=http://model-runner.docker.internal/engines/llama.cpp/v1
REMOTE_BASE_URL=https://openrouter.ai/api/v1
LOCAL_MODEL_NAME=ai/gemma3
REMOTE_MODEL_NAME=qwen/qwen3-30b-a3b
OPENROUTER_API_KEY=YOUR_OPENROUTER_API_KEY

• Replace YOUR_OPENROUTER_API_KEY with your actual OpenRouter key.
• Choose any local or remote model from:
  • 👉 Docker Model Catalog
  • 👉 OpenRouter

4️⃣ Run the App

docker compose up

Then open your browser to:

http://localhost:8501

✅ Your AI chat app is ready to use!

🗂️ Project Structure

.
├── app.py                # Streamlit chat app with LangChain
├── Dockerfile            # Container for running the app
├── docker-compose.yaml   # Defines app + model services
├── requirements.txt      # Python dependencies
└── .env                  # Environment variables (you create this)

🧩 How It Works

• The llm service in docker-compose.yaml uses Docker Model Runner to serve a local LLM (e.g. Gemma).
• The ai-app service is a Streamlit web app that:
  • Stores message history in session state
  • Passes the entire chat context to the LLM for memory-aware responses
  • Lets you switch between local and remote models with a simple checkbox

🚀 Customization

• Change Local Model
  Edit LOCAL_MODEL_NAME and LOCAL_BASE_URL in your .env.

• Change Remote Model
  Edit REMOTE_MODEL_NAME, REMOTE_BASE_URL, and your OPENROUTER_API_KEY.

• Dependencies
  Add any extra Python packages to requirements.txt.

📚 Helpful Links

• 🐳 Docker Model Runner Documentation
• 🔎 Find Models in Docker Catalog
• 🌐 OpenRouter
• 🐍 Streamlit
• 🦜 LangChain for Python

🤝 Contributing

If you'd like to contribute, please create an issue and describe what you have in mind.
I'm trying to keep this repository as close as possible to the video workflow, but if you'd like to take it to the next level, I can split it in two.