I built a phone app at ocr.503mail.com: take a picture of a piece of mail, and it reads the text, writes a short summary ("electric bill, $84.12, due October 15") and reads it out loud. Everything runs on my own Proxmox box, so no mail ever leaves the house. This post is the complete rebuild guide: every file and every command, with a Copy button on each.
How it works
- The phone page (an installable web app) opens the camera and uploads the photo.
- RapidOCR (PaddleOCR's models on ONNX Runtime) pulls out the text in about 1 second.
- Qwen2.5 7B (running in Ollama) writes a 2 to 4 sentence spoken summary: who it's from, what it is, amounts and due dates, and what to do. It runs on the mini PC's built-in Radeon 680M graphics (via Vulkan) and takes about 5 seconds for a typical letter (about 12 seconds on the CPU alone).
- The summary is read out loud, either by Piper (a neural voice on the server, about 1 second) or by a voice installed on the phone (see "Voices" below).
- The Look closer button re-reads the photo with the Qwen2.5-VL 7B vision model instead. It's better on handwriting and messy pieces. It takes about 4 seconds once loaded, or about 20 seconds the first time (it was about a minute on CPU alone).
Nothing is stored: photos, text and audio only exist in memory while a request is being handled. There's one user with one password, and 5 wrong passwords locks that address out for 15 minutes.
Containers involved
| Container | Role | LAN IP |
|---|---|---|
| CT 300 (ocr-reader) | The app: Flask + gunicorn on port 8000, Ollama, RapidOCR, Piper | x.x.0.189 (DHCP) |
| CT 108 (Caddy reverse proxy) | HTTPS for ocr.503mail.com, forwards to CT 300 (ocr-reader) | x.x.0.122 |
| CT 201 (Uptime Kuma) | Monitors the site (status.503mail.com) | |
| CT 202 (creator-signup) | Only used to run the Kuma script, since it already has the Kuma API library and login | x.x.0.198 |
The host is a Ryzen 7 7735HS mini PC (16 threads, 62GB RAM) with no graphics card, but its built-in Radeon 680M runs the AI models through Vulkan, borrowing shared system RAM. The BIOS only gives it 512MB of dedicated memory, which doesn't matter, because it can use about 31GB of shared RAM. CT 300 (ocr-reader) gets 8 cores, 16GB RAM and a 40GB disk; the two models are about 11GB.
No DNS change was needed, because 503mail.com already has a wildcard A record.
Where to run each command
- Proxmox host shell: in the Proxmox web UI click pve → Shell (or SSH to the host as root). Almost every command below runs here. The
pct exec 300 -- ...commands reach into a container from the host. - Inside CT 300 (ocr-reader): in the Proxmox web UI click 300 (ocr-reader) → Console, or run
pct enter 300from the host shell. Only needed for setting the password.
Rebuild, step by step
1. Make a working folder on the host
Proxmox host shell:
mkdir -p /root/ocr-reader && cd /root/ocr-reader
2. Create every file
Paste each block into the Proxmox host shell. Each one writes one file into /root/ocr-reader/.
300-create-ocr-lxc.sh creates the container:
cat > /root/ocr-reader/300-create-ocr-lxc.sh <<'EOF'
#!/bin/bash
# ============================================================
# 300-create-ocr-lxc.sh
# Run this ON YOUR PROXMOX HOST.
# Creates CT 300 for the OCR mail reader (ocr.503mail.com):
# RapidOCR + Ollama (qwen2.5 7B / qwen2.5vl 7B) + Piper TTS.
# Sized for CPU-only LLM inference: 8 cores, 16GB RAM, 40GB disk
# (the two Ollama models are ~11GB together).
# ============================================================
set -euo pipefail
CTID=300
HOSTNAME="ocr-reader"
STORAGE="local-lvm"
TEMPLATE="local:vztmpl/debian-12-standard_12.12-1_amd64.tar.zst"
DISK_SIZE="40"
MEMORY="16384"
SWAP="2048"
CORES="8"
BRIDGE="vmbr0"
IP_CIDR="dhcp"
if pct status "$CTID" >/dev/null 2>&1; then
echo "CT $CTID already exists - refusing to overwrite." >&2
exit 1
fi
echo "==> Creating LXC $CTID ($HOSTNAME)..."
pct create "$CTID" "$TEMPLATE" \
--hostname "$HOSTNAME" \
--storage "$STORAGE" \
--rootfs "${STORAGE}:${DISK_SIZE}" \
--memory "$MEMORY" \
--swap "$SWAP" \
--cores "$CORES" \
--net0 "name=eth0,bridge=${BRIDGE},ip=${IP_CIDR}" \
--features nesting=1 \
--unprivileged 1 \
--onboot 1
echo "==> Starting container..."
pct start "$CTID"
echo "==> Waiting for network..."
sleep 8
echo "==> Container $CTID created and started. IP:"
pct exec "$CTID" -- hostname -I
EOF
301-provision-ocr.sh runs inside the container and installs everything:
cat > /root/ocr-reader/301-provision-ocr.sh <<'EOF'
#!/bin/bash
# ============================================================
# 301-provision-ocr.sh
# Run this INSIDE CT 300 (pct push it in, then
# `pct exec 300 -- bash /root/301-provision-ocr.sh`).
# Installs Python deps (RapidOCR, Piper, Flask), a Piper voice,
# and Ollama with the summary + vision models. Idempotent.
# ============================================================
set -euo pipefail
APP=/opt/ocr-reader
VOICE_BASE="https://huggingface.co/rhasspy/piper-voices/resolve/main/en/en_US/lessac/medium"
echo "==> System packages..."
apt-get update
apt-get -y upgrade
apt-get install -y python3 python3-venv python3-pip curl ca-certificates \
libgl1 libglib2.0-0 zstd
echo "==> App user + dirs..."
id ocrapp >/dev/null 2>&1 || useradd --system --home "$APP" --shell /usr/sbin/nologin ocrapp
mkdir -p "$APP/app/static" "$APP/voices"
echo "==> Python venv..."
[ -d "$APP/venv" ] || python3 -m venv "$APP/venv"
"$APP/venv/bin/pip" install --upgrade pip
"$APP/venv/bin/pip" install flask gunicorn requests pillow numpy \
"rapidocr_onnxruntime>=1.3,<2" piper-tts
echo "==> Piper voice..."
for f in en_US-lessac-medium.onnx en_US-lessac-medium.onnx.json; do
[ -f "$APP/voices/$f" ] || curl -fL -o "$APP/voices/$f" "$VOICE_BASE/$f"
done
echo "==> Ollama..."
command -v ollama >/dev/null 2>&1 || curl -fsSL https://ollama.com/install.sh | sh
systemctl enable --now ollama
sleep 3
ollama pull qwen2.5:7b
ollama pull qwen2.5vl:7b
chown -R ocrapp:ocrapp "$APP"
echo "==> Done. Next: run 302-deploy-ocr-app.sh on the host."
EOF
302-deploy-ocr-app.sh copies the app into the container and (re)starts it:
cat > /root/ocr-reader/302-deploy-ocr-app.sh <<'EOF'
#!/bin/bash
# ============================================================
# 302-deploy-ocr-app.sh
# Run this ON YOUR PROXMOX HOST after scp'ing ocr-reader/app/*
# to /root/ocr-reader/ on the host. Pushes the app into CT 300
# and (re)starts the service. Safe to re-run for every deploy.
# ============================================================
set -euo pipefail
CTID=300
SRC=/root/ocr-reader
DST=/opt/ocr-reader/app
pct push "$CTID" "$SRC/app.py" "$DST/app.py"
pct push "$CTID" "$SRC/set_password.py" "$DST/set_password.py"
pct push "$CTID" "$SRC/index.html" "$DST/static/index.html"
pct push "$CTID" "$SRC/manifest.webmanifest" "$DST/static/manifest.webmanifest"
pct push "$CTID" "$SRC/ocr-reader.service" /etc/systemd/system/ocr-reader.service
pct exec "$CTID" -- chown -R ocrapp:ocrapp /opt/ocr-reader
pct exec "$CTID" -- systemctl daemon-reload
pct exec "$CTID" -- systemctl enable ocr-reader
pct exec "$CTID" -- systemctl restart ocr-reader
sleep 3
pct exec "$CTID" -- systemctl --no-pager status ocr-reader
EOF
303-add-kuma-monitors.py adds the Uptime Kuma monitors:
cat > /root/ocr-reader/303-add-kuma-monitors.py <<'EOF'
"""Add Uptime Kuma monitors for the Mail Reader (ocr.503mail.com, CT 300).
Run inside CT 202 (creator-signup), which already has uptime_kuma_api and the
Kuma credentials in signup_handler.py:
pct push 202 /root/ocr-reader/303-add-kuma-monitors.py /root/303-add-kuma-monitors.py
pct exec 202 -- /opt/creator-signup/venv/bin/python /root/303-add-kuma-monitors.py <CT 300 IP>
Idempotent: skips monitors that already exist by name.
"""
import sys
sys.path.insert(0, "/opt/creator-signup")
import signup_handler as sh
from uptime_kuma_api import UptimeKumaApi, MonitorType
BACKEND_IP = sys.argv[1] # CT 300 (ocr-reader)'s LAN IP
TAG_NAME = "Personal Site"
MONITORS = [
dict(type=MonitorType.HTTP, name="ocr.503mail.com — Mail Reader",
url="https://ocr.503mail.com/healthz", interval=60),
dict(type=MonitorType.PORT, name=f"{BACKEND_IP}:8000 — Mail Reader LXC (backend for ocr.503mail.com)",
hostname=BACKEND_IP, port=8000, interval=60),
]
api = UptimeKumaApi(sh.UPTIME_KUMA_URL)
api.login(sh.UPTIME_KUMA_USER, sh.UPTIME_KUMA_PASS)
existing = {m["name"] for m in api.get_monitors()}
tag = next(t for t in api.get_tags() if t["name"] == TAG_NAME)
for m in MONITORS:
if m["name"] in existing:
print("exists:", m["name"])
continue
monitor_id = api.add_monitor(**m)["monitorID"]
api.add_monitor_tag(tag_id=tag["id"], monitor_id=monitor_id, value="")
print("added:", monitor_id, m["name"])
api.disconnect()
EOF
304-deploy-edge-caddy.sh safely updates CT 108 (Caddy reverse proxy): it makes a backup, validates the new config, and only reloads if it's valid:
cat > /root/ocr-reader/304-deploy-edge-caddy.sh <<'EOF'
#!/bin/bash
# ============================================================
# 304-deploy-edge-caddy.sh
# Run this ON YOUR PROXMOX HOST after scp'ing the edge Caddyfile
# to /root/108_Caddyfile. Backs up CT 108 (Caddy reverse proxy)'s
# live config, pushes the new one, validates, and only reloads if
# valid (restores the backup otherwise).
# ============================================================
set -euo pipefail
CTID=108
STAMP=$(date +%Y%m%d-%H%M%S)
pct exec "$CTID" -- cp /etc/caddy/Caddyfile "/etc/caddy/Caddyfile.bak-$STAMP"
pct push "$CTID" /root/108_Caddyfile /etc/caddy/Caddyfile
if pct exec "$CTID" -- caddy validate --config /etc/caddy/Caddyfile --adapter caddyfile; then
pct exec "$CTID" -- systemctl reload caddy
echo "==> Reloaded. Backup: /etc/caddy/Caddyfile.bak-$STAMP"
else
pct exec "$CTID" -- cp "/etc/caddy/Caddyfile.bak-$STAMP" /etc/caddy/Caddyfile
echo "==> INVALID config - restored backup, Caddy not reloaded." >&2
exit 1
fi
EOF
305-enable-vulkan.sh turns on the Radeon 680M for Ollama (step 5):
cat > /root/ocr-reader/305-enable-vulkan.sh <<'EOF_OUTER1'
#!/bin/bash
# ============================================================
# 305-enable-vulkan.sh
# Run INSIDE CT 300 (ocr-reader), after passing the Radeon 680M
# into the container from the host:
# pct set 300 --dev0 /dev/dri/renderD128,gid=104 --dev1 /dev/dri/card0,gid=44
# pct reboot 300
# Installs the Mesa Vulkan driver (RADV) and turns on Ollama's
# Vulkan backend. Undo: rm the override file + restart ollama.
# ============================================================
set -euo pipefail
apt-get update
apt-get install -y mesa-vulkan-drivers libvulkan1 vulkan-tools
mkdir -p /etc/systemd/system/ollama.service.d
cat > /etc/systemd/system/ollama.service.d/vulkan.conf <<'EOF'
[Service]
Environment=OLLAMA_VULKAN=1
# The 680M is an integrated GPU; Ollama skips those unless told otherwise.
Environment=OLLAMA_IGPU_ENABLE=1
# With the iGPU's 31GB of shared RAM Ollama defaults to a 32K context per
# model; mail prompts are tiny, so cap it to keep both models in memory.
Environment=OLLAMA_CONTEXT_LENGTH=8192
EOF
systemctl daemon-reload
systemctl restart ollama
sleep 3
vulkaninfo --summary 2>/dev/null | grep -E 'deviceName|driverName|apiVersion' || true
EOF_OUTER1
bench-summary.py times the summary step with realistic (uncached) prompts:
cat > /root/ocr-reader/bench-summary.py <<'EOF'
"""Benchmark the summary step (same prompt the app uses) on whatever backend
Ollama is currently using. Run inside CT 300 (ocr-reader):
pct push 300 /root/ocr-reader/bench-summary.py /root/bench-summary.py
pct exec 300 -- /opt/ocr-reader/venv/bin/python /root/bench-summary.py [model]
"""
import random
import sys
import requests
sys.path.insert(0, "/opt/ocr-reader/app")
from app import SUMMARY_PROMPT, SUMMARY_MODEL # noqa: E402
MODEL = sys.argv[1] if len(sys.argv) > 1 else SUMMARY_MODEL
OCR_TEXT = """Pacific Power & Light
PO Box 4500, Portland OR 97208
Jane Sample
123 Example St, Salem OR 97301
Account Number: 4471-2290-18
Statement Date: September 24, 2026
Amount Due: $84.12
Payment Due Date: 0ctober 15, 2026
Please pay by the due date to avoid a $5.00 late fee."""
for run in range(4):
# A unique first line defeats Ollama's prompt cache, like a new letter would.
prompt = f"Scan {run} {random.random()}\n" + SUMMARY_PROMPT + OCR_TEXT
r = requests.post("http://127.0.0.1:11434/api/generate", json={
"model": MODEL, "prompt": prompt, "stream": False,
"keep_alive": "30m", "options": {"temperature": 0, "seed": 1},
}, timeout=600).json()
s = lambda k: r.get(k, 0) / 1e9
print(f"run {run}: total {s('total_duration'):5.1f}s | load {s('load_duration'):4.1f}s | "
f"prompt {r['prompt_eval_count']} tok @ {r['prompt_eval_count'] / max(s('prompt_eval_duration'), 1e-9):6.1f} tok/s | "
f"output {r['eval_count']} tok @ {r['eval_count'] / max(s('eval_duration'), 1e-9):5.1f} tok/s")
print("summary:", r["response"].strip())
EOF
app.py is the backend:
cat > /root/ocr-reader/app.py <<'EOF'
"""OCR mail reader (ocr.503mail.com, CT 300).
Photo of mail -> RapidOCR text -> qwen2.5 summary (Ollama) -> Piper TTS.
"Look closer" re-reads the photo with the qwen2.5vl vision model instead.
Single user. Nothing is stored: images, text and audio live only in memory
for the length of a request, and are never logged.
"""
import base64
import io
import os
import secrets
import threading
import time
import wave
from datetime import timedelta
from functools import wraps
import numpy as np
import requests
from flask import Flask, Response, jsonify, request, send_from_directory, session
from PIL import Image, ImageDraw, ImageOps
from werkzeug.security import check_password_hash
APP_DIR = "/opt/ocr-reader"
STATIC_DIR = os.path.join(APP_DIR, "app", "static")
PASSWORD_FILE = os.path.join(APP_DIR, "password.hash")
SECRET_FILE = os.path.join(APP_DIR, "secret.key")
PIPER_VOICE = os.path.join(APP_DIR, "voices", "en_US-lessac-medium.onnx")
OLLAMA_URL = "http://127.0.0.1:11434"
SUMMARY_MODEL = "qwen2.5:7b"
VISION_MODEL = "qwen2.5vl:7b"
MAX_SIDE = 2000 # downscale photos before OCR
MIN_OCR_SCORE = 0.5 # drop low-confidence OCR boxes
LOGIN_MAX_FAILS = 5
LOGIN_LOCKOUT_SECS = 15 * 60
SUMMARY_PROMPT = """You are reading a piece of postal mail aloud to the person it was sent to. Below is OCR text from a phone photo of it. It may contain OCR errors, barcodes, return addresses and fine print.
Write a short spoken summary: 2 to 4 plain sentences, no lists, no markdown. Cover:
- who it is from and what it is (bill, statement, letter, ad, government notice, etc.)
- any amount due and due date, exactly as written
- what the recipient needs to do, if anything
If it is junk mail or advertising, just say so in one sentence.
Only state facts found in the text. If something important is unclear, say it is unclear instead of guessing.
OCR text:
"""
VISION_PROMPT = """Transcribe all of the text in this photo of postal mail, top to bottom, exactly as written. Include amounts, dates and account numbers exactly. Output only the transcription."""
def _load_secret():
if not os.path.exists(SECRET_FILE):
with open(SECRET_FILE, "w") as f:
f.write(secrets.token_hex(32))
os.chmod(SECRET_FILE, 0o600)
with open(SECRET_FILE) as f:
return f.read().strip()
app = Flask(__name__, static_folder=None)
app.secret_key = _load_secret()
app.config.update(
MAX_CONTENT_LENGTH=25 * 1024 * 1024,
SESSION_COOKIE_SECURE=True,
SESSION_COOKIE_HTTPONLY=True,
SESSION_COOKIE_SAMESITE="Lax",
PERMANENT_SESSION_LIFETIME=timedelta(days=30),
)
# ---------------------------------------------------------------- auth
_login_fails = {} # ip -> [count, first_fail_ts]
_login_lock = threading.Lock()
def _client_ip():
return request.headers.get("X-Forwarded-For", request.remote_addr or "").split(",")[0].strip()
def login_required(fn):
@wraps(fn)
def wrapper(*args, **kwargs):
if not session.get("ok"):
return jsonify(error="Not logged in"), 401
return fn(*args, **kwargs)
return wrapper
@app.post("/api/login")
def login():
ip = _client_ip()
now = time.time()
with _login_lock:
count, first = _login_fails.get(ip, [0, now])
if now - first > LOGIN_LOCKOUT_SECS:
count, first = 0, now
if count >= LOGIN_MAX_FAILS:
return jsonify(error="Too many attempts. Try again later."), 429
if not os.path.exists(PASSWORD_FILE):
return jsonify(error="No password set yet (run set_password.py)."), 503
with open(PASSWORD_FILE) as f:
pw_hash = f.read().strip()
password = (request.get_json(silent=True) or {}).get("password", "")
if not check_password_hash(pw_hash, password):
with _login_lock:
_login_fails[ip] = [count + 1, first]
return jsonify(error="Wrong password"), 401
with _login_lock:
_login_fails.pop(ip, None)
session.permanent = True
session["ok"] = True
return jsonify(ok=True)
@app.post("/api/logout")
def logout():
session.clear()
return jsonify(ok=True)
@app.get("/api/session")
def session_status():
return jsonify(logged_in=bool(session.get("ok")))
# ---------------------------------------------------------------- OCR
_ocr = None
_ocr_lock = threading.Lock()
def get_ocr():
global _ocr
if _ocr is None:
from rapidocr_onnxruntime import RapidOCR
# Explicit thread counts: onnxruntime's default CPU-affinity pinning
# fails inside an LXC and spams the journal with errors.
_ocr = RapidOCR(intra_op_num_threads=4, inter_op_num_threads=1)
return _ocr
def load_image(file_storage):
img = Image.open(file_storage.stream)
img = ImageOps.exif_transpose(img).convert("RGB") # phone photos are often rotated via EXIF
img.thumbnail((MAX_SIDE, MAX_SIDE))
return img
def run_ocr(img):
"""Return OCR text with boxes regrouped into reading-order lines."""
bgr = np.array(img)[:, :, ::-1]
with _ocr_lock:
result, _ = get_ocr()(bgr)
if not result:
return ""
items = []
for box, text, score in result:
if float(score) < MIN_OCR_SCORE:
continue
ys = [p[1] for p in box]
xs = [p[0] for p in box]
items.append((sum(ys) / len(ys), min(xs), max(ys) - min(ys), text))
items.sort()
lines, cur, cur_y, cur_h = [], [], 0, 0
for y, x, h, text in items:
if cur and abs(y - cur_y) > max(cur_h, h) * 0.6:
lines.append(" ".join(t for _, t in sorted(cur)))
cur = []
if not cur:
cur_y, cur_h = y, h
cur.append((x, text))
if cur:
lines.append(" ".join(t for _, t in sorted(cur)))
return "\n".join(lines)
# ---------------------------------------------------------------- LLM
def ollama_chat(model, prompt, images=None, timeout=600):
message = {"role": "user", "content": prompt}
if images:
message["images"] = images
r = requests.post(
f"{OLLAMA_URL}/api/chat",
json={
"model": model,
"messages": [message],
"stream": False,
"keep_alive": "30m",
"options": {"temperature": 0},
},
timeout=timeout,
)
r.raise_for_status()
return r.json()["message"]["content"].strip()
def summarize(text):
if not text.strip():
return "I couldn't find any readable text. Try again with more light and the page flat, or tap Look closer."
return ollama_chat(SUMMARY_MODEL, SUMMARY_PROMPT + text[:8000])
# ---------------------------------------------------------------- TTS
_voice = None
_voice_lock = threading.Lock()
def synthesize(text):
global _voice
with _voice_lock:
if _voice is None:
from piper import PiperVoice
_voice = PiperVoice.load(PIPER_VOICE)
buf = io.BytesIO()
with wave.open(buf, "wb") as wf:
if hasattr(_voice, "synthesize_wav"): # piper-tts >= 1.3
_voice.synthesize_wav(text, wf)
else: # piper-tts 1.2
_voice.synthesize(text, wf)
return buf.getvalue()
# ---------------------------------------------------------------- API
@app.post("/api/scan")
@login_required
def scan():
photo = request.files.get("photo")
if not photo:
return jsonify(error="No photo uploaded"), 400
try:
img = load_image(photo)
except Exception:
return jsonify(error="That file isn't a readable image"), 400
t0 = time.time()
text = run_ocr(img)
t1 = time.time()
try:
summary = summarize(text)
except requests.RequestException:
return jsonify(text=text, summary="", error="Summary model unavailable"), 502
t2 = time.time()
return jsonify(text=text, summary=summary,
ocr_seconds=round(t1 - t0, 1), summary_seconds=round(t2 - t1, 1))
@app.post("/api/look-closer")
@login_required
def look_closer():
photo = request.files.get("photo")
if not photo:
return jsonify(error="No photo uploaded"), 400
try:
img = load_image(photo)
except Exception:
return jsonify(error="That file isn't a readable image"), 400
img.thumbnail((1280, 1280)) # keeps vision-model time sane on CPU
buf = io.BytesIO()
img.save(buf, "JPEG", quality=90)
t0 = time.time()
try:
text = ollama_chat(VISION_MODEL, VISION_PROMPT,
images=[base64.b64encode(buf.getvalue()).decode()])
summary = summarize(text)
except requests.RequestException:
return jsonify(error="Vision model unavailable"), 502
return jsonify(text=text, summary=summary, seconds=round(time.time() - t0, 1))
@app.post("/api/tts")
@login_required
def tts():
text = ((request.get_json(silent=True) or {}).get("text") or "").strip()
if not text:
return jsonify(error="No text"), 400
return Response(synthesize(text[:5000]), mimetype="audio/wav",
headers={"Cache-Control": "no-store"})
@app.get("/healthz")
def healthz():
return "ok"
# ---------------------------------------------------------------- static
@app.get("/")
def index():
return send_from_directory(STATIC_DIR, "index.html")
@app.get("/manifest.webmanifest")
def manifest():
return send_from_directory(STATIC_DIR, "manifest.webmanifest",
mimetype="application/manifest+json")
_icon_cache = {}
@app.get("/icon-<int:size>.png")
def icon(size):
if size not in (180, 192, 512):
return "", 404
if size not in _icon_cache:
img = Image.new("RGB", (size, size), "#111827")
d = ImageDraw.Draw(img)
s = size / 100
d.rounded_rectangle([18 * s, 30 * s, 82 * s, 72 * s], radius=6 * s,
outline="#a78bfa", width=max(2, int(5 * s)))
d.line([(20 * s, 33 * s), (50 * s, 55 * s), (80 * s, 33 * s)],
fill="#a78bfa", width=max(2, int(5 * s)), joint="curve")
out = io.BytesIO()
img.save(out, "PNG")
_icon_cache[size] = out.getvalue()
return Response(_icon_cache[size], mimetype="image/png",
headers={"Cache-Control": "public, max-age=86400"})
EOF
index.html is the phone page:
cat > /root/ocr-reader/index.html <<'EOF'
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover">
<title>Mail Reader</title>
<link rel="manifest" href="/manifest.webmanifest">
<link rel="apple-touch-icon" href="/icon-180.png">
<meta name="apple-mobile-web-app-capable" content="yes">
<meta name="apple-mobile-web-app-title" content="Mail Reader">
<meta name="theme-color" content="#0b0f17">
<style>
:root { --bg:#0b0f17; --card:#151b27; --fg:#e5e7eb; --muted:#9ca3af; --accent:#a78bfa; --err:#f87171; }
* { box-sizing: border-box; }
body { margin:0; background:var(--bg); color:var(--fg);
font:17px/1.5 system-ui, -apple-system, Segoe UI, sans-serif;
padding: max(16px, env(safe-area-inset-top)) 16px max(24px, env(safe-area-inset-bottom)); }
main { max-width: 560px; margin: 0 auto; }
h1 { font-size: 22px; margin: 4px 0 16px; display:flex; justify-content:space-between; align-items:center; }
.hidden { display:none !important; }
.card { background:var(--card); border-radius:14px; padding:16px; margin-bottom:14px; }
button, .scan { font:inherit; border:0; border-radius:12px; cursor:pointer; }
.scan { display:flex; align-items:center; justify-content:center; gap:10px; width:100%;
padding:28px 16px; background:var(--accent); color:#111; font-size:22px; font-weight:700; }
.row { display:flex; gap:10px; margin-top:12px; flex-wrap:wrap; }
.btn { flex:1; padding:14px; background:#252d3d; color:var(--fg); font-weight:600; min-width:120px; }
.btn:disabled { opacity:.45; }
.link { background:none; color:var(--muted); font-size:15px; padding:4px; }
select { width:100%; margin-top:6px; padding:12px; font:inherit; border-radius:10px;
border:1px solid #2a3344; background:#0f141e; color:var(--fg); }
input[type=password] { width:100%; padding:14px; font:inherit; border-radius:10px;
border:1px solid #2a3344; background:#0f141e; color:var(--fg); margin-bottom:12px; }
#status { color:var(--muted); min-height:1.5em; margin:12px 2px; }
#status.err, #loginErr { color:var(--err); }
#summary { font-size:19px; white-space:pre-wrap; margin:0; }
details summary { cursor:pointer; color:var(--muted); }
#fullText { white-space:pre-wrap; font-size:15px; color:#cbd5e1; margin-top:10px; }
.spinner { width:18px; height:18px; border:3px solid #444; border-top-color:var(--accent);
border-radius:50%; animation:spin 1s linear infinite; display:inline-block; vertical-align:-3px; margin-right:8px; }
@keyframes spin { to { transform: rotate(360deg); } }
.alt-link { display:block; text-align:center; color:var(--muted); font-size:15px; margin-top:10px; cursor:pointer; }
.cam { position:fixed; inset:0; background:#000; z-index:10; display:flex; flex-direction:column; }
.cam video { flex:1; width:100%; min-height:0; object-fit:cover; }
.cam-bar { display:flex; align-items:center; justify-content:space-between; gap:10px;
padding:16px 16px max(20px, env(safe-area-inset-bottom)); }
.cam-bar .btn { flex:0 0 auto; min-width:0; padding:12px 14px; }
#camShoot { width:76px; height:76px; border-radius:50%; background:#fff; border:6px solid var(--accent); flex:0 0 auto; }
#camLight.on { background:#facc15; color:#111; }
.cam .alt-link { position:absolute; left:0; right:0; top:max(14px, env(safe-area-inset-top)); color:#ddd;
text-shadow:0 1px 3px #000; margin:0; }
</style>
</head>
<body>
<main>
<h1>Mail Reader <button id="logoutBtn" class="link hidden">Log out</button></h1>
<section id="loginView" class="card hidden">
<form id="loginForm">
<input type="password" id="password" placeholder="Password" autocomplete="current-password" required>
<button class="scan" type="submit" style="padding:16px;font-size:18px">Log in</button>
<p id="loginErr"></p>
</form>
</section>
<section id="appView" class="hidden">
<button class="scan" id="scanBtn">📷 Scan mail</button>
<label class="alt-link" for="photo">or use the phone's camera app</label>
<input type="file" id="photo" accept="image/*" capture="environment" hidden>
<div id="status"></div>
<div id="result" class="card hidden">
<p id="summary"></p>
<div class="row">
<button class="btn" id="playBtn">▶ Play</button>
<button class="btn" id="stopBtn">■ Stop</button>
</div>
<div class="row">
<button class="btn" id="readAllBtn">Read full text</button>
<button class="btn" id="closerBtn">🔍 Look closer</button>
</div>
<details style="margin-top:14px">
<summary>Full text</summary>
<div id="fullText"></div>
</details>
</div>
<div class="card">
<label for="voiceSelect" style="color:var(--muted);font-size:15px">Voice</label>
<select id="voiceSelect"></select>
</div>
</section>
</main>
<!-- In-page camera: lets the page control the phone's light/flash (Chrome on
Android). Safari on iPhone has no torch support - use the camera-app link. -->
<div id="camView" class="cam hidden">
<video id="camVideo" playsinline muted autoplay></video>
<label class="alt-link" for="photo">Use the phone's camera app instead</label>
<div class="cam-bar">
<button class="btn" id="camCancel">Cancel</button>
<button id="camShoot" aria-label="Take photo"></button>
<button class="btn hidden" id="camLight">💡 Light</button>
</div>
</div>
<script>
const $ = id => document.getElementById(id);
const SILENT_WAV = "data:audio/wav;base64,UklGRiQAAABXQVZFZm10IBAAAAABAAEAQB8AAEAfAAABAAgAZGF0YQAAAAA=";
const player = new Audio();
let lastPhoto = null, lastSummary = "", lastText = "", audioUrl = null, busy = false;
function setStatus(msg, opts = {}) {
$("status").className = opts.err ? "err" : "";
$("status").innerHTML = (opts.spin ? '<span class="spinner"></span>' : "") + msg;
}
// iOS/Android only allow audio started from a tap; "unlock" both the
// <audio> element and speechSynthesis during the tap so playback after
// the (slow) server round-trip is allowed.
function unlockAudio() {
player.src = SILENT_WAV;
player.play().catch(() => {});
if ("speechSynthesis" in window) speechSynthesis.speak(new SpeechSynthesisUtterance(""));
}
function stopAudio() {
player.pause();
if ("speechSynthesis" in window) speechSynthesis.cancel();
}
// Voice choice: "server" = Piper on CT 300, otherwise the name of a voice
// installed on this phone (e.g. Eloquence "Reed" on iPhone). Remembered
// per device; first run defaults to Reed if the phone has it.
const SERVER_VOICE = "server";
let voiceChoice = ""; // "" = not decided yet (voice list loads async)
try { voiceChoice = localStorage.getItem("voice") || ""; } catch (e) {}
function deviceVoices() {
return "speechSynthesis" in window ? speechSynthesis.getVoices().filter(v => v.lang.startsWith("en")) : [];
}
function fillVoices() {
const voices = deviceVoices();
if (!voiceChoice && voices.length) {
const reed = voices.find(v => /reed/i.test(v.name));
voiceChoice = reed ? reed.name : SERVER_VOICE;
}
const sel = $("voiceSelect");
sel.innerHTML = "";
sel.add(new Option("Server voice (Piper)", SERVER_VOICE));
voices.forEach(v => sel.add(new Option(`${v.name} (${v.lang})`, v.name)));
sel.value = voiceChoice || SERVER_VOICE;
if (!sel.value) sel.value = SERVER_VOICE; // saved voice no longer installed
}
$("voiceSelect").addEventListener("change", e => {
voiceChoice = e.target.value;
try { localStorage.setItem("voice", voiceChoice); } catch (err) {}
});
fillVoices();
if ("speechSynthesis" in window) speechSynthesis.addEventListener("voiceschanged", fillVoices);
function speakBrowser(text) {
if (!("speechSynthesis" in window)) return;
speechSynthesis.cancel();
const u = new SpeechSynthesisUtterance(text);
const v = deviceVoices().find(v => v.name === voiceChoice);
if (v) { u.voice = v; u.lang = v.lang; }
speechSynthesis.speak(u);
}
// Phone voice if one is picked; otherwise the server's Piper voice, falling
// back to the phone's default voice if Piper fails or takes too long.
async function speak(text) {
stopAudio();
if (!text) return;
if (voiceChoice && voiceChoice !== SERVER_VOICE && deviceVoices().some(v => v.name === voiceChoice))
return speakBrowser(text);
let fellBack = false;
const fallback = setTimeout(() => { fellBack = true; speakBrowser(text); }, 4000);
try {
const r = await fetch("/api/tts", { method: "POST", headers: { "Content-Type": "application/json" },
body: JSON.stringify({ text }) });
if (!r.ok) throw new Error();
const blob = await r.blob();
clearTimeout(fallback);
if (fellBack) return;
if (audioUrl) URL.revokeObjectURL(audioUrl);
audioUrl = URL.createObjectURL(blob);
player.src = audioUrl;
await player.play();
} catch (e) {
clearTimeout(fallback);
if (!fellBack) speakBrowser(text);
}
}
function showResult(data) {
lastSummary = data.summary || "";
lastText = data.text || "";
$("summary").textContent = lastSummary || "(no summary)";
$("fullText").textContent = lastText || "(no text found)";
$("result").classList.remove("hidden");
}
async function send(url, photo, label) {
if (busy) return;
busy = true;
$("closerBtn").disabled = true;
stopAudio();
setStatus(label, { spin: true });
const t0 = Date.now();
try {
const fd = new FormData();
fd.append("photo", photo);
const r = await fetch(url, { method: "POST", body: fd });
if (r.status === 401) return showLogin();
const data = await r.json();
if (!r.ok && !data.text) throw new Error(data.error || "Something went wrong");
showResult(data);
setStatus(data.error ? data.error : `Done in ${Math.round((Date.now() - t0) / 1000)}s`, { err: !!data.error });
speak(lastSummary || lastText);
} catch (e) {
setStatus(e.message || "Couldn't reach the server", { err: true });
} finally {
busy = false;
$("closerBtn").disabled = false;
}
}
function handlePhoto(f) {
lastPhoto = f;
$("result").classList.add("hidden");
send("/api/scan", f, "Reading your mail…");
}
// ---- In-page camera with light/flash control ----
let camStream = null, camTrack = null, lightOn = false;
try { lightOn = localStorage.getItem("light") === "on"; } catch (e) {}
async function openCamera() {
try {
camStream = await navigator.mediaDevices.getUserMedia({ audio: false, video: {
facingMode: { ideal: "environment" }, width: { ideal: 3840 }, height: { ideal: 2160 } } });
} catch (e) {
setStatus("Couldn't open the camera in the page. Use the camera-app link instead.", { err: true });
return;
}
camTrack = camStream.getVideoTracks()[0];
$("camVideo").srcObject = camStream;
$("camView").classList.remove("hidden");
const caps = camTrack.getCapabilities ? camTrack.getCapabilities() : {};
$("camLight").classList.toggle("hidden", !caps.torch);
if (caps.torch) setLight(lightOn);
}
async function setLight(on) {
lightOn = on;
try { localStorage.setItem("light", on ? "on" : "off"); } catch (e) {}
$("camLight").classList.toggle("on", on);
$("camLight").innerHTML = on ? "💡 Light on" : "💡 Light off";
try { await camTrack.applyConstraints({ advanced: [{ torch: on }] }); } catch (e) {}
}
function closeCamera() {
if (camStream) camStream.getTracks().forEach(t => t.stop());
camStream = camTrack = null;
$("camVideo").srcObject = null;
$("camView").classList.add("hidden");
}
function frameToBlob() {
const v = $("camVideo"), c = document.createElement("canvas");
c.width = v.videoWidth;
c.height = v.videoHeight;
c.getContext("2d").drawImage(v, 0, 0);
return new Promise(res => c.toBlob(res, "image/jpeg", 0.92));
}
// Full-resolution still (with real flash if the light is on) where the
// browser supports ImageCapture; otherwise grab the current preview frame.
async function takePhoto() {
if (!("ImageCapture" in window)) return null;
try {
const ic = new ImageCapture(camTrack);
const opts = {};
if (lightOn) {
const pc = await ic.getPhotoCapabilities();
if ((pc.fillLightMode || []).includes("flash")) opts.fillLightMode = "flash";
}
const timeout = new Promise(res => setTimeout(() => res(null), 8000));
return await Promise.race([ic.takePhoto(opts), timeout]);
} catch (e) { return null; }
}
$("scanBtn").addEventListener("click", () => {
unlockAudio();
if (navigator.mediaDevices && navigator.mediaDevices.getUserMedia) openCamera();
else $("photo").click(); // no in-page camera: straight to the camera app
});
$("camLight").addEventListener("click", () => setLight(!lightOn));
$("camCancel").addEventListener("click", closeCamera);
$("camShoot").addEventListener("click", async () => {
unlockAudio();
$("camShoot").disabled = true;
const blob = (await takePhoto()) || (await frameToBlob());
$("camShoot").disabled = false;
closeCamera();
if (blob) handlePhoto(new File([blob], "scan.jpg", { type: blob.type || "image/jpeg" }));
});
// Release our camera stream before the phone's camera app needs the camera.
document.querySelectorAll('label[for="photo"]').forEach(l => l.addEventListener("click", closeCamera));
$("photo").addEventListener("change", e => {
const f = e.target.files[0];
e.target.value = "";
closeCamera();
if (f) handlePhoto(f);
});
$("closerBtn").addEventListener("click", () => {
if (!lastPhoto) return;
unlockAudio();
send("/api/look-closer", lastPhoto, "Looking closer — this can take a minute or two…");
});
$("playBtn").addEventListener("click", () => { unlockAudio(); speak(lastSummary); });
$("readAllBtn").addEventListener("click", () => { unlockAudio(); speak(lastText); });
$("stopBtn").addEventListener("click", stopAudio);
function showLogin() {
$("appView").classList.add("hidden");
$("logoutBtn").classList.add("hidden");
$("loginView").classList.remove("hidden");
$("password").focus();
}
function showApp() {
$("loginView").classList.add("hidden");
$("appView").classList.remove("hidden");
$("logoutBtn").classList.remove("hidden");
}
$("loginForm").addEventListener("submit", async e => {
e.preventDefault();
$("loginErr").textContent = "";
const r = await fetch("/api/login", { method: "POST", headers: { "Content-Type": "application/json" },
body: JSON.stringify({ password: $("password").value }) });
if (r.ok) { $("password").value = ""; showApp(); }
else $("loginErr").textContent = (await r.json()).error || "Login failed";
});
$("logoutBtn").addEventListener("click", async () => {
await fetch("/api/logout", { method: "POST" });
showLogin();
});
fetch("/api/session").then(r => r.json()).then(d => d.logged_in ? showApp() : showLogin())
.catch(() => setStatus("Couldn't reach the server", { err: true }));
</script>
</body>
</html>
EOF
manifest.webmanifest makes it installable on the home screen:
cat > /root/ocr-reader/manifest.webmanifest <<'EOF'
{
"name": "Mail Reader",
"short_name": "Mail Reader",
"start_url": "/",
"display": "standalone",
"background_color": "#0b0f17",
"theme_color": "#0b0f17",
"icons": [
{ "src": "/icon-192.png", "sizes": "192x192", "type": "image/png" },
{ "src": "/icon-512.png", "sizes": "512x512", "type": "image/png" }
]
}
EOF
ocr-reader.service is the systemd unit:
cat > /root/ocr-reader/ocr-reader.service <<'EOF'
[Unit]
Description=OCR mail reader (ocr.503mail.com)
After=network.target ollama.service
Wants=ollama.service
[Service]
User=ocrapp
Group=ocrapp
WorkingDirectory=/opt/ocr-reader/app
Environment=HOME=/opt/ocr-reader
# One worker: OCR/TTS models live in-process; threads handle concurrency.
ExecStart=/opt/ocr-reader/venv/bin/gunicorn -w 1 --threads 4 --timeout 600 -b 0.0.0.0:8000 app:app
Restart=on-failure
[Install]
WantedBy=multi-user.target
EOF
set_password.py sets the login password:
cat > /root/ocr-reader/set_password.py <<'EOF'
"""Set the single-user login password. Run inside CT 300:
pct enter 300
/opt/ocr-reader/venv/bin/python /opt/ocr-reader/app/set_password.py
"""
import getpass
import os
import pwd
from werkzeug.security import generate_password_hash
PASSWORD_FILE = "/opt/ocr-reader/password.hash"
pw = getpass.getpass("New password: ")
if len(pw) < 10:
raise SystemExit("Use at least 10 characters.")
if pw != getpass.getpass("Repeat: "):
raise SystemExit("Passwords don't match.")
with open(PASSWORD_FILE, "w") as f:
f.write(generate_password_hash(pw))
user = pwd.getpwnam("ocrapp")
os.chown(PASSWORD_FILE, user.pw_uid, user.pw_gid)
os.chmod(PASSWORD_FILE, 0o600)
print("Password set. No restart needed.")
EOF
3. Create CT 300 (ocr-reader)
Proxmox host shell. It prints the container's IP at the end.
bash /root/ocr-reader/300-create-ocr-lxc.sh
4. Install everything inside CT 300 (ocr-reader)
Proxmox host shell. This takes about 20 to 30 minutes, mostly the 11GB model download. Output goes to a log file (see Troubleshooting).
pct push 300 /root/ocr-reader/301-provision-ocr.sh /root/301-provision-ocr.sh
pct exec 300 -- bash /root/301-provision-ocr.sh > /root/ocr-reader/provision.log 2>&1; echo "exit code $?"
exit code 0 means it worked. Confirm both models are there:
pct exec 300 -- ollama list
5. Turn on the Radeon 680M graphics (about 2.5x faster)
Proxmox host shell. Pass the GPU into the container. The gid numbers are the render (104) and video (44) groups inside CT 300 (ocr-reader); check them with pct exec 300 -- getent group render video:
pct set 300 --dev0 /dev/dri/renderD128,gid=104 --dev1 /dev/dri/card0,gid=44
pct reboot 300
Install the Vulkan driver and switch Ollama to the GPU:
pct push 300 /root/ocr-reader/305-enable-vulkan.sh /root/305-enable-vulkan.sh
pct exec 300 -- bash /root/305-enable-vulkan.sh
It should list AMD Radeon Graphics (RADV REMBRANDT). Benchmark it; expect about 5s per run on the GPU vs about 12s on CPU:
pct push 300 /root/ocr-reader/bench-summary.py /root/bench-summary.py
pct exec 300 -- /opt/ocr-reader/venv/bin/python /root/bench-summary.py
6. Deploy the app
Proxmox host shell. Re-run this any time the app files change.
bash /root/ocr-reader/302-deploy-ocr-app.sh
7. Set the login password
Inside CT 300 (ocr-reader), using its Console or pct enter 300:
/opt/ocr-reader/venv/bin/python /opt/ocr-reader/app/set_password.py
8. Put it on the internet through CT 108 (Caddy reverse proxy)
Proxmox host shell. First, pull down the live Caddy config:
pct pull 108 /etc/caddy/Caddyfile /root/108_Caddyfile
Add the ocr.503mail.com block, filling in CT 300 (ocr-reader)'s current IP automatically. It does nothing if the block is already there:
grep -q "ocr.503mail.com" /root/108_Caddyfile || cat >> /root/108_Caddyfile <<EOF
# Mail Reader (CT 300 ocr-reader). The app has its own login.
ocr.503mail.com {
request_body {
max_size 25MB
}
reverse_proxy $(pct exec 300 -- hostname -I | awk '{print $1}'):8000
}
EOF
Push it with the backup, validate and reload script:
bash /root/ocr-reader/304-deploy-edge-caddy.sh
Check it from anywhere. It should print ok:
curl -s https://ocr.503mail.com/healthz
9. Add the Uptime Kuma monitors
Proxmox host shell:
pct push 202 /root/ocr-reader/303-add-kuma-monitors.py /root/303-add-kuma-monitors.py
pct exec 202 -- /opt/creator-signup/venv/bin/python /root/303-add-kuma-monitors.py "$(pct exec 300 -- hostname -I | awk '{print $1}')"
10. Install it on the phone
Open https://ocr.503mail.com, log in, then:
- iPhone (Safari): Share → Add to Home Screen.
- Android (Chrome): ⋮ menu → Add to Home screen / Install app.
Camera and light
Scan mail opens a camera view inside the page, with a shutter button and a 💡 Light button.
- Android (Chrome, e.g. Pixel 7): Light keeps the phone's flashlight on while you line up the letter, and fires the real flash when the photo is taken (where the phone supports it). The choice is remembered. If you don't see the Light button, that phone's browser doesn't support controlling the light.
- iPhone: Safari doesn't let web pages use the flashlight, so there's no Light button. Tap "or use the phone's camera app" instead and use the camera app's own flash.
- If the page can't open the camera (permission denied, or no camera), it says so. The camera-app link always works.
- Flash can cause glare on glossy envelopes. If the text near the glare comes out wrong, tilt the letter a little or turn the light off.
Voices
The Voice menu on the page lists "Server voice (Piper)" plus every English voice installed on the phone, and remembers your choice on that phone.
- Eloquence "Reed" on iPhone: Apple includes the Eloquence voices for free. Go to Settings → Accessibility → Spoken Content → Voices → English → Eloquence → download Reed. The page picks Reed automatically the first time if it's installed.
- Server voice (Piper): used when no phone voice is picked. If the server is slow (more than 4 seconds), the page falls back to the phone's default voice.
Troubleshooting: where the logs are
The app (scans, logins, errors)
- Where: the systemd journal inside CT 300 (ocr-reader).
- Open: the Proxmox host shell (or the CT 300 Console without the
pct exec 300 --part).
pct exec 300 -- journalctl -u ocr-reader -n 100 --no-pager
Follow it live while you scan something:
pct exec 300 -- journalctl -u ocr-reader -f
- Look for:
Booting workermeans it started fine. A PythonTracebackis a real error. A few redpthread_setaffinity_np failedlines are harmless: ONNX Runtime can't pin CPU cores inside an LXC. - Restart it:
pct exec 300 -- systemctl restart ocr-reader
The AI models (Ollama)
- Where: the systemd journal inside CT 300 (ocr-reader).
- Open: the Proxmox host shell.
pct exec 300 -- journalctl -u ollama -n 100 --no-pager
See which models are installed, and which are loaded right now:
pct exec 300 -- ollama list
pct exec 300 -- ollama ps
- Look for: in
ollama ps, the PROCESSOR column should say100% GPU. If it says100% CPU, the Radeon isn't being used: check the journal fordropping integrated GPU(the override file from step 5 is missing) and runpct exec 300 -- vulkaninfo --summary(it should listRADV REMBRANDT; if not, the/dev/dripassthrough from step 5 is missing). A healthy start logsinference compute ... library=Vulkan ... type=iGPU.model not foundmeans a model needs pulling again (pct exec 300 -- ollama pull qwen2.5:7borqwen2.5vl:7b). Out-of-memory errors mean CT 300 (ocr-reader) needs more RAM.
HTTPS, certificates and 502 errors (Caddy)
- Where: the systemd journal inside CT 108 (Caddy reverse proxy).
- Open: the Proxmox host shell.
pct exec 108 -- journalctl -u caddy -n 100 --no-pager
- Look for:
certificate obtained successfullymeans HTTPS is working.dial tcp ... connection refused/ a 502 means CT 300 (ocr-reader) or the app is down, or CT 300's DHCP IP changed. Compare its current IP:
pct exec 300 -- hostname -I
with the one in the Caddy config:
pct exec 108 -- grep -A6 ocr.503mail.com /etc/caddy/Caddyfile
If they differ, redo step 8 (the backup files are /etc/caddy/Caddyfile.bak-* on CT 108).
The install (step 4)
- Where: a file on the Proxmox host,
/root/ocr-reader/provision.log. - Open: the Proxmox host shell.
tail -n 50 /root/ocr-reader/provision.log
Uptime Kuma
- Where: https://status.503mail.com. The monitors are "ocr.503mail.com — Mail Reader" (the public site) and "…:8000 — Mail Reader LXC" (the app port on CT 300 (ocr-reader)), both tagged Personal Site.
- Read it as: if the public check is down but the port check is up, it's Caddy or the network. If both are down, it's CT 300 (ocr-reader).
Quick health check
curl -s https://ocr.503mail.com/healthz
pct exec 300 -- systemctl is-active ocr-reader ollama
Notes
- Measured on a sample bill: OCR 0.9s, summary 5.1s on the Radeon 680M (12.0s on the CPU alone), Piper 1.2s. Look closer: 3.5s warm, 19s cold on the 680M (57s cold on the CPU alone).
- Benchmark gotcha: re-sending the exact same prompt hits Ollama's prompt cache and looks far faster than a real new letter.
bench-summary.pyadds a random first line to every run so the numbers are realistic. - The summary model is told to only state facts from the text. The full OCR text is always shown under the summary, so amounts can be double-checked.
- Change the password any time by re-running step 7. Redeploy after editing the app with step 6. To benchmark the summary speed, run
bench-summary.py(step 5).