This is the semantic bridge the whole avatar estate was building toward: a live system that understands what the conversation means, chooses a visual representation, and expresses it through animated sketch lines, ASCII/glyph fields, dots, color and morphing forms — while the voice only drives energy, never content. Everything below is running now at voice.netshow.ai/alive-site/semantic-stage.html.
Open the Semantic Stage dashboard → Freestyle Stage (the ancestor)Decides what the scene means. Phrase and topic events become compact, validated VisualPlanPatch JSON.
Decides how strongly it moves. The certified AvatarEngine turns the live stream into envelope + 14 visemes + blink/gaze/brow.
Deterministic local rendering only. Seeded boil, minimum-jerk morphs, glyph transport — no LLM, no fetch, no randomness in the loop.
The one rule that makes it intelligent AND stable: audio may shake the scene, only meaning may change it.
Interruption law: one frame — barge-in cancels every unplayed beat, zeroes articulation, keeps the scene. Session loss → the world stays alive and settles calm. The LLM is never in the frame loop.
| Existing piece | Where | What the stage takes from it |
|---|---|---|
| AliveBridge | alive-site/alive-bridge.js | session lifecycle, five states, caption stream, wordSinks fan-out, ontool page-tool hook, interrupt law — consumed, never edited |
| LiveAgentSource | avatar-lab/js/sources.js | mint → mic → WebRTC → data channel → barge → /end beacon (frozen, certified) |
| AvatarEngine | avatar-lab/js/engine.js | the whole audio-feature bus: envelope, MFCC visemes, blink/gaze/brows — analysis-only tap |
| Figure registry (12 beings) | alive-site/freeform-figures.js | every curated colored-ASCII archetype parses straight into the glyph layer as a morphable subject, open face zone included |
| Freestyle-stage laws | alive-site/freestyle-stage.html | session.update tool merge, dc tap pattern, exactly-once tool replies, rate windows, caption letterbox, $0 tour pattern |
| Sketch-line taste ladder | HybridFace / sketch-studio | 2-pass jittered ink, per-stroke boil phases (~90 ms), tapered width, seeded everything — frozen ink reads stiff, so nothing is frozen |
suite: test-alive-site/semantic-stage.cjs — 60 checks green (units · scene shots · morph determinism · runtime · tool transport)
Every figure keeps an open face zone where the live sketch eyes, brows and mouth keep articulating with the voice — the freestyle-stage magic, now morphable. Unknown subjects get an honest stylized treatment, never a fake likeness. Any scene can hold the presenter beside it (hybrid mode).
Matched strokes — resampled to 48 points, orientation chosen by correspondence error, interpolated with minimum jerk:
Matching is deterministic: cost = centroid distance + length mismatch + layer distance − role affinity (exact role −0.45, same family −0.22). Unmatched outgoing ink lingers then dissolves; unmatched incoming ink draws itself progressively — background layers first.
Glyph cells travel as stable-identity particles (spatial-hash nearest assignment, capped at 1600 transports), swap characters exactly once mid-flight, lerp color continuously, and stagger on seeded delays — no shimmer, ever. The continuity rule is carried by roles: hair strokes become ridge lines, contours become branches, foam becomes clouds.
| Criterion | Status |
|---|---|
| Semantic events change the subject without restarting the page | PROVEN |
| Avatar → mountain → tree → person → beach → diagram → avatar morph chain | PROVEN (mock tour + shots) |
| Audio changes motion/articulation but never chooses content | PROVEN (suite-pinned) |
| Word-anchored beats land on their words; interruption cancels future beats, keeps scene | PROVEN (suite-pinned) |
| No per-frame LLM calls; deterministic seeded rendering; shot-mode reproducibility | PROVEN (pixel-identical reloads) |
| Glyph fields temporally stable; sketch lines boil on purpose, never rerandomize | PROVEN |
| Five states + alert overlay + graceful session loss | PROVEN |
| Arbitrary subjects beyond presets (desert+camel, whale underwater, dreamed forms) | PROVEN (V2: composition + draw primitives + needs-draw nudge; suite-pinned) |
| Expressive emotional face (squint/sass/surprise, gestures, sentiment) | PROVEN (V2 expression engine; shots + suite) |
| Named person → reference-resolved exact likeness | STYLIZED PATH ONLY (honest fallback; reference compiler is the next lane) |
| SDF silhouette morphs for unrelated closed shapes | COVERED BY stroke+transport in v1; SDF lane documented, not built |
| Live model firing emit_scene_patch in a paid call | TRANSPORT PROVEN WITH FAKE DC — first paid drill awaits Michael's word (spend gate) |
Michael's verdict on v1: "it's preset, it's in a box… you got to release that capability of dreaming." V2 removes the box with three layers, all live now:
1 · COMPOSE — entities:[{kind, at:{x,y,w}, motion}] places up to four beings INSIDE any scene with deterministic motion (walk/swim/fly/drift): a camel crossing the desert, the whale swimming under the water — the exact scenes v1 couldn't make. 18 fleet-authored entities (camel, horse, dog, fish, bird, boat, butterfly, person, house, car, rocket, jellyfish, turtle, crab, snake, flower, cactus, kite) + 5 new environments (desert, underwater, meadow, storm, sunset) + every v1 preset doubles as an entity.
2 · DREAM — draw:[…] primitives let the model AUTHOR anything from its own visual memory: control-point strokes (auto-smoothed), rings, glyph blobs, texture bands, dot fields — validated, capped, compiled into the same living ink as the presets. The mock tour dreams a hot-air balloon live; the model gets the identical path.
3 · THE NUDGE — an unknown kind never dead-ends: the tool replies needs-draw with authoring instructions and opens the dream window; when a visitor asks for an unknown being, the stage whispers a system note telling the model to draw it. The lockout is structurally impossible now.
Plus the emotional face: a six-channel expression engine (joy, sass, squint, surprise, concern, curiosity) with a real blink machine, saccades, speech-accent brow flicks, sentiment hearing, and gestures (wink, tongue, brow-raise, nod) — settable by the model (expression/gesture patch fields), by phrase sentiment, by voice state (thinking squints), and by dashboard chips.
V4 — THE REFERENCE VISUAL CORTEX (the real unlock): "see the image, don't produce the image." For ANY kind with no preset — a lion, Donald Trump, the Eiffel Tower — the stage now resolves ONE small reference image (Wikipedia thumbnails: keyless, CORS-open, encyclopedic), looks at it ONCE on the semantic clock, flood-mattes the background (per-side medians + protected center), and emulates the subject as living glyph cells — honestly labeled with its source. Nothing is rejected anymore: unknown kinds apply instantly as a name-card and morph into the conjured emulation seconds later; the model is told "keep talking, it's forming." The V3 needs-draw rejection (which taught the model to retreat to presets) and the mid-response response.create pileups (the slowness) are both gone; the tool schema and stage briefing are ⅓ their old size. Persona requests conjure the person locally and the model speaks in character.
V3 — THE VISUAL CORTEX SEED (same day, after the Caesar test failed live): the model's NATIVE dream language is glyph art, not coordinates. art on any subject/entity accepts {#RRGGBB}-painted ASCII (the proven become_form format) and compiles into a first-class subject; with faceBox, the LIVE animated face rides the portrait's open zone — so "take me back to Rome, I want to talk to Caesar" = colosseum backdrop + a Caesar portrait the model authors itself + its own living face inside, speaking in character. Persona intents ("talk to / meet / pretend to be X") are detected and nudged as BECOME instructions; place intents ("take me to X") widen the command grammar. The 🏛 chip and the mock tour carry a curated Caesar as the deterministic proof.