kinescript

Characters that talk.

An avatar is a character drawn once, as SVG parts or as a glTF model, in a folder with an avatar.json. A scene gives it a line, and its mouth follows the words. It blinks, breathes and glances on its own, and smiles, points and looks where the scene says, each on a cue.

The built-in host presents a chart

"avatar": "kine:host" is the starter: a plain figure in the scene's palette with all nine mouth shapes, speaking with Kokoro. The line is text in the scene; kine voice speaks it, and its marked words cue the chart, the glance at it, the point and the surprise.

0.0 s
  1. sales
  2. host
  3. host-say
avatar-host.jsonthe whole scene
{  "canvas": {"w": 1920, "h": 1080, "fps": 30, "duration": "auto", "bg": "$void", "poster": "host-say:third"},  "layers": [    {"use": "barChart", "id": "sales", "data": [["Q1", 42], ["Q2", 58], ["Q3", 91], ["Q4", 64]],     "x": "66%", "y": "52%", "w": 980, "h": 600, "at": "host-say:sales", "until": "end"},    {"id": "host", "use": "avatar", "avatar": "kine:host", "x": "20%", "y": "58%", "h": 900,     "until": "host-say.out+1.5",     "say": {"text": "Hi, I'm the host. [[sales]]Sales grew all year, and the [[third]]third quarter was the best one. Write the line, and I [[say]]say it."},     "expressions": [{"name": "smile", "at": "host-say.in", "dur": 1.2},                     {"name": "surprise", "at": "host-say:third", "dur": 1.2},                     {"name": "happy", "at": "host-say:say"}],     "gaze": [{"at": 0, "target": "camera"}, {"at": "host-say:sales-0.2", "target": "#sales"},              {"at": "host-say:say-0.3", "target": "camera"}],     "gestures": [{"name": "wave", "at": "host-say.in"}, {"name": "point", "at": "host-say:third"},                  {"name": "nod", "at": "host-say:say"}]}  ]}

A presenter from generated pictures

Maya began as fourteen pictures from an image model: one portrait, then edits of it that change only the mouth or the eyes. kine avatar from-images keyed out the green background, lined each picture up with the portrait, and turned what each one changed into a patch over it. She talks with nine mouth shapes, blinks, looks at the title, and smiles and looks surprised on cue.

0.0 s
  1. card
  2. sub
  3. maya
  4. maya-say
avatar-maya.jsonthe whole scene
{  "canvas": {"w": 1920, "h": 1080, "fps": 30, "duration": "auto", "bg": "$void", "poster": "maya-say:look+0.4"},  "layers": [    {"id": "card", "type": "text", "text": "Pictures in.\nA presenter out.", "x": "70%", "y": "42%", "size": 96,     "weight": 800, "at": "maya-say:lined", "until": "end", "enter": {"name": "rise", "dur": 0.4}},    {"id": "sub", "type": "text", "text": "kine avatar from-images", "x": "70%", "y": "60%", "size": 44,     "font": "IBM Plex Mono", "fill": "$accent2", "at": "maya-say:lined+0.5", "until": "end", "enter": "fade"},    {"id": "maya", "use": "avatar", "avatar": "../../examples/avatars/maya", "x": "27%", "y": "56%", "h": 1180,     "until": "maya-say.out+2",     "say": {"text": "Hi, I'm Maya. I started as fourteen pictures from an image model. Kinescript [[lined]]lined them up, and now I [[talk]]talk, I blink, and I [[look]]look at your work."},     "expressions": [{"name": "happy", "at": "maya-say.in", "dur": 1.6},                     {"name": "surprise", "at": "maya-say:lined", "dur": 0.8},                     {"name": "happy", "at": "maya-say.out", "dur": 1.5}],     "gaze": [{"at": 0, "target": "camera"}, {"at": "maya-say:look", "target": "#card"},              {"at": "maya-say.out-0.3", "target": "camera"}],     "gestures": [{"name": "nod", "at": "maya-say:talk"}, {"name": "tilt", "at": "maya-say.out"}]}  ]}

Reactions that show on cue

Rin came from fifteen generated pictures, like Maya, plus four effects drawn as SVG: sparkles, a sweat drop, an anger mark and surprise lines. A part marked "effect": true stays hidden until an expression or gesture names it, so each one shows on its word and goes again.

0.0 s
  1. card
  2. sub
  3. rin
  4. rin-say
avatar-rin.jsonthe whole scene
{  "canvas": {"w": 1920, "h": 1080, "fps": 30, "duration": "auto", "bg": "$void", "poster": "rin-say:spark+0.5"},  "layers": [    {"id": "card", "type": "text", "text": "Reactions,\non cue.", "x": "70%", "y": "42%", "size": 104,     "weight": 800, "at": "rin-say:spark", "until": "end", "enter": {"name": "rise", "dur": 0.4}},    {"id": "sub", "type": "text", "text": "\"effect\": true", "x": "70%", "y": "60%", "size": 44,     "font": "IBM Plex Mono", "fill": "$accent2", "at": "rin-say:spark+0.5", "until": "end", "enter": "fade"},    {"id": "rin", "use": "avatar", "avatar": "../../examples/avatars/rin", "x": "27%", "y": "56%", "h": 1180,     "until": "rin-say.out+2",     "say": {"text": "Hi, I'm Rin! I'm drawn anime style, so I react like it. Good news? [[spark]]Sparkles! Too much to do? [[sweat]]A sweat drop. A failed render? [[fume]]I fume. Then kine check tells me why, and I'm [[fine]]fine."},     "expressions": [{"name": "happy", "at": "rin-say.in", "dur": 1.4},                     {"name": "joy", "at": "rin-say:spark", "dur": 1.3},                     {"name": "nervous", "at": "rin-say:sweat", "dur": 1.6},                     {"name": "annoyed", "at": "rin-say:fume", "dur": 1.3},                     {"name": "happy", "at": "rin-say:fine", "dur": 2.0}],     "gaze": [{"at": 0, "target": "camera"}, {"at": "rin-say:spark+0.2", "target": "#card"},              {"at": "rin-say:sweat", "target": "camera"}],     "gestures": [{"name": "sparkle", "at": "rin-say:spark"}, {"name": "sweat", "at": "rin-say:sweat"},                  {"name": "fume", "at": "rin-say:fume"}, {"name": "shake", "at": "rin-say:fume+0.1"},                  {"name": "nod", "at": "rin-say:fine"}]}  ]}

Two drawn characters, talking to each other

Cue is the Kinescript mascot: a robot whose mouth is a waveform and whose antenna pings. Slate is a clapperboard whose clapper is its jaw, and on "cut" its board says so. Both are SVG parts drawn by brand/make_mascots.py, coloured by palette roles so a brand recolours them. Each line starts at a cue in the other's: "at": "cue-say1.out+0.3".

0.0 s
  1. title
  2. cue
  3. slate
  4. cue-say1
  5. cue-say2
  6. slate-say1
  7. slate-say2
avatar-duo.jsonthe whole scene
{  "canvas": {"w": 1920, "h": 1080, "fps": 30, "duration": "auto", "bg": "$void", "poster": "slate-say2:cut+0.3"},  "layers": [    {"id": "title", "type": "text", "text": "Drawn in SVG. Coloured by the brand.", "x": "50%", "y": "11%", "size": 56,     "weight": 700, "at": "cue-say1.in", "until": "end", "enter": "fade"},    {"id": "cue", "use": "avatar", "avatar": "../../examples/avatars/cue", "x": "31%", "y": "64%", "h": 780,     "until": "end",     "say": [{"text": "Hi, I'm Cue. My mouth is a [[wave]]waveform, and my antenna [[ping]]pings on every cue.",              "at": "slate-say1.out+0.3"},             {"text": "That's a [[wrap]]wrap.", "at": "slate-say2.out+0.4"}],     "expressions": [{"name": "wow", "at": "slate-say2:cut", "dur": 1.0},                     {"name": "happy", "at": "cue-say2.in", "dur": 1.8}],     "gaze": [{"at": 0, "target": "right"}, {"at": "cue-say1.in", "target": "camera"},              {"at": "slate-say2.in", "target": "right"}, {"at": "cue-say2.in", "target": "camera"}],     "gestures": [{"name": "wave", "at": "cue-say1.in"}, {"name": "ping", "at": "cue-say1:ping"},                  {"name": "bounce", "at": "cue-say2:wrap"}]},    {"id": "slate", "use": "avatar", "avatar": "../../examples/avatars/slate", "x": "69%", "y": "60%", "h": 780,     "until": "end",     "say": [{"text": "Scene one, take one. [[snap]]Action!", "at": 0.4},             {"text": "I'm Slate. Every word is a little clap. And when the take is done... [[cut]]cut!",              "at": "cue-say1.out+0.3"}],     "expressions": [{"name": "cut", "at": "slate-say2:cut", "dur": 1.3},                     {"name": "happy", "at": "cue-say2.in", "dur": 1.8}],     "gaze": [{"at": 0, "target": "camera"}, {"at": "cue-say1.in", "target": "left"},              {"at": "slate-say2.in", "target": "camera"}],     "gestures": [{"name": "snap", "at": "slate-say1:snap"}, {"name": "hop", "at": "cue-say2:wrap"}]}  ]}

A 3D avatar from a glTF file

Its mouth shapes, blink and expressions are the model's morph targets by name, its gestures are the model's animation clips, and a gaze turns the model. This head is generated by examples/make_media.py, so the example needs no downloaded model.

0.0 s
  1. card
  2. sub
  3. sam
  4. sam-say
avatar-3d.jsonthe whole scene
{  "canvas": {"w": 1920, "h": 1080, "fps": 30, "duration": "auto", "bg": "$void", "poster": "sam-say:make+0.6"},  "layers": [    {"id": "card", "type": "text", "text": "Make it move.", "x": "68%", "y": "46%", "size": 130, "weight": 800,     "at": "sam-say:make", "until": "end", "enter": {"name": "rise", "dur": 0.4}},    {"id": "sub", "type": "text", "text": "A glTF head, its mouth and face as morph targets.", "x": "68%",     "y": "58%", "size": 40, "fill": "$muted", "at": "sam-say:make+0.5", "until": "end", "enter": "fade"},    {"id": "sam", "use": "avatar", "avatar": "../../examples/media/head", "x": "28%", "y": "54%", "h": 860, "until": "sam-say.out+3",     "say": {"text": "Hi, I'm Sam. I'm a [[gltf]]glTF head. Write a line, and I'll [[make]]make it move."},     "expressions": [{"name": "smile", "at": "sam-say.in", "dur": 1.4},                     {"name": "surprise", "at": "sam-say:gltf", "dur": 0.9},                     {"name": "smile", "at": "sam-say:make", "weight": 0.8}],     "gaze": [{"at": 0, "target": "camera"}, {"at": "sam-say:make-0.2", "target": "#card"},              {"at": "sam-say.out", "target": "camera"}],     "gestures": [{"name": "nod", "at": "sam-say:write"}]}  ]}

Every pose on one sheet: kine look FOLDER

An agent cannot watch a mouth move, so the sheet shows each mouth shape (and what a missing one falls back to), the eyes closed and looking aside, each expression, and each gesture where it moves most. A broken pivot or a missing shape shows here before a render. kine avatar check FOLDER lists the same problems as text.

The built-in host's pose sheet: neutral, the nine mouth shapes, eyes closed, looking left and right, the five expressions and the six gestures, each in its own labelled cell.
The built-in host: nine mouths, five expressions, six gestures.
The 3D head's pose sheet: the mouth shapes as morph weights, eyes closed, looking left and right, smile and surprise, and the nod and shake clips.
The 3D head: the same poses, as morph weights and clips.
Maya's pose sheet: the nine mouth shapes, eyes closed, looking left and right, the happy and surprise expressions and the gestures, all from generated pictures.
Maya: every pose from generated pictures.
Rin's pose sheet: the nine mouth shapes, eyes closed, looking left and right, the happy, joy, surprise, nervous and annoyed expressions, and the gestures, with sparkles, surprise lines, a sweat drop and an anger mark only in the poses that show them.
Rin: effects show only in the poses that name them.
Cue's pose sheet: nine waveform mouths, eyes closed and looking aside, the happy, wow, error and think expressions with sparks and an alert mark, and the nod, shake, wave, ping and bounce gestures.
Cue: a waveform for a mouth.
Slate's pose sheet: the clapper open at nine angles as mouth shapes, eyes closed and looking aside, the cut expression with squeezed eyes, CUT on the board and motion lines, happy with stars, wow and think, and the snap, hop, nod and shake gestures.
Slate: the clapper is the jaw.
Mouths from words
A voice's word times become mouth shapes by spelling rules, with Kokoro's phonemes where it gives them. kine probe --mouth shows the track; kine check says when a line has no word times and the mouth only follows loudness.
Idle life
Blinks, breathing and small glances, timed from the layer's id and seed, so every render of a frame is the same. No blink lands as a line starts or at a gesture's peak.
Acting on cues
expressions, gestures and gaze are lists on the layer, each at a cue such as "host-say:third". Kinescript checks them; you write them.
From pictures
kine avatar from-images FOLDER turns a portrait and edits of it (a blink, mouth shapes, looks and expressions) into an avatar. --init writes the prompts for each picture, and the report names any picture that changed more than it should.
Effects
A part with "effect": true is hidden until an expression or gesture names it: sparkles, a sweat drop, motion lines. fx-NAME.svg next to the pictures becomes one.
Your own avatar
kine avatar new FOLDER copies the host as SVG parts to redraw. Colours can be palette roles, so an avatar takes the scene's brand, and a brand can name avatars and their voices.
Recorded speech
say also takes a sound in audio, recorded or spoken, with a words file, so a real voice-over drives the mouth too.
Cost
A drawn avatar costs little more than a still picture of it. A 3D head costs what any model of its size costs, moving or not.