How a voice-following teleprompter actually works

Most “voice-activated” prompters only listen for noise. A true voice-following prompter transcribes what you say and finds those words in the whole script, so it can wait, skip and go back with you. You can tell the two apart in about sixty seconds.

What “voice-activated” usually means

The phrase gets used for three different mechanisms, and the difference decides whether a teleprompter helps you or fights you.

The oldest mechanism is a motor: the text crawls at a fixed speed and you keep up. The second is sound activation: the text moves while the microphone hears something and stops when it goes quiet. The third, and the one we built gotoprompter around, is word tracking: the app converts your speech to text and matches those words against the script.

All three are sometimes marketed as “voice controlled”. Only the third one knows where you are.

Sound detection

Sound-activated scrolling is simple and robust. A level meter watches the microphone; above a threshold, the script advances at a preset speed; below it, the script pauses. It needs no speech recognition, so it works in any language and with any accent, and it never mishears you because it never listens to words at all.

The limits are just as simple. It cannot tell the difference between you reading the script and you saying “sorry, can you hear me now?” to a colleague. It cannot tell that you skipped a paragraph. And the pace is still a pace you set in advance, so a sentence you deliver slowly drifts behind the highlight and a sentence you rush drifts ahead.

We keep Sound mode in both apps because it is the right tool for some rooms: a noisy trade-show floor, a language the recognizer does not support, or a person who simply prefers a steady crawl that stops when they stop.

Word tracking

Word tracking replaces the level meter with a speech recognizer. As you talk, the recognizer produces a stream of words, usually a few at a time, often revising the last one or two as more audio arrives. The prompter has to decide, several times a second, which word in the script you just said.

Doing that well is harder than it sounds. Recognizers make mistakes, people say “um”, numbers come out as digits when the script has words, and the same short word can appear forty times on one page. A naive implementation that looks for the next word in line falls apart the moment you ad-lib a sentence.

Our engine tokenizes the script once, normalizes both sides (lower case, punctuation stripped, numbers expanded), and then matches the most recent spoken words against a window of the script around the current position. It prefers the nearest plausible match, ignores words that are too common to be evidence on their own, and refuses to jump on a single matching word. When the near window finds nothing for a while, it widens the search.

Whole-script matching

The difference between a good and a great voice prompter is what happens when you leave the expected path.

Say you are reading a product demo and a customer asks a question. You answer for a minute, then come back to the script two paragraphs further down because the question already covered the middle part. A near-window matcher is still waiting for the sentence you abandoned. A whole-script matcher recognizes three consecutive words from the new paragraph, confirms they appear nowhere nearer, and jumps the highlight to where you are.

That is why we match against the entire script, not just the next line, and why the jump requires a short run of matching words rather than one. One word is a coincidence; three in a row in the same place is you.

Backward scrolling

Going back is the test most prompters fail. Repeating a sentence because you fluffed it is the most common thing a presenter does, and a prompter that only moves forward leaves the highlight stranded on the next line while you re-read the previous one.

Our matcher treats backward matches as first-class. If the words you just said appear a few lines up, it moves the highlight up. The only asymmetry is caution: because scripts repeat phrases, a long backward jump needs more matching words than a short forward step before we trust it.

The same mechanism powers voice commands. “Go back” moves to the start of the previous paragraph, “go next” to the next one, and “go start” or “go finish” to either end. They work because the engine already knows paragraph boundaries and the exact position of every word.

How to test an app in 60 seconds

You do not have to trust a feature list. Paste any script of ten short paragraphs into the app you are evaluating and run through these four moves:

1. Pause mid-sentence for five seconds. The highlight should freeze, not creep.
2. Say something off-script for a sentence or two, then resume. The highlight should stay put during the ad-lib and pick up where you resume.
3. Skip two paragraphs and start reading there. Within a few words the highlight should jump forward to you.
4. Repeat the previous sentence. The highlight should move back to it.

A sound-activated prompter passes the first test and fails the other three. A next-line word tracker passes the first two. A whole-script tracker passes all four. Whichever you pick, you will know what you bought.

Frequently asked questions

Does word tracking need an internet connection? It depends on where the recognizer runs. Our Mac app uses Apple’s on-device recognition and works offline. The web app uses the browser’s speech engine; Safari and recent Chrome versions process most languages on the device, while some browsers may send audio to the vendor’s service.

Why does the highlight sometimes lag by a word or two? Recognizers wait for a little more audio before committing to a word, so there is always a short delay. A good prompter compensates by keeping the active line slightly above the centre so the next words are already visible.

What happens with names and jargon the recognizer gets wrong? The matcher ignores words it cannot find and keeps going on the ones it can. A script full of unusual names still tracks as long as the ordinary words between them line up. If a passage is nothing but jargon, Sound mode is the fallback.

Can I mix languages in one script? Put the foreign words in square brackets so they are shown but not matched, and the sync holds while you speak your main language.

---

Related: how-to-read-a-script-without-looking-like-youre-reading · teleprompter-in-60-languages · best-teleprompter-app-for-mac