Development notes

I built an all-day recording system for Mac, too

Xuesheng Guo

This article preserves the product version, test conditions and views at the time of writing. Features, pricing, data handling and download options may have changed; consult the current product page. Original illustrations and screenshots were not supplied.

Captions appear while recording continues, and earlier text begins to finalize before the recording ends. I built a long-duration Mac recording system for my own recording tool.

In the previous article, I described saving Apple Watch audio in segments and sending it progressively to iPhone for transcription. After the watch, I turned to the Mac. At a computer, sound may come from a microphone or the computer itself: discussions, online calls and passing ideas. Could a recorder stay quietly nearby, then make the audio easy to find, organize and replay?

I added a long-recording system to TalkTuck for Mac. The original article showed the real recording screen with waveform, source switches and batch finalization progress; the live transcript below it could still contain recognition errors. That image was not included with the supplied text.

The first approach was simple: record, show live text, and transcribe the entire audio again after stopping. It worked for short recordings, but longer sessions raised questions. Would memory keep growing with the audio? Could we improve imperfect live text earlier? Must hours of audio wait until stop before finalization begins? How do we avoid a wall of text or a single sentence being wrongly split among speakers?

I divided the work into four parts: reliably preserve audio, make recording unobtrusive, organize text in stages, and then make the results usable by people or agents.

Save the sound before transcribing

A Mac has more sound sources than a watch. I want microphone recording, system audio, or both together for online discussions. The sources are saved separately during capture against the same timeline. This preserves evidence: if a device disconnects, I can tell which source failed, and later align them for mixed playback or transcription.

Audio does not accumulate indefinitely in memory. Following the segment approach, normalized audio is sealed into AAC segments approximately every 10 seconds. Each segment has recording identity, order, start position, valid frame count and verification information. A final manifest records the expected segments. The original technical diagram showing these separate sources and checks was not provided with the text.

The manifest separates audio already written from unfinished material. After interruption, the app can inspect saved segments and attempt recovery, while preserving incomplete or questionable files. My priority remains: capture sound first, then let text catch up.

Segmented storage does not mean audio can never be lost. Permissions, disconnected input devices, storage space and system state can affect capture. The app can only try to detect problems, preserve what has already been written, and leave evidence for recovery.

Keep recording while the window gets smaller

Unlike on the watch, I'm usually doing other work on the Mac while recording. Keeping the main window open to monitor status can cover the material I'm working with. So I built a mini floating recording bar with recording state, pause, resume, stop and live captions.

Expand the main window when you need the full transcript; browse older recordings without first stopping the current one. Sources can be switched during recording: microphone, system audio, or both. The original article included a real mini-bar screenshot; live captions can still be revised and the screenshot was not provided with the text.

One boundary: system audio means audio captured from the computer, not only from a particular meeting app. What is captured depends on permissions, devices and software. Recording does not continue after the Mac sleeps or shuts down.

A small window does not automatically mean less background work. I throttled refresh in mini and hidden states: short live captions still appear without redrawing the entire long transcript every time; the waveform updates at most twice per second, and stops refreshing while the window is hidden.

This mainly reduces interface overhead. Recognition still runs, so I won't claim the floating window has proven low power consumption.

Three stages of text: see it, correct it, finalize it

Live captions and a final transcript have different timing needs. During recording, I want words to appear soon after they are spoken; waiting minutes defeats live preview. Better recognition can require more context.

The first stage is live captions. After recording begins, streaming recognition displays what it hears. Mandarin and English use lightweight streaming recognition; Cantonese also retains live captions. This stage favors speed and allows later changes.

The second stage is quick correction. For Mandarin and English, live recognition accumulates complete utterances and aims to submit them for stronger recognition after about 30 seconds. It does not cut a sentence every time a 30-second timer fires: I try to submit complete utterances, with a maximum batch of 54 seconds. If correction cannot keep up, the original live text remains rather than letting pending audio grow without bound. Usable text stays visible, and later results update it when available.

The third stage is batch finalization. Previously, the entire recording was processed again after stopping. Now an earlier eligible batch can begin finalizing while recording continues. In the calibrated policy for devices with 16 GB or more, Mandarin batches are at most 480 seconds and English batches at most 360 seconds. Near the limit, the system prefers a recent silence boundary; if none is suitable, it retains three seconds of context at the boundary to help join the two sides. Cantonese and lower-memory devices continue using more conservative policies.

After stopping, completed batches are reused, in-progress batches finish and the tail is processed. A long recording need not restart all completed work merely because Stop was pressed. The stages have distinct roles: live captions prioritize timeliness, quick correction improves interim readability, and batch finalization performs fuller recognition and organization. All depend on the same original audio; lag in one stage does not cause its deletion. The original three-stage diagram was not included with the text.

Which models did I use?

Instead of asking one model to do everything, I chose a model for each task:

Qwen and Sortformer run through MLX on Mac; streaming Paraformer uses ONNX Runtime. Models must first be prepared locally and recognition runs locally. Whether account sync is enabled afterwards is a separate control. Each model has limits; combining them cannot guarantee every word or speaker is correct.

Batches can split; sentences and speakers should not be split arbitrarily

A processing boundary is not necessarily a sentence boundary. If a sentence crosses batches, naive concatenation may duplicate or omit words near the boundary. If speaker recognition restarts numbering for each batch, the same person may appear as different people.

Each batch records the original-audio interval it owns, with adjacent intervals meeting end to end. Where overlap is needed, I compare returned word-level times with earlier confirmed text. If uncertain, I retain candidates and original audio rather than silently deleting words.

Speaker state carries into the next batch so one recording can use a consistent set of speaker identities where possible. Layout considers attribution of complete sentences before paragraph breaks: for the same speaker, Chinese uses roughly 50 characters and sentence-ending punctuation as guides; pauses over five seconds break a paragraph; confirmed speaker changes start another. English preserves complete words and considers sentence punctuation, quotation marks, decimals and abbreviations across languages.

I once saw a standalone Chinese “I” split into a line because its timestamp wasn't aligned. The current approach tries to preserve the entire sentence; if the full sentence cannot be timed reliably, it is marked as needing alignment, while any reliable partial playback range is retained rather than inventing a precise timestamp. The original sentence and speaker diagram was not supplied with the text.

Speaker models are not always right. Similar voices, overlapping speakers and noise can make attribution uncertain. I'd rather show “undetermined” than combine different people merely for a tidy interface.

Background work needs memory headroom

Processing while recording adds recognition work. Even in mini mode, a small window would not feel lightweight if several large models ran in parallel. I therefore avoid copying a full PCM audio buffer for each finalization batch into the queue. The queue primarily stores references to original-audio intervals and reads sealed segments when needed; quick correction uses a bounded short-audio buffer.

Heavy Mandarin and English inference tasks run sequentially, and their subprocesses exit after completion to reduce overlapping memory use. Audio capture and lightweight live captions need not wait for finalization.

Testing also exposed an exception: repeatedly concatenated English stress audio drove model output to a length limit. Saving a partial result directly could silently omit later text. Under that protection condition, the program does not commit the truncated candidate; it retries only the failed batch with shorter windows. The recognized body of that stress sample was ultimately retained, but some timing remained unreliable and was marked for confirmation. That is a fallback, not proof complex recognition and alignment are solved.

In my tests, long-batch heavy inference had a short physical-memory peak of about 6–7 GiB, with model memory released between tasks; the production client sampled about 109 MiB of physical memory when idle. These are machine- and test-specific results, not a promise for all Macs. I have not measured battery life or actual power, so I can describe resource controls but not claim low-power all-day operation.

After recording, let an agent help use it

The real problem with a long recording is not merely producing a huge transcript, but finding time to read it. The Mac client focuses on managing original audio and transcripts: group, search, star and replay. Personal hotwords can add common names, project names and terms to help recognition; they are not a dictionary that guarantees replacement every time.

To extract highlights, find the context of a decision or explain a technical term, I prefer an agent I already use. TalkTuck provides CLI, MCP and Skill integration so an agent can query saved recordings and transcripts and organize them for the task.

I haven't put every analysis into a fixed in-app template: technical discussions, interviews and daily conversations need different things. An external agent can handle them flexibly. Which service it uses and whether it connects to the internet depend on the user's agent configuration.

Original audio and transcripts are first stored locally. In the scheme described at the time, signing in to a TalkTuck account syncs recording information, final transcripts, groups and original audio by default; sync can be paused in settings, and temporary live captions are not uploaded. Check the scope of any authorization when giving another agent or service access.

One eight-hour real recording, and more work ahead

Yesterday I recorded about 8 hours, 2 minutes and 13 seconds on the Mac. On review, it had 2,894 microphone audio segments and a local transcript; the library showed processing complete and the tail had corresponding transcript entries.

This was a real long recording, not merely an index test using repeated short audio. But eight hours of duration does not mean eight hours of speech; some speaker analysis remained uncertain and I had not prepared a word-by-word reference. It cannot establish that every sentence was recognized correctly. It used the processing flow at the time and is not an eight-hour continuous acceptance test of today's three-stage flow.

For the new batch flow, I also replayed the approximately five-minute recording mentioned earlier and about ten minutes of Chinese and English audio made by human readers. In the English case, 354.5 seconds had finalized before recording stopped. In this check, the controlled Chinese sample's 126 sentences and English sample's 61 sentences showed no confirmed speaker-attribution errors or paragraphs merged across actual speaker changes.

Separately, an accelerated eight-hour-scale interval-index test checked coverage and recovery. It is distinct from the real long recording: the former did not run eight hours of continuous model inference, and the latter did not have word-by-word accuracy annotation. The controlled human-reader samples were spliced together and cannot replace natural, noisy multi-speaker testing.

Next I want to test an entire real working day: device switching, recovery after audio loss, text catch-up, memory and actual power use. I'll report once those measurements exist.

For the watch, much of the work was getting sound reliably from watch to phone. On the Mac the question became how audio, live text and final text can move at their own pace while recording continues, then rejoin one timeline.

Save the sound first; organize the words next. Understanding them can continue with the tools you prefer.