From Evidence to Motion: The Recordly Scene
Recordly's next problem begins after capture succeeds. A video file, cursor samples, and companion tracks describe what happened, but they do not yet explain how the viewer should see it. This chapter focuses on the scene Recordly builds at each media timestamp: how camera motion, cursor polish, backgrounds, webcam footage, captions, and annotations become one composition, and why sharing those semantics between preview and export matters. The result is a way to judge automatic motion as useful evidence-driven assistance rather than as magic understanding of the screen.
One timestamp, several coordinated layers
The most important shift is from “video plus effects” to “a scene evaluated at time.” Recordly's preview initializes a PixiJS scene with a camera container, a filtered video layer, a masked video sprite, and a cursor layer. Other visual elements are synchronized around that scene: the background fills the frame, the webcam is placed relative to the composition, captions become active over time ranges, and annotations are positioned against the recording area.
The source video remains the anchor. It provides the current frame and the dimensions of the captured content. A crop and padded layout determine which part of that frame belongs inside the final presentation. A mask can impose rounded or squircle-shaped corners. The camera container then applies the active zoom and position, while effects such as motion blur operate on the layer that should move with the camera. This ordering matters. A shadow around the recorded content should not behave like a shadow attached to the whole application window, and a cursor that is meant to sit over the content should not be clipped by the content mask.
At the same timestamp, other layers may be active or inactive. A caption cue can appear only during a particular interval. An annotation can have a start and end time, a position, a size, and a stacking order. A webcam clip can be offset relative to the main recording and cropped into a bubble. A video wallpaper can loop behind the main scene. These layers do not all use the same rendering technology: the preview combines Pixi with HTML and canvas-backed elements, while the export renderer can stage frames or rasterize overlays when direct GPU upload is unreliable. What unifies them is their relationship to the scene clock and layout geometry.
This organization gives Recordly a useful separation of concerns. The video layer does not decide where the camera looks. The cursor renderer does not own the timeline. The webcam layout code does not decide when a caption appears. Each responsibility produces state or geometry that the scene can combine. The system still has to coordinate them, but coordination is easier to reason about when the boundaries are explicit.
The scene can also explain a common editing expectation: changing a visual setting while playback is paused should update the exact current frame, not introduce movement simply because the render loop continued to tick. Recordly's preview distinguishes a playing frame from a paused frame and from a seek. It composes only when the media timestamp changes or when a paused frame must be refreshed. That prevents a spring-driven camera or cursor from drifting while a user is inspecting a still moment.
Motion is derived, not baked into the source
Recordly's cursor and camera behavior are reconstructions from time-indexed evidence plus editor settings. Cursor samples are interpolated between neighboring timestamps. The resulting position is projected through the current crop, so a cursor outside the visible crop is treated differently from one inside it. A smoothing state then chases the target using spring behavior. The rendered cursor can use different assets and styles, show a shadow, rotate with sway, blur in the direction of movement, bounce on a click, or draw a click ring.
That reconstruction is powerful because it separates the physical event from its presentation. A click remains an interaction sample; the size, color, duration, and visual form of the click effect are editor choices. A cursor path remains a sequence of observations; smoothing controls whether the displayed cursor follows every small movement or produces a calmer trajectory. The distinction also creates a boundary: if the source has no telemetry, or if the crop removes the observed location, Recordly cannot manufacture reliable cursor behavior from the source pixels alone.
Zoom works through a similar chain. A zoom region has a time interval, a depth, a focus, and a mode. A manual region uses an explicit focus. An automatic region can use cursor telemetry to follow the activity, so the camera may move toward where the pointer is working rather than toward a generic center point. The zoom resolver produces a target scale, focus, and transition progress. A transform function converts that target into a camera position and scale that fit the current scene geometry.
The motion between target states is a separate decision. During playback, spring state can make the camera approach a target smoothly. During a seek, or when the preview needs to snap to a paused frame, the system evaluates the exact target instead of carrying a physical-looking transition across a discontinuity. In classic mode, it can likewise use the direct transform. This is why the preview must reason from media time rather than from the wall clock: if a speed region changes the relationship between source time and output time, a wall-clock animation would produce a different camera path from the encoded video.
Automatic zoom should therefore be described carefully. Recordly can infer a useful focus from cursor activity and predefined regions, but the repository does not support a claim that it understands the meaning of the interface being recorded. It knows where the cursor moved, when an interaction occurred, and what settings the creator supplied. That is enough to produce a plausible presentation in many demos, but it is not enough to know whether the pointer's movement represents the most important idea on screen. Manual focus remains the correction when an automatic emphasis is visually wrong.
The camera also affects overlays. A webcam bubble can be allowed to react to zoom by scaling inversely with the scene scale. Without that compensation, the bubble would grow with the content and could become disproportionately large while the camera moves in. With it, the webcam can remain a stable presenter element while the recorded interface receives the emphasis. This is not merely a style preference; it is a relationship between the camera transform and the coordinate system in which the overlay is placed.
Why a webcam bubble is a scene problem
A webcam overlay illustrates why independent stickers are a misleading mental model. Recordly tracks a webcam source with its own time offset, waits for usable media, and resolves a crop-aware source rectangle. It can mirror the source, choose a preset or custom position, scale the bubble against the preview or export dimensions, compute roundness, and apply layered shadows. The layout uses the source's actual aspect ratio so that a crop does not silently stretch the speaker's image.
During preview, the webcam video must seek or play to the time corresponding to the main timeline. During export, a decoder-backed frame source may be used first, with a media-element fallback if that source fails. If the webcam frame is not ready or has drifted too far from the expected timestamp, the exporter can retain the last usable frame rather than inject an arbitrary one. That preserves continuity, but it is also a visible limit: a cached frame is a recovery behavior, not proof that the webcam stream remained perfectly synchronized.
Backgrounds have similar timing and geometry concerns. A color or gradient is static, but an image must be fitted to the output frame and a video wallpaper must be synchronized and looped. Blur is scaled relative to the rendered width. If a blur setting were treated as a literal number of pixels, a preview at one size and an export at another size would look materially different. Recordly's shared scene-effect calculation instead derives blur and shadow metrics from a reference width, keeping the visual relationship more stable across resolutions.
Captions and annotations add a second kind of alignment. Their visibility is time-based, but their placement is scene-based. A caption must be measured, laid out, and positioned relative to the output frame. An annotation may be attached to the recording area and should follow the camera transform when its meaning is tied to the content. Export may rasterize a caption or annotation into a canvas before sending the result through the scene renderer. The implementation can change, but the reader-facing invariant is that the overlay should appear at the intended time and in the intended relationship to the recorded content.
Preview and export share meaning, not necessarily conditions
Recordly deliberately reuses scene helpers across preview and export. Both paths resolve zoom targets, use motion-smoothing logic, apply crop and layout calculations, render cursor states, and compute webcam geometry. Preview and export also try to compose once per meaningful media timestamp. This shared semantic layer reduces a common failure in visual editors: the preview and the saved file each implement “the same” effect with slightly different timing or geometry.
The sharing is not an assertion that the two environments are identical. Preview depends on an interactive renderer, a live media element, the available Pixi backend, and a user resizing a window. Export may render at a different resolution, decode frames sequentially, and choose a native or WebCodecs encoder. Fonts, GPU drivers, media readiness, and hardware paths can differ. Recordly therefore scales effect values rather than assuming that one pixel value means the same thing everywhere, and it keeps a 2D fallback available when the preview's preferred renderer cannot initialize.
Native export paths make the distinction sharper. A specialized compositor can be faster for a subset of scenes, but the current exporter skips native static-layout routes for combinations such as video wallpapers, click effects, annotations, captions, certain webcam shapes, or timeline features that the route cannot model with shared-renderer fidelity. In those cases, the shared renderer is not a slow afterthought. It is the compatibility path that knows how to interpret the complete scene.
This is the central trade-off of Recordly's scene architecture. A narrow implementation can optimize one visual arrangement aggressively. A shared semantic model can preserve more combinations of motion, overlays, timing, and editing state, but it must manage more synchronization and more fallback conditions. Recordly chooses both approaches, using native paths when their contract is sufficient and retaining the shared scene when it is not.
The consequence for a creator is practical. If the preview looks right, that is useful evidence that the project state and scene renderer agree, but it is not a guarantee that every output route can reproduce it identically. If an export chooses a fallback, the question is whether the fallback preserves the scene's meaning, not whether the internal path stayed the same. And if an automatic zoom or cursor effect looks wrong, the correction belongs in the evidence-to-motion relationship—focus, timing, smoothing, or visibility—not in the original screen recording.
A scene is therefore the bridge between capture evidence and durable editorial intent. It gives the raw media a camera, a cursor, a frame, and synchronized companions. The next problem is to preserve the decisions that control those layers beyond the current editor session. That is why Recordly's timeline and .recordly project model matter: the scene can only be reproduced if its intent, and the media paths on which that intent depends, are made durable.