Terms for Live Visuals and How to Speak the Language

What is the difference between VJing and generative art in a live performance context?

VJing involves the real-time manipulation and sequencing of pre-rendered clips, whereas generative art relies on code-based algorithms to create visuals that evolve dynamically without fixed durations.

Use these metrics to balance your hardware resources, as generative patches often demand significantly more GPU power than standard video playback.

How do latency and frame rate impact the synchronization of audio and video?

Latency represents the time lag between an audio trigger and the visual response, while frame rate determines the smoothness of that motion. When these aren't aligned, the brain perceives a disconnect, making the visuals feel detached from the music. Many people assume that a higher frame rate automatically fixes sync issues, but it actually increases the processing load, which can paradoxically introduce more input lag if the hardware cannot keep up with the rendering demands. Prioritize a steady 60fps over higher resolutions to keep the response time tight and the synchronization feeling natural to the audience.

What are the primary hardware components required for a professional live visual setup?

A professional rig centers on a high-performance workstation capable of processing multiple video streams simultaneously without thermal throttling.

• Graphics Card: An NVIDIA RTX 4080 or better provides the necessary CUDA cores to handle complex geometry and real-time texture rendering.
• Capture Card: A Blackmagic DeckLink or similar PCIe device is needed to ingest live camera feeds with minimal processing delay.
• Solid State Drive: An NVMe drive with read speeds exceeding 3,000MB/s prevents stuttering when triggering multiple high-bitrate video files.
• Control Surface: A dedicated MIDI controller like the Akai APC40 offers physical faders and buttons for tactile, reliable interaction during a performance.

How does mapping technology allow projections to conform to irregular physical surfaces?

Yes, mapping software uses a grid-based transformation matrix to warp video pixels across non-planar geometries. By placing control points on the corners of a physical object, the software distorts the output to match the surface precisely, often correcting for perspective shifts across angles as wide as 180 degrees.

What role does MIDI play in controlling visual software during a live set?

MIDI acts as the bridge between physical hardware and software parameters, allowing an operator to map buttons, knobs, and faders to specific visual functions. By sending digital messages, the software responds instantly to external input, letting the performer trigger clips or adjust effects without touching a mouse. For instance, mapping a fader to the opacity of a layer allows for smooth, manual transitions that feel musical and intuitive.

However, MIDI is a legacy protocol with limited bandwidth, meaning it can occasionally struggle with high-density data streams. If you need to send massive amounts of precise data, such as per-pixel control or complex animation states, OSC (Open Sound Control) is often a more robust alternative. It offers higher resolution and faster communication speeds, making it better suited for modern, high-fidelity visual installations that require deep integration between software environments.

What are the most common file formats and codecs used for high-performance video playback?

The industry standard for playback is the HAP codec, which offloads decompression to the GPU for high-performance efficiency. Files are typically stored as .mov containers, allowing for smooth playback of 4K content at 60fps without taxing the CPU, which is essential for maintaining a consistent visual flow during live sets.

How do shaders and GLSL programming enable real-time visual effects?

Shaders are small programs written in GLSL (OpenGL Shading Language) that run directly on the GPU to calculate the color of each pixel. Because they execute in parallel across thousands of processor cores, they can generate complex, fluid animations or reactive light patterns that would be impossible to render as standard video files. A common example is a fractal noise shader that shifts based on audio frequency analysis, creating a visual that responds instantly to the beat.

The nuance lies in the complexity of the code, as poorly optimized shaders can quickly saturate the GPU and cause frame drops. While simple shaders are efficient, nesting too many loops or complex mathematical functions can lead to performance bottlenecks. Always profile your code during development to make sure it maintains a consistent frame rate, as even a minor logic error can cause the entire visual output to stutter or freeze.

What is the significance of alpha channels and transparency in layering live visual content?

Alpha channels provide the metadata necessary for layering multiple video sources, enabling the transparency required to composite complex visual scenes.

• RGBA Color: This 32-bit format includes the red, green, blue, and alpha channels, allowing for precise control over pixel-level opacity.
• PNG Sequences: These are often used for high-quality motion graphics because they support lossless transparency, though they require significant storage space.
• Masking Layers: Using a secondary greyscale video as a matte allows you to dynamically reveal or hide parts of an image during a live performance.
• Blending Modes: These mathematical operations determine how transparent layers interact, such as using "Add" to make bright elements pop against dark backgrounds.