The process of inspecting time-based media is inherently time consuming. Whether it is born-digital content or digitized materials, it is difficult to inspect everything in its entirety. There have been some technological advances in the audio engineering field that aid in inspection through the visualization of audio data over time. Waveform, vectorscope, and spectrogram visualizations are currently found in open-source and commercial software, providing fast and detailed insight into the content without having to resort into traditional, real-time, linear approaches. Unfortunately, similar innovations into visualizing video data have been more limited.
Inspired by audio spectrograms, I developed an algorithm based on video waveform monitors to measure and visualize video luminance data over time as way to produce a video analog to the audio spectrogram. From my tests of this algorithm, I’m able to get a bird’s-eye view of the entire video and spot areas of potential points of interest without playing video in real time.
With this visualization, exact edit location and styles (such as hard-cuts vs fades), as well as camera movement, can be inferred. Noise, dropouts, and tape creases produce distinctive visual signatures and are identifiable.
Combined with tradition approaches, this method could be used to assist in the description and analysis of large amounts of video data within a fraction of the time it normally takes