Category: JAX Laboratory

We submitted update 1.3 to the Apple AppStore today. This update adds 8 new shaders as promised.

Some thoughts for extension

I think, we will overwork the entire concept.

At the beginning we merely had the simple idea to provide an easy to use audio visualizer based on the AudioUnit framework, which can select different prepared shaders and apply several audio analysis results to the rendering of visuals via parametric control in realtime…

But now I can see the huge potential of this and want to apply a more complex concept. So we probably will introduce a second dynamic texture layer, which can blend with the first. This significantly will increase the possibilities. Both shaders will be independently in their parameters and computation.

But this will require to specialise the shaders even more. These must support true alpha transparency and additionally all commonly used graphics composition blend modi on the shader level. This all will increase the current parameter set significantly, of course.

The performance of the Metal framework is excellent and 2 layers should work without any problems according the GPU/CPU usage. ( You may have recognized already, that the audio unit is using CPU around 1 percent at current state and so outperforms any other realtime visualizer we have seen yet. )

A later extension ( PRO version? ) also could integrate a shader coder environment, meaning a text editor and just in time (JIT) compiler, where the user can put in own code, which then is validated and compiled in realtime to a new user created Metal shader. This shader coder, of course would user Apples shader language consequently, which is somewhat different than the commonly used GLSL language and also more strict regarding memory sharing and usage.

The significantly increased parameter set probably also would require a new parameter editor and so on. So maybe this all is for a “PRO” version of the JAX AudioVisualizer.

A third step could be a MIDI based parameter sequencer, which allows to create a complete timeline driven storyboard with the entire system for controlled playback.

The public TestFlight beta of JAX AudioVisualizer was released today and is available here.

A first manual is available too.

LATEST NEWS: The app (version 1.1) is not officially released yet to the app store, but we actually have the first expansion pack with 8 new shaders in preparation.

The JAX AudioVisualizer will receive expansion packs and function updates regularly, which are free for all early adopters.

Version 1.1 is scheduled for automatic release to the Apple AppStore since days. Version 1.2 added support for loading user files and version 1.3 will add 8 new Metal shaders. Both update versions are ready for shipping too but Apple delayed the release (as usually) due to rejections for minor and/or unacceptable reasons (they do for instance not understand what the app does and how an audio unit is supposed to work).

We just scheduled the release of our new AudioUnit, the JAX AudioVisualizer, available as a super universal release for iOS, iPadOS, visionOS and macOS. The app will be available as PUBLIC TestFlight release on all platforms too and appear in the app store with an introduction price.

JAX AudioVisualizer is a new audio visualization tool, written from scratch, in form of a special app extension, namely: AudioUnit (AUv3). It is available as universal release on iOS, iPadOS, visionOS and macOS with a single purchase.

AudioViszalizer‘s graphics core is based on Apples native performance shaders using the Metal framework, which is integral part of all Apple‘s operating systems. Metal shaders are highly optimized for mobile devices and deliver excellent performance with low CPU/GPU utilisation. Metal works always best with latest devices from Apple. It actually will outperform any OpenGL shader framework with excellence.

JAX AudioVisualzer is implemented as a classic audio effect unit with MIDI support but does not alter the audio stream in any way, just analyzing and passing it thru. It can be inserted anywhere, prior the main effect processing or after. 

The audio signal will be analyzed inside the CPU audio kernel and certain frequency and energy components calculated and extracted from the realtime stream. After that, the shaders will be feed with synchronized data for adapting to their shader algorithms in various ways but always inside the GPU. So the performance-heavy part will happen inside the graphic chips and not alter the common audio performance. Internal textures are sized moderately, for keeping inside app extension’s memory limitations.

There are loads of different style, high quality preset shaders available, which can be selected via the integrated preset browser. The size of the window can be adjusted for certain popular screen ratios, as 1:1, 4:3, and 16:9. The window of the audio unit is freely sizeable and moveable if the host even supports it. Frame rate is adjusted to 60Hz at maximum and depends on the general processing speed of the system.

The audio unit is thought for host applications, which can host and handle audio effect units in AUv3 format with own graphical user interface. Mainly music application hosts. The embedding app is a distribution envelope and provides some sound files for testing. By installing the app, the visualizer is available to all supporting apps in the system.

With the operating system’s inbuilt screen recording mechanisms, the visual output of the AudioVisualizer unit can be mixed with the audio stream in realtime and exported to performance videos. Or the unit can just be seen as a tool for enjoyment, while working and listening to music.

We recommend to switch the unit explicitely off, if visual output is not required.

JAX AudioVisualizer is available soon for one-time-payment lifetime access and not as subscription and we hope users will honor this.

Many of our developments and releases are initially based on JAX Laboratory prototypes, which most likely never will see the light as it. Such a prototype is for instance the JAX Algorithmic Composer.

The JAX Algorithmic Composer, a tool for generating multi-channel MIDI patterns. The video shows auto-generating MIDI files in a bulk and saving to disk. The core of the chord generator, the polythytmic and polymetric generators are for instance used with our JAX MIDI Jinny PRO release.

There is an extra page for JAX Laboratory items.

There is an extra page for JAX Laboratory items.

There is an extra page for JAX Laboratory items.