v1.2.3: Motion Capture, Music Viz, Cathedral

Get ready to step into an immense, music-visualizing environment, where each ringer hit creates new colors that slowly spiral into the sky. And to better enjoy the music-viz view, you can now sit back and watch “robot” avatars perform the music from loops! To get started watching a “show” right away, load up one of the new layouts provided by EXA (via your “Load Layouts” menu). You can record your own avatar performances using the loop recorder, which now captures your motions at the same time as it records your musical events. This release includes an experimental version of EXA that runs on a Mac, along with dozens of other improvements, optimizations, and fixes. Thank you to the EXA community for the feedback and ideas! Please watch/share the new update video below, and read through this page to learn more. v1.2.3 RELEASE VIDEO https://www.youtube.com/watch?v=nhu58-weAZM (https://www.youtube.com/watch?v=nhu58-weAZM) MOTION CAPTURE

• When you select the new “Motions” checkbox on the loop recorder, it will capture your headset and handle movements while you perform the loop. When both “Ringers” and “Motions” are selected, a loop must contain at least one ringer hit. When only “Motions” is selected, the loop will record all motions that occur (and no ringer events).

• A robot-looking avatar appears whenever you start playback of a loop with “Motions” data. This avatar follows the motions that were used to perform the notes in the loop. The avatar’s tools appear to hit the ringers, generating the same tool/cursor tails (the glowing streaks) and lit-up ringers that occur during live performance.

• To achieve smooth motion when the loop “rolls over” (when playback continues from the end to the start), the avatar’s motions are blended. During the last beat (or after the last note in the loop, whichever is last), the avatar smooths between its current position/velocity and the position/velocity needed at the start of the loop. This smoothing also does its best to handle scenarios where beats are added/removed (using the “Loops > Duration” menu) from the start/end of a loop.

• To support layout adjustments, the avatar will follow the associated ringers (the ones that are touched in the loop recording) as you move those ringers around. For best results, move the entire group of ringers. Moving individual ringers also works – the avatar’s handle will smoothly adjust itself to hit the moved ringer, and its head will slide accordingly.

• You can disable the avatar display via the “Loops > Motions” menu, which provides checkboxes for the headset and both handles. Currently, the ringers in the loop will still light up, even if the associated avatar handle is not displayed.

• As described above, the avatar motions are connected to the ringer positions. Deleting a ringer that was used in the loop will not affect the loop’s audio playback, but this may cause the avatar to move in an undesirable way. Notably, this prevents the avatar from “following” the position of that ringer, since it is no longer present.

• The avatar motion also updates to match a loop’s quantization level and the current metronome tempo.

• The motion capture also watches for tool changes, remembering the tools used to start the loop, as well as any tools that you change during the recording. Note that you can even switch to the “camera” tool (by opening the “Layout > Save > Add Photo” menu) during motion recording.

• The avatar headset features some music-viz elements that glow and change color based on the notes that the avatar is currently playing.

• The avatar quickly scales in/out upon the start/stop of loop playback. The avatar headset also scales out when your own VR headset approaches it, preventing it from occluding your view; it reappears once these two objects are further apart.

MUSIC VIZ / CATHEDRAL ENVIRONMENT

• EXA now features a new space/environment called the “cathedral”, shaped by simple posts that form walls around the layout area and a roofline that covers it. Large, decorative diamond shapes float below the peaks of the cathedral.

• Immense walls that fill with music visualizations surrounding the layout area and the entire cathedral. These walls are shaped like a tall, twisted cylinder. With each ringer hit, the music visualizations start at ground-level and spiral up into the sky. A reflective floor shows the entire visualization in reverse, as if it were stretching far below the layout area.

• To avoid excess visual conflict with the layout and ringers, each note in the music visualizer begins with a darkened and partially-desaturated color. As that note rises up, it gets brighter and more vivid, then fades to black as it reaches the very top.

• The music visualizer picks a note’s horizontal position based upon a ringer’s note (the selected pitch/key). The color of this note is based on the color of the ringer, and its brightness is based on the volume of the audio that the ringer is currently producing.

• The music visualization can be disabled or adjusted via the “App > Visuals” menu. The “Show Music Viz” checkbox sets the visibility of the entire music visualization. The “Blur Music Viz” checkbox specifies whether the colors in the visualization should be blurred slightly, or if they should stay sharply in their original “lane”.

• In this new environment, the four main lights have been moved upwards and outwards from the layout center. The white spheres that used to represent these light sources are now removed.

VISUAL / INTERACTION

• Added more segments and smoothing to the tool/cursor tails. Made them thinner and shorter.

• Added support for an arbitrary number of tool/cursor tails for use with the motion-capture playback.

• Improved the brightness/color of the various ringer and menu highlight states (grabbed, selected, hovered, nearest-to-cursor).

• Reduced the proximity radius of loops within a section by 50% (from 4cm to 2cm) to avoid accidental grabbing/moving of loops.

• Updated the menu item proximity/hover distance to match up with the menu's grab distance.

• Updated the menu summoner to use the nearest position to camera (rather than center point) so the menu doesn't appear far away on large objects (like a long loop).

• Updated the nearest-on-edge calculation to handle long, rectangular objects better (notably, large loops and sections).

OTHER

• Added example layouts to the EXA package, which automatically install in your “Layouts” directory. You can find and load these via the “Layouts > Load” menu. These example layouts often come from the EXA development update and music performance videos. Just start the section or loops in the layout, then enjoy the show!

• Updated the camera tool (in the “Save Layout” menu) to save each photo in your “Documents/EXA VR/user-[SteamID]/Photos” directory.

• Updated the loading/saving of layout files to occur as a “background” process, so large files won’t stall the framerate.

• Improved the “building” process, upon loading a new layout, to distribute processing more evenly. Notably, this avoids stalled framerates when building large, motion-containing loops.

• AdjustedImproved performance throughout the app, such asSimplified ringers when they are not vibrating. Reduced ringer vibration quality when you are far away. Prevented all work on loop/section display when the panel is facing away. Adjusted mesh materials to support dynamic batching (which wasn’t working as expected in EXA’s “deferred rendering” mode). Avoided unnecessary processing on buttons and sizes/positions within the menu panels.

• Prevented the recorded bow/prox events from being quantized in a loop. These tools don’t create exact start/hit times, so quantization typically causes the notes to shift in an undesirable way.

• Disabled the metronome's tempo slider while a loop is counting down or recording.

• Added a 10-minute limit on loop recording. The additional data collected by bow/prox tools and motion captures can build up quickly, this time limit prevents the creation of huge loops that might crash the app.

• Added a notification for an “AppSettings.json” loading failure.

• Added a notification when files cannot be moved to EXA’s new 'save' directory.

• Removed the "pretty" JSON formatting from the saved layout files to reduce the file sizes by half (or more).

• Implemented experimental Mac port of the app. Please contact Zach if you’re interested in testing it out.

FIXES

• Fixed issue causing some "Sounds" menu buttons to be highlighted (in yellow text) incorrectly when they have similar names to the ones in the active sound path.

• Fixed issue with cloning by disabling the "Delete" and "Load Layout" buttons during the clone process.

• Fixed issue with section cloning by (while the cloning process is active) disabling grab on its contained loops and preventing new loops being added to the section.

• Fixed issue causing sustained notes (from bow/prox tools) in a loop to never stop upon deleting the currently-playing loop.

• Fixed issue causing the bottom-of-handle tool/cursors to generate highlights as if it were the "nearest grab cursor".

• Fixed issue causing a section positioned inside a group to be impossible to grab.

• Fixed (potentially) an issue causing a new loop recorder to not appear after completing a recording. Added a failsafe creation of a new loop recorder if one is not present during the span of two seconds.

• Fixed issue causing recorded proximity/bow notes to occasionally playback incorrectly within a loop, typically with a "volume burst" at the start (instead of a smooth fade-in).

• Fixed issue causing odd audio distortions (a subtle flanger-like effect) at the beginning of notes when the app first starts or when changing streams.

PATCH 1

• Fixed issue causing the motion recording to fail when using Steam's mixed-reality mode. This mode causes a additional, incomplete "Steam controller manager" to be added to the scene, and the motion recording system was trying to use it.

• Fixed issue causing all the menu interface corners/borders (which appear for proximity/selection state) to be overdrawn (regardless of depth) by the document's image material.

• Added a new error handler while loading a Soundfont file, so that a failure on loading/parsing a particular sample won't prevent the rest of the file from loading. If you notice a disabled (semi-transparent) ringer using a note+sample combination that should be present in the Soundfont file, please let Zach know.