Combining Performers with Soundtracks
some personal experiences
(pdf version)

Why combine live and pre-recorded?

•  skilled live performers provide a level of musicality and often virtuosity that is entirely different from what is possible with electroacoustic means, and therefore can be regarded as complementary
  
  
•  the live performer provides not merely a visual focus for the audience, but can act as a "persona" in their interaction with the soundtrack (some relationships are suggested below); this can lead to a sense of theatricality and drama; however, since the performer draws visual focus, I tend to keep the soundtrack very strong and present, not ambient (I prefer a soundtrack that is interesting to listen to on its own, yet leaves enough "space" for the live performer)
  
  
•  an interesting timbral dialogue can be established between the performer and soundtrack, particularly if the sound sources used for the soundtrack are derived from the performer, or other related sounds (in the early days with electronic synthesis, it was very difficult to match the richness of the live performer's sound with the available synthesis techniques, though digital FM synthesis was an improvement ; this has become much easier with sample-based work)

Some possible relationships between the live performer(s) and the soundtrack:

•   "figure in a landscape": the soundtrack provides an enhanced sonic environment, particularly when instrumental or vocal materials used in producing the soundtrack are those provided by the performer, and electroacoustically enhanced (examples are East Wind, Nightwatch, Inside, Tongues of Angels, Bamboo, Silk and Stone, and The Way of the Spirit where the acoustic virtuosity of the performer is pitted against the enhanced instrumental sounds on the soundtrack)
  
  
•   timbral extension: the vocal or instrument's pitches and timbre extend and complement the often stretched sounds on the soundtrack; sometimes the source sounds are entirely different from the live component and the performer blends with the soundtrack or at times emerges as a foreground voice (e.g. Dominion, Steam which are based on environmental sounds) whereas in From The Unseen World, the solo piano attempts to blend with the convolved and stretched sounds of the instrument in a form of spectral composition (example page from Steam)
  
  
•   text enhancement: when there is significant text material on the soundtrack (e.g. Song of Songs, Wings of Fire, Androgyne, Mon Amour, Beauty and the Beast) the live instrumentalist can mirror and elaborate on the text; this can involve deriving the melodic line from the inflection patterns of the voice (particularly when the voice is time-stretched) and elaborating on the pitches musically (example page from Song of Songs)
  
  
•   when the performer is a vocalist, then a spectrally based approach is useful, i.e. the vocal pitches can be derived from, for instance, the resonated and stretched sounds on the soundtrack, giving them a pitch reference (e.g. Powers of Two, Enigma, Thou and I, Orpheus Ascending, Twin Souls); in most cases, I have found that a unique "mode" of pitches has emerged from the soundtrack, presumably related to the harmonics that have become prominent via processing (example page from Beyond, a section of Powers of Two)
  
  
•   algorithmic extensions and counterpoint to the soundtrack: in my early works, I used the POD compositional system with its stochastic use of  Poisson distributed events within designed tendency masks to generate the live part (suitably adapted to Western rhythmic notation); with Nautilus, Trigon, Aerial and Sonic Landscape No. 4, both the digital soundtrack and the live part resulted from the same algorithms; with East Wind and Nightwatch, the soundtrack was created with analog processing of the instrumental sounds, and most of the live part created algorithmically (during a period when digital synthesis was unavailable); example page from Nightwatch starting at 5:20
  

Practical considerations:

•  I always insist on the live performer being miked which allows the person at the mixing console to balance the levels between live and the soundtrack; also this feeds the live performer's sound to the same speakers as a spatial enhancement (usually only front and side speakers, not rear ones)
  
  
•  synchronization: the score can include ideal timing marks for key moments of synchronization, with a fair degree of freedom in between (performers always worry about "getting behind" the fixed soundtrack at first and use a stopwatch for security, but with greater familiarity, they end up just treating the soundtrack as a dependable accompanist, knowing aurally what's coming next); still, it's best to leave a bit of "flex" in the live score and the timing of the soundtrack (in some cases I even leave the exact rhythm to the performer, example from Wings of Fire)
  
  
•  some works designed for a dedicated soloist (e.g. Song of Songs, Androgyne, Mon Amour) have very tight synchronization throughout, and key bits of text or other sound events are noted in the score; this works best for someone who will perform the piece many times and really make it their own (example page from Song of Songs, Androgyne, Mon Amour)
  
  
•  singers worry about their pitch references, but when the vocal part is derived from pitches/harmonics found on the soundtrack (as described above), singing in tune becomes easier
  
  
•  live performers may require foldback speakers either to hear themselves or to hear the soundtrack better (a separate mix can be fed to a foldback pair of speakers)
  
  
•  when recording a mixed piece for demonstration or publication, I never use a live pickup; instead, I arrange a studio recording where the performer can hear the soundtrack over headphones, and then the mix happens later when all the edited sections of the live part have been selected and arranged in the proper synch; I tend to favour a strong level for the pre-recorded material in the final mix

---