Concatenative synthesis

December 13, 2015 — January 16, 2017

Hilbert space
machine listening
making things
music
Figure 1

Transferring timbre from one sound to another; Synthesis by example. When you refer to “concatenative synthesis” or an “Audio mosaic,” you usually mean using a granular synthesis method. This being the epoch of neural networks, someone will probably get style transfer for audio functioning soon.

I’ve published in this area. See Mosaic Style Transfer using Sparse Autocorrelograms.

The most comprehensive overview of classic concatenative stuff IMO is contained in Graham Coleman’s doctoral dissertation, Coleman (2015), which frames it in terms of loss functions and descriptors.

There are a few classic implementations about.

Audio analogies

Related: analysis-resynthesis, learning gamelan.

1 References

Amatriain, Bonada, Loscos, et al. 2003. “Content-Based Transformations.” Journal of New Music Research.
Aucouturier, and Pachet. 2006. “Jamming with Plunderphonics: Interactive Concatenative Synthesis of Music.” Journal of New Music Research.
Blumensath, and Davies. 2004. “On Shift-Invariant Sparse Coding.” In Independent Component Analysis and Blind Signal Separation.
———. 2006. “Sparse and Shift-Invariant Representations of Music.” IEEE Transactions on Audio, Speech and Language Processing.
Coleman. 2015. “Descriptor control of sound transformations and mosaicing synthesis.”
Collins. 2012. “Even More Errant Sound Synthesis.” In.
Collins, and Sturm. 2011. “Sound Cross-Synthesis and Morphing Using Dictionary-Based Methods.” In International Computer Music Conference.
Cont, Dubnov, and Assayag. 2007. “GUIDAGE: A Fast Audio Query Guided Assemblage.” In.
Driedger, Muller, and Ewert. 2014. “Improving Time-Scale Modification of Music Signals Using Harmonic-Percussive Separation.” IEEE Signal Processing Letters.
Ellis, Cotton, and Mandel. 2008. “Cross-Correlation of Beat-Synchronous Representations for Music Similarity.” In IEEE International Conference on Acoustics, Speech and Signal Processing, 2008. ICASSP 2008.
Forrester, and Keane. 2009. “Recent Advances in Surrogate-Based Optimization.” Progress in Aerospace Sciences.
Gatys, Ecker, and Bethge. 2015. “A Neural Algorithm of Artistic Style.” arXiv:1508.06576 [Cs, q-Bio].
Green, and Bass. 1984. “Representing Periodic Waveforms with Nonorthogonal Basis Functions.” IEEE Transactions on Circuits and Systems.
Kersten, and Purwins. 2010. “Sound Texture Synthesis with Hidden Markov Tree Models in the Wavelet Domain.” In.
Kowalski, Siedenburg, and Dorfler. 2013. “Social Sparsity! Neighborhood Systems Enrich Structured Shrinkage Operators.” IEEE Transactions on Signal Processing.
Kronland-Martinet, Guillemain, and Ystad. 1997. “Modelling of Natural Sounds by Time–Frequency and Wavelet Representations.” Organised Sound.
Masri, Bateman, and Canagarajah. 1997a. “A Review of Time–Frequency Representations, with Application to Sound/Music Analysis–Resynthesis.” Organised Sound.
———. 1997b. “The Importance of the Time–Frequency Representation for Sound/Music Analysis–Resynthesis.” Organised Sound.
Mital, Grierson, and Smith. 2013. “Corpus-Based Visual Synthesis: An Approach for Artistic Stylization.” In.
Neidinger. 2010. “Introduction to Automatic Differentiation and MATLAB Object-Oriented Programming.” SIAM Review.
Queipo, Haftka, Shyy, et al. 2005. “Surrogate-Based Analysis and Optimization.” Progress in Aerospace Sciences.
Rebollo-Neira, and Lowe. 2002. “Optimized Orthogonal Matching Pursuit Approach.” IEEE Signal Processing Letters.
Roma, Green, and Tremblay. 2020. “Audio Morphing Using Matrix Decomposition and Optimal Transport.” In DAFX 2020.
Schwarz. 2007. “Corpus-Based Concatenative Synthesis.” IEEE Signal Processing Magazine.
———. 2011. “State of the Art in Sound Texture Synthesis.” In Proceedings of DAFx-11.
Simon, Basu, Salesin, et al. 2005. “Audio Analogies: Creating New Music from an Existing Performance by Concatenative Synthesis.” In Proceedings of the 2005 International Computer Music Conference.
Sturm, Bob L. 2006. “Adaptive Concatenative Sound Synthesis and Its Application to Micromontage Composition.” Computer Music Journal.
———. 2011. “Sparse Vector Distributions and Recovery from Compressed Sensing.”
Sturm, Bob L., Roads, McLeran, et al. 2009. “Analysis, Visualization, and Transformation of Audio Signals Using Dictionary-Based Methods.” Journal of New Music Research.
Sturm, B. L., Shynk, Daudet, et al. 2008. “Dark Energy in Sparse Atomic Estimations.” Trans. Audio, Speech and Lang. Proc.
Tachibana, Ono, and Sagayama. 2014. “Singing Voice Enhancement in Monaural Music Signals Based on Two-Stage Harmonic/Percussive Sound Separation on Multiple Resolution Spectrograms.” Audio, Speech, and Language Processing, IEEE/ACM Transactions on.