Wirtinger calculus

It’s not complicated / It’s complex


How do you differentiate real-valued functions of complex arguments? Wirtinger calculus. This is a ridiculous hack that happens to work very well for signal processing over the complex field, especially in optimisation. It arises naturally in, for example, phase retrieval, (Zhang and Liang 2016; Candes, Li, and Soltanolkotabi 2015; Chen and Candès 2015; Seuret and Gouaisbaut 2013). Because of its area of popularity, this will almost surely arise in combination also of matrix calculus.

There are various write-ups of it; this will be a literature review of those because they are of highly variable quality and I want to ensure I understand what I’m playing with her.

(Bouboulis 2010) has a punchy intro as part of a paper which wishes to take Wirtinger derivatives inside inner products.

Wirtinger’s calculus has become very popular in the signal processing community mainly in the context of complex adaptive filtering, as a means of computing, in an elegant way, gradients of real valued cost functions defined on complex domains ( \(\mathbb{C}^{\nu}\) ). Such functions, obviously, are not holomorphic and therefore the complex derivative cannot be used. Instead, if we consider that the cost function is defined on a Euclidean domain with a double dimensionality (\(\mathbb{R}^{2\nu}\)), then the real derivatives may be employed. The price of this approach is that the computations become cumbersome and tedious. Wirtinger’s calculus provides an alternative equivalent formulation, that is based on simple rules and principles and which bears a great resemblance to the rules of the standard complex derivative… A common misconception …is that Wirtinger’s calculus uses an alternative definition of derivatives and therefore results in different gradient rules in minimization problems. We should emphasize that the theoretical foundation of Wirtinger’s calculus is the common definition of the real derivative. However, it turns out that when the complex structure is taken into account, the real derivatives may be described using an equivalent and more elegant formulation which bears a surprising resemblance with the complex derivative. Therefore, simple rules may be derived and the computations of the gradients, which may become tedious if the double dimensional space \(\mathbb{R}^{2\nu}\), is considered, are simplified.

The extension to Hilbert space operations is nifty.

Other favoured resources:

  • Terry Tao, 246a 1: Complex differentiation
  • (Fischer 2005) is a short take-me-straight-there one.
  • (Adali, Schreier, and Scharf 2011) introduces more machinery for complex valued signal processing.
  • (Caracalla and Roebel 2017) shows an actual application in a musical signal processing optimization problem, which is very simple and yet possibly the only non-trivial and also comprehensible application in any of the introductions.

Adali, T., P. J. Schreier, and L. L. Scharf. 2011. “Complex-Valued Signal Processing: The Proper Way to Deal with Impropriety.” IEEE Transactions on Signal Processing 59 (11, 11): 5101–25. https://doi.org/10.1109/TSP.2011.2162954.

Bouboulis, P. 2010. “Wirtinger’s Calculus in General Hilbert Spaces,” May. http://arxiv.org/abs/1005.5170.

Candes, Emmanuel, Xiaodong Li, and Mahdi Soltanolkotabi. 2015. “Phase Retrieval via Wirtinger Flow: Theory and Algorithms.” IEEE Transactions on Information Theory 61 (4, 4): 1985–2007. https://doi.org/10.1109/TIT.2015.2399924.

Caracalla, Hugo, and Axel Roebel. 2017. “Gradient Conversion Between Time and Frequency Domains Using Wirtinger Calculus.” In International Conference on Digital Audio Effects (DAFx-17), Edinburgh, UK, September 5–9, 2017, 5. http://www.dafx17.eca.ed.ac.uk/papers/DAFx17_paper_51.pdf.

Chen, Yuxin, and Emmanuel Candès. 2015. “Solving Random Quadratic Systems of Equations Is Nearly as Easy as Solving Linear Systems.” In Advances in Neural Information Processing Systems 28, edited by C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, 739–47. Curran Associates, Inc. http://papers.nips.cc/paper/5743-solving-random-quadratic-systems-of-equations-is-nearly-as-easy-as-solving-linear-systems.pdf.

Fischer, Robert F. H. 2005. “Appendix A: Wirtinger Calculus.” In Precoding and Signal Shaping for Digital Transmission, by Robert F. H. Fischer, 405–13. Hoboken, NJ, USA: John Wiley & Sons, Inc. https://doi.org/10.1002/0471439002.app1.

Hjørungnes, Are. 2011. Complex-Valued Matrix Derivatives: With Applications in Signal Processing and Communications. Cambridge: Cambridge University Press.

Hunger, Raphael. 2007. “An Introduction to Complex Differentials and Complex Differentiability,” 20.

Schreier, Peter J., and Louis L. Scharf. 2010. Statistical Signal Processing of Complex-Valued Data: The Theory of Improper and Noncircular Signals. Cambridge: Cambridge University Press.

Seuret, Alexandre, and Frédéric Gouaisbaut. 2013. “Wirtinger-Based Integral Inequality: Application to Time-Delay Systems.” Automatica 49 (9): 2860–6. https://hal.archives-ouvertes.fr/hal-00855159.

Zhang, Huishuai, and Yingbin Liang. 2016. “Reshaped Wirtinger Flow for Solving Quadratic System of Equations.” In Advances in Neural Information Processing Systems 29, edited by D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett, 2622–30. Curran Associates, Inc. http://papers.nips.cc/paper/6319-reshaped-wirtinger-flow-for-solving-quadratic-system-of-equations.pdf.