Algebraic signal processing
Algebraic signal processing (ASP) is an emerging area of theoretical signal processing (SP). In the algebraic theory of signal processing, a set of filters is treated as an (abstract) algebra, a set of signals is treated as a module or vector space, and convolution is treated as an algebra representation. The advantage of algebraic signal processing is its generality and portability.
History
In the original formulation of algebraic signal processing by Puschel and Moura, the signals are collected in an -module for some algebra of filters, and filtering is given by the action of on the -module.[1]
Definitions
Let be a field, for instance the complex numbers, and be a -algebra (i.e. a vector space over with a binary operation that is linear in both arguments) treated as a set of filters. Suppose is a vector space representing a set signals. A representation of consists of an algebra homomorphism where is the algebra of linear transformations with composition (equivalent, in the finite-dimensional case, to matrix multiplication). For convenience, we write for the endomorphism . To be an algebra homomorphism, must not only be a linear transformation, but also satisfy the property
Given a signal , convolution of the signal by a filter yields a new signal . Some additional terminology is needed from the representation theory of algebras. A subset is said to generate the algebra if every element of can be represented as polynomials in the elements of . The image of a generator is called a shift operator. In all practically all examples, convolutions are formed as polynomials in generated by shift operators. However, this is not necessarily the case for a representation of an arbitrary algebra.
Examples
Discrete Signal Processing
In discrete signal processing (DSP), the signal space is the set of complex-valued functions with bounded energy (i.e. square-integrable functions). This means the infinite series where is the modulus of a complex number. The shift operator is given by the linear endomorphism . The filter space is the algebra of polynomials with complex coefficients and convolution is given by where is an element of the algebra. Filtering a signal by , then yields because .
Graph Signal Processing
A weighted graph is an undirected graph with pseudometric on the node set written . A graph signal is simply a real-valued function on the set of nodes of the graph. In graph neural networks, graph signals are sometimes called features. The signal space is the set of all graph signals where is a set of nodes in . The filter algebra is the algebra of polynomials in one indeterminate . There a few possible choices for a graph shift operator (GSO). The (un)normalized weighted adjacency matrix of is a popular choice, as well as the (un)normalized graph Laplacian . The choice is dependent on performance and design considerations. If is the GSO, then a graph convolution is the linear transformation for some , and convolution of a graph signal by a filter yields a new graph signal .
Other Examples
Other mathematical objects with their own proposed signal-processing frameworks are algebraic signal models. These objects include including quivers,[2] graphons,[3] semilattices,[4] finite groups, and Lie groups,[5] and others.
Intertwining Maps
In the framework of representation theory, relationships between two representations of the same algebra are described with intertwining maps which in the context of signal processing translates to transformations of signals that respect the algebra structure. Suppose and are two different representations of . An intertwining map is a linear transformation such that
Intuitively, this means that filtering a signal by then transforming it with is equivalent to first transforming a signal with , then filtering by . The z transform[1] is a prototypical example of an intertwining map.
Algebraic Neural Networks
Inspired by a recent perspective that popular graph neural networks (GNNs) architectures are in fact convolutional neural networks (CNNs),[6] recent work has been focused on developing novel neural network architectures from the algebraic point-of-view.[7][8] An algebraic neural network is a composition of algebraic convolutions, possibly with multiple features and feature aggregations, and nonlinearities.
References
- Puschel, M.; Moura, J. (2008). "Algebraic Signal Processing Theory: Foundation and 1-D Time". IEEE Transactions on Signal Processing. 56 (8): 3572–3585. Bibcode:2008ITSP...56.3572P. doi:10.1109/TSP.2008.925261. ISSN 1053-587X. S2CID 206797175.
- Parada-Mayorga, Alejandro; Riess, Hans; Ribeiro, Alejandro; Ghrist, Robert (2020-10-22). "Quiver Signal Processing (QSP)". arXiv:2010.11525 [eess.SP].
- Ruiz, Luana; Chamon, Luiz F. O.; Ribeiro, Alejandro (2021). "Graphon Signal Processing". IEEE Transactions on Signal Processing. 69: 4961–4976. arXiv:2003.05030. Bibcode:2021ITSP...69.4961R. doi:10.1109/TSP.2021.3106857. ISSN 1053-587X. S2CID 212657497.
- Puschel, Markus; Seifert, Bastian; Wendler, Chris (2021). "Discrete Signal Processing on Meet/Join Lattices". IEEE Transactions on Signal Processing. 69: 3571–3584. arXiv:2012.04358. Bibcode:2021ITSP...69.3571P. doi:10.1109/TSP.2021.3081036. ISSN 1053-587X. S2CID 227736440.
- Bernardini, Riccardo; Rinaldo, Roberto (2021). "Demystifying Lie Group Methods for Signal Processing: A Tutorial". IEEE Signal Processing Magazine. 38 (2): 45–64. Bibcode:2021ISPM...38b..45B. doi:10.1109/MSP.2020.3023540. ISSN 1053-5888. S2CID 232071730.
- Gama, Fernando; Isufi, Elvin; Leus, Geert; Ribeiro, Alejandro (2020). "Graphs, Convolutions, and Neural Networks: From Graph Filters to Graph Neural Networks". IEEE Signal Processing Magazine. 37 (6): 128–138. arXiv:2003.03777. Bibcode:2020ISPM...37f.128G. doi:10.1109/MSP.2020.3016143. ISSN 1053-5888. S2CID 226292855.
- Parada-Mayorga, Alejandro; Ribeiro, Alejandro (2021). "Algebraic Neural Networks: Stability to Deformations". IEEE Transactions on Signal Processing. 69: 3351–3366. arXiv:2009.01433. Bibcode:2021ITSP...69.3351P. doi:10.1109/TSP.2021.3084537. ISSN 1053-587X. S2CID 221517145.
- Parada-Mayorga, Alejandro; Butler, Landon; Ribeiro, Alejandro (2022). "Convolutional Filtering and Neural Networks with Non Commutative Algebras". arXiv:2108.09923 [cs.LG].
External links
- Smart Project: Algebraic Theory of Signal Processing at the Department of Electrical and Computer Engineering at Carnegie Mellon University.
- Lecture 12: "Algebraic Neural Networks," University of Pennsylvania (ESE 514).