Publications
M. O. Heikkinen, K. Drosos, A. Politis, and T. Virtanen, “Method and apparatus for training and using a microphone geometry assisted encoder model to generate spatial audio signals,” U.S. Patent US20260065918A1, filed Aug. 28, 2025; published Mar. 05, 2026
A system for training a microphone geometry assisted encoder model and then utilizing the trained model to generate spatial audio signals that have been captured by a plurality of microphones. In a method for generating spatial audio signals, the method includes receiving geometry data related to a plurality of microphones of an audio capturing device and audio signal data captured by the plurality of microphones. The method also includes generating a spatial audio signal based on an output of a trained microphone geometry assisted encoder model. The trained microphone geometry assisted encoder model includes a geometry encoder configured to encode the geometry data and a signal encoder configured to encode the audio signal data. The trained microphone geometry assisted encoder model further includes a signal decoder having a plurality of layers and configured to generate the output upon which the spatial audio signal is based.
This patent addresses a practical limitation of machine-learning-based spatial audio capture: neural models are typically designed for a particular microphone arrangement, so using the same technology on a different phone or recording device may require generating new training data and retraining the model. The proposed approach makes the microphone configuration itself an input to the neural network. A dedicated geometry encoder processes information describing the microphone array—such as the number and positions of the microphones and, in some embodiments, their directional acoustic responses—while a separate encoder processes the recorded audio. The two sources of information are combined by the model, which predicts a set of filters that transforms the microphone recordings into a spatial audio representation such as Ambisonics. The model can be trained using many simulated sound scenes and different microphone configurations so that it learns not only how to reconstruct spatial audio, but also how that reconstruction should change when the microphone arrangement changes. The main benefit is therefore a spatial-audio capture system that can be deployed across devices with different microphone layouts with little or no retraining, reducing the engineering effort normally required to adapt a neural spatial-audio system to each new device.
K. Drosos, M. O. Heikkinen, J. T. Vilkamo, P. Tsiaflakis, “Model for speech enhancement,” U.S. Patent US20260065922A1, filed Aug. 15, 2025; published Mar. 05, 2026
Examples of the disclosure relate to a model that can be used for speech enhancement. The model comprises an encoder part comprising a sequence of encoding layers and caused to receive input data. The input data is based on a current frame of a noisy speech signal and one or more past frames of the noisy speech signal. The sequence of encoding layers is caused to process the input data so that output data of the encoder part comprises a reduced number of the multiple frequency positions and a single temporal position. The model also comprises a decoder part comprising a sequence of decoding layers caused to receive data from a prior decoding layer. The output data of the decoder part comprises multiple frequency positions and a single temporal position. The output data of the decoder part is for post processing to provide an output signal for speech enhancement.
This patent presents a computationally efficient neural-network architecture for speech enhancement, targeting real-time use on devices with limited processing and memory resources. The approach uses a causal U-Net that processes the current noisy speech frame together with information from previous frames, allowing it to exploit both short- and longer-term speech patterns without processing a large time-frequency region throughout the entire network. During encoding, the representation is progressively compressed until only a single temporal position remains, while the decoder reconstructs the required frequency resolution using skip connections from corresponding encoder layers. A post-processing stage can then use the resulting features to estimate an enhancement mask or directly produce enhanced speech. This design reduces unnecessary computation along the time dimension while retaining temporal information relevant to distinguishing speech from interference. The architecture is applicable not only to speech denoising but also to related enhancement problems such as echo suppression, dereverberation, bandwidth expansion, wind-noise removal, and recovery of missing speech. Rather than reporting a specific benchmark result, the patent focuses on the architectural principle and its main benefit: enabling effective neural speech enhancement with substantially lower computational requirements, making such processing more practical for mobile, wearable, and other resource-constrained devices.
K. Drosos, M. O. Heikkinen, S. Vesa, and M. T. Vilermo, “Speech and noise disentanglement for acoustic echo cancellation,” U.S. Patent US20260080885A1, filed Aug. 27 , 2025; published Mar. 19, 2026
The present disclosure relates to an apparatus, that obtains a far-end signal and a near-end microphone signal, determines, based on at least the far-end signal, a far-end speech signal estimate and a far-end noise signal estimate, determines, based on at least the near-end microphone signal, a near-end microphone speech signal estimate and a near-end microphone noise signal estimate, determines, based on at least the far-end speech signal estimate and the near-end microphone speech signal estimate, a predicted near-end speech signal, determines, based on at least the far-end noise signal estimate and the near-end microphone noise signal estimate, a predicted near-end noise signal and outputs at least the predicted near-end speech signal and predicted near-end noise signal.
This patent addresses a limitation of conventional acoustic echo cancellation in calls and teleconferencing: speech and background ambience are typically processed together, making it difficult to remove echo effectively while also preserving the environmental sounds that contribute to a natural and immersive communication experience. The proposed approach first separates both the far-end signal and the locally captured microphone signal into speech and noise/ambience components. It then treats these components independently, comparing far-end speech with near-end speech to remove speech echo, and far-end noise with near-end noise to remove the corresponding echoed ambience. The resulting local speech and local environmental sound are therefore estimated separately and can either be transmitted independently or recombined depending on the communication system. Neural source-separation or denoising models can be used for the initial decomposition, followed by conditioned models that determine which parts originate locally and which are echoes of the remote signal. By breaking the problem into these simpler stages, the approach is intended to make neural echo-cancellation systems easier to train while also allowing background ambience to be deliberately preserved rather than automatically suppressed, supporting more natural and potentially spatial or immersive audio communication.
P. Tsiaflakis, M. T. Tammi, and K. Drosos, “Apparatus, methods and computer programs for noise suppression,” U.S. Patent US20250210055A1, filed Dec. 20, 2024; published Jun 26, 2025
Examples of the disclosure relate noise suppression for audio signals in a communication setting. An apparatus obtains at least one audio signal for a current frame or one or more previous frames, based on at least two microphone signals for the current frame or one or more previous frames. The apparatus uses a program code to predict an output signal for a future frame based, at least in part, on the at least one audio signal for the current frame or one or more previous frames and uses the output signal for processing the future frame of the at least two microphone signals in a first audio signal process and uses the output signal for processing the future frame of an output of the first audio signal process in a second audio signal process to enable noise suppression.
T. Virtanen, T. Heittola, S. Zhao, S. Gharib, and K. Drosos, “Privacy-preserving sound representation,” U.S. Patent US20230317086A1, filed Oct. 5, 2022; published Oct. 12, 2023
According to an example embodiment, a method (200) for audio-based monitoring is provided, the method (200) comprising: deriving (202), via usage of a predefined conversion model (M), based on audio data that represents sounds captured in a monitored space, one or more audio features that are descriptive of at least one characteristic of said sounds; identifying (204) respective occurrences of one or more predefined acoustic events in said space based on the one or more audio features; and carrying out (206), in response to identifying an occurrence of at least one of said one or more predefined acoustic events, one or more predefined actions associated with said at least one of said one or more predefined acoustic events, wherein said conversion model (M) is trained to provide said one or more audio features such that they include information that facilitates identification of respective occurrences of said one or more predefined acoustic events while preventing identification of speech characteristics.
