implicit_filter#

implicit_filter(glogp, R, L)#

Implicit Modal Filter: Generates filters for Long Implicit Layer used in Hyena Architecture.

Note

This function reduces the memory consumption by a factor of order.

Parameters:

glogp (torch.Tensor) – Log of the probability of the modal frequencies.
R (torch.Tensor) – Parameters of the filter.
L (int) – Length of the sequence.

Returns:

Filters of shape (batch_size, L).

Return type:

torch.Tensor

Example

d_model = 128
order = 16
glogp = torch.randn((d_model, order), dtype=torch.float32, device="cuda")
R = torch.randn((d_model, order), dtype=torch.float32, device="cuda")
L = 1024
h = implicit_filter(glogp, R, L)
print(h.shape)  # torch.Size([128, 1024])