linalg.matrix_norm in PyTorch

#pytorch #norm #matrix #regularization

*Memos:

My post explains linalg.norm().
My post explains linalg.vector_norm().

linalg.matrix_norm() can get the 0D or more D tensor of the one or more elements computed with matrix norm from the 2D or more D tensor of zero or more elements as shown below:

*Memos:

linalg.matrix_norm() can be used with torch but not with a tensor.
The 1st argument with torch is input(Required-Type:tensor of float or complex): *Memos:
- It must be the 2D or more D tensor of zero or more elements.
- A complex tensor returns a float tensor even if dtype=torch.complex64 is set to linalg.norm() which is a bug.
The 2nd argument with torch is ord(Optional-Default:None-Type:int, float or str). *It sets a norm.
The 3rd argument with torch is dim(Optional-Default:None-Type:tuple or list of int). *The length of tuple or list of int must be 2.
The 4th argument with torch is keepdim(Optional-Default:False-Type:bool). *My post explains keepdim argument.
There is dtype argument with torch(Optional-Default:None-Type:dtype): *Memos:
- If it's None, it's inferred from input.
- dtype= must be used.
- My post explains dtype argument.
There is out argument with torch(Optional-Default:None-Type:tensor): *Memos:
- out= must be used.
- My post explains out argument.
ord supports the following norms: *Memos:
- inf can be torch.inf, float('inf'), etc:
- For vector, there are also L3 norm(ord=3), L4 norm(ord=4) etc.

`ord`	Matrix norm
`'fro'`(Default)	Frobenius norm
`'nuc'`	Nuclear norm
`inf`	`max(sum(abs(x), dim=1))`
`-inf`	`min(sum(abs(x), dim=1))`
`0`	Not supported
`1`	`max(sum(abs(x), dim=0))`
`-1`	`min(sum(abs(x), dim=0))`
`2`	The largest singular value of SVD(Singular Value Decomposition)
`-2`	The smallest singular value of SVD
Other `int` or `float`	Not supported

import torch
from torch import linalg

my_tensor = torch.tensor([[-3., -2., -1., 0.],
                          [1., 2., 3., 4.]])
linalg.matrix_norm(input=my_tensor) # Frobenius norm
# tensor(6.6332)

linalg.matrix_norm(input=my_tensor, ord=1)
# tensor(4.)

linalg.matrix_norm(input=my_tensor, ord=-1)
# tensor(4.)

linalg.matrix_norm(input=my_tensor, ord=2)
# tensor(5.8997)

linalg.matrix_norm(input=my_tensor, ord=-2)
# tensor(3.0321)

linalg.matrix_norm(input=my_tensor, ord=torch.inf)
# tensor(10.)

linalg.matrix_norm(input=my_tensor, ord=-torch.inf)
# tensor(6.)

linalg.matrix_norm(input=my_tensor, ord='fro') # Frobenius norm
# tensor(6.6332)

linalg.matrix_norm(input=my_tensor, ord='nuc') # Nuclear norm
# tensor(8.9318)

my_tensor = torch.tensor([[[-3., -2.], [-1., 0.]],
                          [[1., 2.], [3., 4.]]])
linalg.matrix_norm(input=my_tensor) # Frobenius norm
# tensor([3.7417, 5.4772])

linalg.matrix_norm(input=my_tensor, ord=1)
linalg.matrix_norm(input=my_tensor, ord=1, dim=(1, 2))
linalg.matrix_norm(input=my_tensor, ord=1, dim=(-2, -1))
# tensor([4., 6.])

linalg.matrix_norm(input=my_tensor, ord=-1)
linalg.matrix_norm(input=my_tensor, ord=-1, dim=(1, 2))
linalg.matrix_norm(input=my_tensor, ord=-1, dim=(-2, -1))
# tensor([2., 4.])

linalg.matrix_norm(input=my_tensor, ord=2)
linalg.matrix_norm(input=my_tensor, ord=2, dim=(1, 2))
linalg.matrix_norm(input=my_tensor, ord=2, dim=(-2, -1))
# tensor([3.7025, 5.4650])

linalg.matrix_norm(input=my_tensor, ord=-2)
linalg.matrix_norm(input=my_tensor, ord=-2, dim=(1, 2))
linalg.matrix_norm(input=my_tensor, ord=-2, dim=(-2, -1))
# tensor([0.5402, 0.3660])

linalg.matrix_norm(input=my_tensor, ord=torch.inf)
linalg.matrix_norm(input=my_tensor, ord=torch.inf, dim=(1, 2))
linalg.matrix_norm(input=my_tensor, ord=torch.inf, dim=(-2, -1))
# tensor([5., 7.])

linalg.matrix_norm(input=my_tensor, ord=-torch.inf)
linalg.matrix_norm(input=my_tensor, ord=-torch.inf, dim=(1, 2))
linalg.matrix_norm(input=my_tensor, ord=-torch.inf, dim=(-2, -1))
# tensor([1., 3.])

# Frobenius norm
linalg.matrix_norm(input=my_tensor, ord='fro')
linalg.matrix_norm(input=my_tensor, ord='fro', dim=(1, 2))
linalg.matrix_norm(input=my_tensor, ord='fro', dim=(-2, -1))
# tensor([3.7417, 5.4772])

# Nuclear norm
linalg.matrix_norm(input=my_tensor, ord='nuc')
linalg.matrix_norm(input=my_tensor, ord='nuc', dim=(1, 2))
linalg.matrix_norm(input=my_tensor, ord='nuc', dim=(-2, -1))
# tensor([4.2426, 5.8310])

my_tensor = torch.tensor([[[-3.+0.j, -2.+0.j], [-1.+0.j, 0.+0.j]],
                          [[1.+0.j, 2.+0.j], [3.+0.j, 4.+0.j]]])
linalg.matrix_norm(input=my_tensor, dtype=torch.complex64) # Frobenius norm
# tensor([3.7417, 5.4772])

DEV Community

linalg.matrix_norm in PyTorch

Top comments (0)

Read next

Work Life Balance

if else

if()

Conditional operator