The Tensor: The fundamental data structure of ML

#webdev #machinelearning #javascript #python

In the field of computing, data is crucial and the way we store it is vital. Data structures are a means of organizing and storing data within a program, making it possible to manipulate the data using algorithms.

As a web developer, you may be familiar with the data structure known as the array, but in the field of machine learning, a crucial data structure is the tensor. Tensors are multi-dimensional arrays that can store and manipulate large amounts of data, making them essential for deep learning algorithms like neural networks, which rely on them to perform complex mathematical operations and achieve desired results.

What is a Tensor

In many ways, a tensor is similar to an array - in fact, an array can be thought of as a type of tensor. However, a tensor is a multidimensional data structure where all values in it share the same type.

Tensors can be categorized by their ranks, which represent the number of dimensions in the tensor. Additionally, an important attribute of a tensor is its shape, which describes the size of each dimension.
Two tensors can be of the same rank but have different shapes. An analogy to understand tensors is to think of a library containing books and shelves. Let's explore the different tensor ranks.

Rank-0 Tensor

A rank-0 tensor is also known as a scalar, and it has the lowest rank of all tensors. This type of tensor is equivalent to a variable and represents a single value. Rank-0 tensors have no shape because they are just individual values, but they do have a data type.

Using the library analogy, a rank-0 tensor can be thought of as a single book taken down from a shelf.

let r0 = 10 // a variable is equivalent to a rank-zero tensor
typeof r0 // this has a type of number

Rank-1 Tensor

A rank-1 tensor is equivalent to a one-dimensional array and can be thought of as a row of books on a shelf. All values in a rank-1 tensor must be of the same data type. The shape of a rank-1 tensor is defined by its length. It is also known as a vector.

let r1 = [1, 2, 3, 4] // an array is equivalent to a rank-1 tensor
r1.length // This can be considered a rank-1 tensor with 4 scalars

Rank-2 Tensor

A tensor with a rank of 2 is equivalent to a bookshelf with multiple rows stacked upon each other. It is an array of arrays. All rank 2 tensors vary by their shape which is two-dimensional. The example below has the shape of 3 by 3 or simply (3, 3).

[[1, 2, 3],
 [4, 5, 6],
 [7, 8, 9],]

A rank 2 tensor with the shape of 2 by 3 or (2, 3):

[[1, 2],
 [3, 4],
 [5, 6]]

A rank-2 tensor is also known as a matrix.

Rank-3 Tensor

A rank-3 tensor is an array of matrices or rank-2 tensors. The shape of a rank-3 tensor is three-dimensional, and it can vary depending on the number of rows, columns, and depth.

Using the library analogy, a rank-3 tensor can be visualized as a library with several shelves of books. The shelves can represent individual matrices that have their corresponding heights and lengths representing their rows and columns.

A rank-3 tensor with the shape of (3, 3, 2) and a data type of strings would be an array of two 3 by 3 matrices, as shown in the example below:

[[['A', 'B', 'C'],
  ['D', 'E', 'F'],
  ['G', 'I', 'J'],],

 [['K', 'L', 'M'],
  ['N', 'O', 'P'],
  ['Q', 'R', 'S'],]]

Tensors can have ranks beyond rank-3. The idea is the same for higher ranks, where each rank contains an array of the previous rank.
The number of dimensions in a tensor increases with its rank, and the shape of each rank will depend on the amount of data being stored in it.

Why Tensor?

Tensors are similar to arrays in that they store data, but what makes them special is that they are optimized for mathematical operations.
While arrays are mainly designed for storage, and can only perform simple operations like pushing or popping elements, tensors are more versatile and can perform a wide variety of mathematical operations.

In JavaScript, there is no built-in tensor data structure, but we can use a library like TensorFlow.js to work with them.
TensorFlow.js also provides a set of tensor operations that enable us to perform complex mathematical computations on tensors.

For example, we can use the tf.tensor1d method to create a rank-1 tensor, and then use the add method to add it to another tensor of the same shape and type:

let a = tf.tensor1d([1, 2, 3, 4]); 
let b = tf.tensor1d([5, 6, 7, 8]);
a.add(b) // [6, 8, 10, 12]

Tensors in the real world.

Tensors can be used to represent the real-world. Tensors have a versatile structure that enables the representation of all kinds of data. They are used with algorithms like neural networks to perform tasks like Image classification, Object detection, and Natural Language Processing. Here are a few ways of using tensors to represent real-world data:

Images

Images can be represented as tensors.The smallest unit of an image is called the pixel which can be represented as a rank-0 tensor.

A greyscale image is an image that consists of a matrix(rank-2 tensor) with pixels as values. Each pixel has an intensity value that determines if it will be dark or light. its shape is determined by its width and height.

Coloured images can be represented as rank-3 tensors which consist of three sets of images called channels. The three channels represent the intensity of the colors red, green, and blue, respectively.

Videos

Videos can be represented as rank-4 tensors. Videos are made up of stacks of images called frames.

Each frame of the video is a tensor of rank 3 that contains matrices of the 3 color channels.

In this article, I have introduced the fundamental data structure of machine learning, the tensor. In the next article, I will dive deeper into the library known as TensorFlow.js, which is a machine-learning library for JavaScript.