{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Numpy introduction\n",
"\n",
"## Tensor operations\n",
"\n",
"You **must** learn to code using tensor operation only.\n",
"In other word, you **should not** use for loops in your code when compute things.\n",
"The only for loops you need are already written!\n",
"\n",
"The reason is that it makes everything simpler to understand and that Numpy (or other libraries) can vectorize the operations on CPU (or GPU in the case of libraries like Pytorch) to accelarate computation.\n",
"\n",
"https://en.wikipedia.org/wiki/Array_programming"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"import numpy as np"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Numpy is one of the most popular numerical computation library in Python.\n",
"For this lab exercise we are mainly interested in tensor computation.\n",
"\n",
"It is really important that you take time to understand how Numpy works. A short tutorial is available here: https://cs231n.github.io/python-numpy-tutorial/\n",
"\n",
"Take time to do a few test, understand the different operation, the different between in-place and out-of-place operations, etc.\n",
"The most important resource you **must** use is the numpy documentation.\n",
"As we usually say in computer science: Read The F*cking Manual https://numpy.org/doc/stable/reference/index.html\n",
"\n",
"## Examples"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# create a 2D tensor of shape (2, 5) full of zeros\n",
"# by default the tensor will contain elements of type float\n",
"t = np.zeros((2, 5))\n",
"print(\"Shape of the tensor: \", t.shape)\n",
"print(\"Content:\")\n",
"print(t)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# You can reshape tensors\n",
"# When you reshape a tensor, it does not change the data order in the underlying memory.\n",
"# By default, this is the \"C array order\", also called the row-major format.\n",
"# If you don't know about this, check the wikipedia page:\n",
"# https://en.wikipedia.org/wiki/Row-_and_column-major_order\n",
"\n",
"# for example, the following operation will reshape t as a vector with ten elements\n",
"t = t.reshape(-1) # -1 means \"put every there\"\n",
"print(t.shape)\n",
"\n",
"# here instead of having a vector we build a tensor with a single row with ten elements\n",
"t = t.reshape(1, -1)\n",
"# of cours we could have done t = t.reshape(1, 10)\n",
"print(t.shape)\n",
"\n",
"# here a tensor with a single column with ten elements\n",
"t = t.reshape(-1, 1)\n",
"# of cours we could have done t = t.reshape(10, 1)\n",
"print(t.shape)\n",
"\n",
"# reshape into the original shape\n",
"t = t.reshape(2, -1)\n",
"print(t.shape)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# this creates a vector with values from 0 to 3 (not included)\n",
"t = np.arange(4)\n",
"print(t)\n",
"\n",
"# reshape\n",
"t = t.reshape((2, 2))\n",
"print(t)\n",
"\n",
"# .T returns the transposed tensor\n",
"print(t.T)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# set the first element of the second row to one and display the new data\n",
"# this is an in-place operation: it directly modifes the tensor memory\n",
"t[1, 0] = 1.\n",
"print(\"New content:\")\n",
"print(t)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# multiply the content of the tensor by two\n",
"# this is an out-of-place operation: it does not modify the tensory memory but creates a new one\n",
"t2 = 2 * t\n",
"\n",
"print(\"Original tensor:\")\n",
"print(t)\n",
"print(\"New tensor:\")\n",
"print(t2)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# do the same thing but in-place\n",
"t *= 2\n",
"print(\"New content:\")\n",
"print(t)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# There are two multiplication operators:\n",
"# * is the element wise multiplication operator (also called the Hadamard product)\n",
"# @ is the matrix multiplication operator\n",
"\n",
"a = np.arange(4).reshape(2, 2)\n",
"# one_like create a tensor with the same properties (i.e. type and shape) than the argument\n",
"# but filled with ones\n",
"b = 2 * np.ones_like(a) \n",
"\n",
"print(\"a:\")\n",
"print(a)\n",
"print(\"b:\")\n",
"print(b)\n",
"print()\n",
"\n",
"# element wise multiplication\n",
"c = a * b\n",
"print(\"a * b:\")\n",
"print(c)\n",
"print()\n",
"\n",
"# matrix multiplication\n",
"c = a @ b\n",
"print(\"a @ b:\")\n",
"print(c)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# you can easily retrieve one row or one column of a tensor\n",
"print(\"a:\")\n",
"print(a)\n",
"print()\n",
"\n",
"print(\"first row of a:\")\n",
"print(a[0])\n",
"print()\n",
"\n",
"print(\"first column of a:\")\n",
"print(a[:, 0])"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# the same approach can be used to update the data in-place\n",
"print(\"a:\")\n",
"print(a)\n",
"print()\n",
"\n",
"# set the second colums elements to 10\n",
"a[:, 1] = 10.\n",
"print(\"after update:\")\n",
"print(a)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"One of the most important feature you have to understand is **broadcasting**.\n",
"You can read the following article to understand operation broadcasting: https://numpy.org/devdocs/user/theory.broadcasting.html\n",
"\n",
"It is a very important concept that is really helpful in numpy and other numerical computation library.\n",
"The documentation often explain of broadcasting is implemented for a given operation, check for example the matrix multiplication page: https://numpy.org/doc/stable/reference/generated/numpy.matmul.html"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"a = np.arange(6).reshape(2, -1)\n",
"print(\"a: \")\n",
"print(a)\n",
"print()\n",
"\n",
"# we will multpliy the first row by 2 and the second row by 4 by using operation broadcasting\n",
"# np.array can be used to create a tensor from python data\n",
"b = np.array([2, 4]).reshape((2, 1))\n",
"c = a * b\n",
"\n",
"print(\"new tensor:\")\n",
"print(c)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.6"
}
},
"nbformat": 4,
"nbformat_minor": 4
}