{"cells":[{"cell_type":"markdown","source":["\n","\n","## **Problems with NumPy Broadcasting**"],"metadata":{"id":"aFN-BWJ4Znet"},"id":"aFN-BWJ4Znet"},{"cell_type":"code","execution_count":null,"id":"d900bf88-153d-43a3-96da-64b7f2ee49a5","metadata":{"id":"d900bf88-153d-43a3-96da-64b7f2ee49a5"},"outputs":[],"source":["import numpy as np"]},{"cell_type":"markdown","source":["### **a. Element-wise multiplication**\n"],"metadata":{"id":"A1piy7dPftRn"},"id":"A1piy7dPftRn"},{"cell_type":"markdown","source":["When working with 1D and 2D arrays in NumPy, a noteworthy challenge arises.\n","\n","When you element-wise multiply a column vector with a shape of (a, 1) and a row vector with a shape of (1, a) in NumPy, the result is a 2D matrix with dimensions (a, a) and not a vector. This occurs due to NumPy's implicit broadcasting of both vectors to 2D matrices before performing element-wise multiplication.\n","\n","The same broadcasting behavior occurs if the second input is a 1D array of shape (a,) instead of a row vector. NumPy treats 1D arrays as row vectors during operations, leading to the same outcome.\n","\n","The only case where expected vector algebra behavior occurs is when both inputs are either column vectors or row vectors/1D arrays, as their shapes broadcast consistently.\n","\n","It is very important to examine the shapes following elementwise multiplication when column vectors are involved."],"metadata":{"id":"G8Ebamd5QVLx"},"id":"G8Ebamd5QVLx"},{"cell_type":"code","source":["# Creating a 1D NumPy array from the list\n","numbers_list = [1, 2, 3, 4]\n","array_1 = np.array(numbers_list)\n","print(\"Array 1 is \\n\", array_1, \" with dimensions \", array_1.shape)\n","\n","# Creating another 1D NumPy array from the list\n","numbers_list = [5, 6, 7, 8]\n","array_2 = np.array(numbers_list)\n","print(\"Array 2 is \\n\", array_2, \" with dimensions \", array_2.shape)\n","\n","# Creating a 2D array with a column vector\n","column_vector = np.array([[1], [2], [3], [4]])\n","print(\"Column vector is \\n\", column_vector, \" with dimensions \", column_vector.shape)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"qgxTKfjzPwDz","outputId":"f98ed134-eb82-4cb0-e347-c86ad5593e7e"},"id":"qgxTKfjzPwDz","execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["Array 1 is \n"," [1 2 3 4]  with dimensions  (4,)\n","Array 2 is \n"," [5 6 7 8]  with dimensions  (4,)\n","Column vector is \n"," [[1]\n"," [2]\n"," [3]\n"," [4]]  with dimensions  (4, 1)\n"]}]},{"cell_type":"code","source":["# CASE 1: Performing Element-wise multiplication of two 1D arrays: Array_1 and Array_2\n","result_array_1 = array_1 * array_2\n","print(\"The result of element-wise multiplication is:\", result_array_1,\" with dimensions \",result_array_1.shape)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"XxrwGpfbUVPP","outputId":"e7dc5907-fd08-4f19-a55c-fb3d304b359f"},"id":"XxrwGpfbUVPP","execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["The result of element-wise multiplication is: [ 5 12 21 32]  with dimensions  (4,)\n"]}]},{"cell_type":"code","source":["# CASE 2: Performing Element-wise multiplication of a column_vector and a 1D array(array_1)\n","result_array_2 = column_vector * array_1\n","print(\"The result of element-wise multiplication is:\\n\", result_array_2,\" with dimensions \",result_array_2.shape)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"BnPin97NUEGJ","outputId":"a6b6f300-bbfd-4c58-e2a0-cb5120ae5652"},"id":"BnPin97NUEGJ","execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["The result of element-wise multiplication is:\n"," [[ 1  2  3  4]\n"," [ 2  4  6  8]\n"," [ 3  6  9 12]\n"," [ 4  8 12 16]]  with dimensions  (4, 4)\n"]}]},{"cell_type":"markdown","source":["\n","When NumPy uses the @ operator to multiply two 1D arrays, it treats one of them as a row vector and the other as a column vector, resulting in a scalar value."],"metadata":{"id":"dTYp9bL3EhSo"},"id":"dTYp9bL3EhSo"},{"cell_type":"code","source":["# CASE 3: Performing Matrix multiplication of two 1D arrays: Array_1 and Array_2\n","\n","result_array_3 = array_1 @ array_2\n","print(\"The result of matrix multiplication is:\", result_array_3)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"v4tVAh5k_vk4","outputId":"c5eddaad-4358-4ffa-e3fe-381ba6739fb5"},"id":"v4tVAh5k_vk4","execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["The result of matrix multiplication is: 70\n"]}]},{"cell_type":"markdown","source":["### **b. Element-wise Addition**\n"],"metadata":{"id":"_uWt0uXLf4BL"},"id":"_uWt0uXLf4BL"},{"cell_type":"markdown","source":["Similar broadcasting behavior what we saw above is observed in the context of element-wise addition"],"metadata":{"id":"sCjQeiknf_Km"},"id":"sCjQeiknf_Km"},{"cell_type":"code","source":["# CASE 4: Performing Element-wise Addition of two  1D arrays: Array_1 and Array_2\n","\n","result_array_4 = array_1 + array_2\n","print(\"The result of element-wise Addition is:\", result_array_4,\" with dimensions \",result_array_4.shape)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"FOqNIsKEgBxh","outputId":"6582709c-2ba6-4f19-e6bf-7a878121ea32"},"id":"FOqNIsKEgBxh","execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["The result of element-wise Addition is: [ 6  8 10 12]  with dimensions  (4,)\n"]}]},{"cell_type":"code","source":["# CASE 5: Performing Element-wise Addition of a column_vector and a 1D array(array_1)\n","\n","result_array_5 = column_vector + array_1\n","print(\"The result of element-wise Addition is:\\n\", result_array_5,\" with dimensions \",result_array_5.shape)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"rsLWoTwIgBzc","outputId":"5c27524c-b29d-45fa-dfa3-5b3866f1787b"},"id":"rsLWoTwIgBzc","execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["The result of element-wise Addition is:\n"," [[2 3 4 5]\n"," [3 4 5 6]\n"," [4 5 6 7]\n"," [5 6 7 8]]  with dimensions  (4, 4)\n"]}]},{"cell_type":"markdown","source":["### **c. Swapping matrix multiplication (@) with element wise multiplication (*)**\n"],"metadata":{"id":"opUugd36mISJ"},"id":"opUugd36mISJ"},{"cell_type":"markdown","source":["Consider a scenario where you want to perform a matrix multiplication. However, by mistake you use an asterisk * instead of a matrix multiplication operator (@). Despite the incorrect operator, the code will still run without error due to the broadcasting functionality in Python. The shapes matching might lead you to think that what you are doing is right whereas in reality, a matrix multiplication was required. You will get wrong values."],"metadata":{"id":"2bMbG_pym4EI"},"id":"2bMbG_pym4EI"},{"cell_type":"code","source":["# Create a 4x4 2D array with random values between 0 and 1\n","array_3 = np.random.rand(4, 4)\n","print(\"Randomly Generated 2D Array is :\\n\", array_3,\" with dimensions \", array_3.shape)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"yWQM08ykpSm6","outputId":"fb603590-0dd5-451f-afa7-461fe6d80486"},"id":"yWQM08ykpSm6","execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["Randomly Generated 2D Array is :\n"," [[0.96596709 0.64017236 0.28261285 0.57271614]\n"," [0.85576492 0.2828296  0.05598984 0.54849124]\n"," [0.72233659 0.2010986  0.77504161 0.55601639]\n"," [0.80827201 0.75502135 0.9870186  0.85875641]]  with dimensions  (4, 4)\n"]}]},{"cell_type":"code","source":["# CASE 6: Matrix multiplication (@) involving a 1D array acting as a row vector and a 2D array\n","result_array_6 = array_1 @ array_3\n","print(\"The result of Matrix multiplication is:\\n\", result_array_6 ,\" with dimensions \",result_array_6.shape)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"wMWbcDlPp8Zq","outputId":"b0b06e01-d98c-48d1-c2c6-b94bd83139ae"},"id":"wMWbcDlPp8Zq","execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["The result of Matrix multiplication is:\n"," [8.07759472 4.82921276 6.66779175 6.77277346]  with dimensions  (4,)\n"]}]},{"cell_type":"code","source":["# CASE 7: Substituting matrix multiplication (@) with element-wise multiplication (*) still functions due to Broadcasting\n","\n","result_array_7 = array_1 * array_3\n","print(\"The result of element-wise multiplication is:\\n\", result_array_7 ,\" with dimensions \",result_array_7.shape)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"0WuRq8RBj3A5","outputId":"7d04bd4e-cc32-4afd-fcb9-7ef7fa3bb48e"},"id":"0WuRq8RBj3A5","execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["The result of element-wise multiplication is:\n"," [[0.96596709 1.28034472 0.84783854 2.29086456]\n"," [0.85576492 0.5656592  0.16796952 2.19396498]\n"," [0.72233659 0.4021972  2.32512484 2.22406557]\n"," [0.80827201 1.5100427  2.96105579 3.43502566]]  with dimensions  (4, 4)\n"]}]},{"cell_type":"markdown","source":["Similar observations can be made when using PyTorch instead of NumPy for matrix multiplication"],"metadata":{"id":"aU3iRFXWUZSt"},"id":"aU3iRFXWUZSt"},{"cell_type":"code","source":["import torch"],"metadata":{"id":"4HUHBURV0wsC"},"id":"4HUHBURV0wsC","execution_count":null,"outputs":[]},{"cell_type":"code","source":["#Create a 1D tensor\n","\n","tensor1 = torch.tensor([1, 2, 3, 4])\n","print(\"Shape of tensor1:\", tensor1.shape)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"3n7F8aYf09hX","outputId":"c7324a90-c9df-4d36-87dc-3e47d3985f3d"},"id":"3n7F8aYf09hX","execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["Shape of tensor1: torch.Size([4])\n"]}]},{"cell_type":"code","source":["#Create a random 2D tensor\n","\n","tensor2 = torch.randint(0, 10, (4, 4))\n","print(\"Shape of tensor2:\", tensor2.shape)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"Mge-65Tp1C_q","outputId":"a1f29789-27dc-4561-b8df-49197ad3c4cb"},"id":"Mge-65Tp1C_q","execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["Shape of tensor2: torch.Size([4, 4])\n"]}]},{"cell_type":"code","source":["# CASE 8: Matrix multiplication (@) involving a 1D array acting as a row vector and a 2D array\n","\n","matrix_product = tensor1 @ tensor2\n","print(\"Matrix multiplication result:\\n\", matrix_product, \" with dimensions \",matrix_product.shape)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"O94YrpAk1NeR","outputId":"59e835a8-8585-4186-eea1-c7ed2d6e5945"},"id":"O94YrpAk1NeR","execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["Matrix multiplication result:\n"," tensor([53, 16, 44, 43])  with dimensions  torch.Size([4])\n"]}]},{"cell_type":"code","source":["# CASE 9: Substituting matrix multiplication (@) with element-wise multiplication (*) still functions due to Broadcasting\n","\n","elementwise_product = tensor1 * tensor2\n","print(\"Element-wise multiplication result:\\n\", elementwise_product, \" with dimensions \",elementwise_product.shape)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"Il56LSgOXPbE","outputId":"788d4aec-208d-45ab-8afd-11ac687a443c"},"id":"Il56LSgOXPbE","execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["Element-wise multiplication result:\n"," tensor([[ 7,  8, 15,  0],\n","        [ 9,  0,  3,  0],\n","        [ 8,  8, 21, 20],\n","        [ 1,  0, 12, 28]])  with dimensions  torch.Size([4, 4])\n"]}]},{"cell_type":"code","source":[],"metadata":{"id":"tBC2TAwoLC5R"},"id":"tBC2TAwoLC5R","execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["# JAX"],"metadata":{"id":"fxroA-6uLMwI"},"id":"fxroA-6uLMwI"},{"cell_type":"markdown","source":["JAX is a library for array-oriented numerical computation (à la NumPy), with automatic differentiation and JIT compilation to enable high-performance machine learning research.\n","\n","This document provides a quick overview of essential JAX features, so you can get started with JAX quickly:\n","\n","JAX provides a unified NumPy-like interface to computations that run on CPU, GPU, or TPU, in local or distributed settings.\n","\n","JAX features built-in Just-In-Time (JIT) compilation via Open XLA, an open-source machine learning compiler ecosystem.\n","\n","JAX functions support efficient evaluation of gradients via its automatic differentiation transformations.\n","\n","JAX functions can be automatically vectorized to efficiently map them over arrays representing batches of inputs."],"metadata":{"id":"BRTxothaLKbd"},"id":"BRTxothaLKbd"},{"cell_type":"code","source":["Further Reading:https://jax.readthedocs.io/en/latest/quickstart.html"],"metadata":{"id":"j3mEK7i8LL18"},"id":"j3mEK7i8LL18","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3 (ipykernel)","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.9.6"},"colab":{"provenance":[]}},"nbformat":4,"nbformat_minor":5}