{"id":2903,"date":"2025-10-16T14:14:21","date_gmt":"2025-10-16T14:14:21","guid":{"rendered":"https:\/\/blog.samarthya.me\/wps\/?p=2903"},"modified":"2025-10-16T14:16:58","modified_gmt":"2025-10-16T14:16:58","slug":"matrix-multiplication-to-gpus","status":"publish","type":"post","link":"https:\/\/blog.samarthya.me\/wps\/2025\/10\/16\/matrix-multiplication-to-gpus\/","title":{"rendered":"Matrix Multiplication to GPU’s"},"content":{"rendered":"\n
\"\"<\/figure>\n\n\n\n

If you are like me, you must have stumbled on this question – Why is matrix multiplication so significant – especially in the AI\/ML universe.<\/p>\n\n\n\n

How I arrived to this question?<\/h2>\n\n\n\n

When I first dove into Machine Learning (ML)<\/strong> and Deep Learning<\/strong>, I hit a wall. Terms like “backpropagation,” “Transformer models,” and “convolutional layers” flew past, making the whole process feel like abstract magic. I was proficient at running<\/em> the high-level Python code in frameworks like PyTorch, but I wasn’t truly understanding<\/em> what was happening under the hood. I felt like a technician blindly operating a complex machine, not an engineer who knew its blueprint.<\/p>\n\n\n\n

My initial struggle was with the sheer complexity. I needed to find the single, fundamental building block<\/strong> that all modern AI rested on.<\/p>\n\n\n\n

Aha! Moment<\/h2>\n\n\n\n

My breakthrough came with the stark realization that despite all the fancy names, the single most critical, repeated operation in a neural network is simply Matrix Multiplication<\/strong> (often seen as a Generalized Matrix Multiplication, or GEMM).<\/p>\n\n\n\n

Why is this operation so central?<\/p>\n\n\n\n

    \n
  • The Neuron’s Math:<\/strong> In a neural network, a single artificial neuron<\/strong> takes a set of inputs and multiplies each one by its corresponding weight<\/strong> before summing them up. Every step of a neural network’s forward pass\u2014from input to output\u2014is a weighted sum. In a fully connected layer, this is represented by multiplying the input data matrix<\/strong> (X) by the layer\u2019s weight matrix<\/strong> (W) to get a pre-activation output.\n
      \n
    • Input Data:<\/strong> Your image pixels, words, or feature data (represented as a vector<\/strong> or a matrix<\/strong>).<\/li>\n\n\n\n
    • Weights:<\/strong> The parameters the network learns during training. These are typically organized into a large Weight Matrix<\/strong>.<\/li>\n<\/ul>\n<\/li>\n\n\n\n
    • The Learning:<\/strong> Even during the learning process (backpropagation<\/code><\/strong>), the calculation of gradients (the adjustments for the weights) requires the extensive use of matrix transposes and products.\n
        \n
      • When a neural network learns, it adjusts its weights to minimize error. This process, called backpropagation<\/code><\/strong>, involves calculating how much each weight contributed to the final error using calculus (specifically, the chain rule).<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n\n\n\n

        I learned that every image classification, language generation, and stock prediction was, at its core, the result of billions of these matrix operations<\/strong> being executed in sequence. The challenge shifted from “What is a neural network?” to “How do we do these billions of matrix multiplications fast<\/em>?”<\/p>\n\n\n\n

        Matrix multiplication is the fundamental mathematical operation behind every layer of a neural network. Understanding this is key to understanding AI’s computational needs.<\/p><\/blockquote><\/figure>\n\n\n\n

        Because AI models need to perform this operation billions<\/em> of times on massive matrices (sometimes involving trillions of parameters for Large Language Models), an operation that is slow on a sequential CPU becomes instantly accelerated on a parallel GPU.<\/p>\n\n\n\n

        The nature of matrix multiplication is perfectly parallelizable<\/strong>: each element in the output matrix can be computed independently, making it an ideal task for the GPU’s thousands of cores and specialized Tensor Cores<\/strong>.<\/p>\n\n\n\n

        \n\n\n\n
        \n