{"id":2898,"date":"2025-10-10T13:17:58","date_gmt":"2025-10-10T13:17:58","guid":{"rendered":"https:\/\/blog.samarthya.me\/wps\/?p=2898"},"modified":"2025-10-10T13:18:00","modified_gmt":"2025-10-10T13:18:00","slug":"the-core-divide","status":"publish","type":"post","link":"https:\/\/blog.samarthya.me\/wps\/2025\/10\/10\/the-core-divide\/","title":{"rendered":"The Core Divide"},"content":{"rendered":"\n
\"\"<\/figure>\n\n\n\n

At the heart of machine learning, algorithms learn from data. The key distinction between supervised and unsupervised learning is the type of data<\/strong> used for training.<\/p>\n\n\n\n

Supervised Learning: Learning with a Teacher<\/h2>\n\n\n\n

Supervised learning algorithms are trained using a dataset that is labeled<\/strong>. This means for every input, there is a known, correct output (the “answer” or “label”). The algorithm’s job is to learn the mapping function from the input features to the output label.<\/p>\n\n\n\n

Characteristic<\/td>Description<\/td>Analogy<\/td><\/tr><\/thead>
Data<\/strong><\/td>Labeled Data<\/strong> (Input Features + Correct Output\/Label)<\/td>A student practicing math problems where they have the answer key<\/strong>.<\/td><\/tr>
Goal<\/strong><\/td>Predictive:<\/strong> Predict a known outcome on new, unseen data.<\/td>Predict the price of a house or whether an email is spam.<\/td><\/tr>
Main Tasks<\/strong><\/td>1. Classification<\/strong> (predict a discrete category)
2. Regression<\/strong> (predict a continuous value)<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n

When a machine learning algorithm is “trained” on data, it transforms from a generic mathematical blueprint<\/strong> into a specific, highly tuned statistical model<\/strong>.<\/p>– Google<\/cite><\/blockquote><\/figure>\n\n\n\n

Unsupervised Learning: Learning by Observation<\/h2>\n\n\n\n

Unsupervised learning algorithms are trained using a dataset that is unlabeled<\/strong>. There is no “answer key.” The algorithm’s job is to explore the data, find hidden patterns, and infer intrinsic structures or natural groupings without any guidance.<\/p>\n\n\n\n

Characteristic<\/td>Description<\/td>Analogy<\/td><\/tr><\/thead>
Data<\/strong><\/td>Unlabeled Data<\/strong> (Input Features only)<\/td>A student given a collection of different objects and asked to group them<\/strong> based on their characteristics.<\/td><\/tr>
Goal<\/strong><\/td>Descriptive:<\/strong> Discover patterns, structures, or natural groupings within the data.<\/td>Segment customers into groups with similar buying habits.<\/td><\/tr>
Main Tasks<\/strong><\/td>1. Clustering<\/strong> (grouping similar data points)
2. Dimensionality Reduction<\/strong> (simplifying data)<\/td>		<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n

Note<\/h2>\n\n\n\n
\n

The algorithm itself is the logic<\/strong> or the blueprint.<\/p>\n<\/blockquote>\n\n\n\n

Example: Linear Regression<\/strong><\/h3>\n\n\n\n
    \n
  • When you first define this algorithm<\/code>, the weights (wi<\/sub>) and the bias (b) are unknown<\/strong> (or randomly initialized).<\/li>\n\n\n\n
  • Algorithm (Blueprint):<\/strong> The logic is the formula y=w1<\/sub>\u200bx1<\/sub>\u200b+w2<\/sub>\u200bx2<\/sub>\u200b+\u22ef+b<\/code>.<\/li>\n\n\n\n
  • It defines a relationship where the output (y) is a sum of inputs (xi<\/sub>\u200b) multiplied by certain numbers called weights<\/strong> (wi\u200b<\/sub>), plus a constant bias<\/strong> (b).<\/li>\n<\/ul>\n\n\n\n

    This logic (the formula) is what the programmer writes. It’s an empty shell waiting to be filled.<\/p>\n\n\n\n

    \n

    “Training<\/code>” is the process of using data to find the optimal values for these unknown numbers (the weights and the bias) so that the formula y=\u2026 gives the most accurate output (y) for the given inputs (xi\u200b<\/sub>).<\/p>\n<\/blockquote>\n\n\n\n

    Step<\/td>What the Algorithm Does<\/td>What is Stored (The “Learning”)<\/td><\/tr><\/thead>
    Input<\/strong><\/td>Reads a batch of labeled training data (e.g., x1<\/sub>\u200b=2, x2<\/sub>\u200b=5, True Answer y=12).<\/td>Nothing yet\u2014just processing data.<\/em><\/td><\/tr>
    Prediction<\/strong><\/td>Uses the current (random) weights to calculate an output: ypred\u200b<\/sub>=w1<\/sub>\u200bx1<\/sub>\u200b+w2<\/sub>\u200bx2<\/sub>\u200b+b.<\/td>Nothing yet\u2014just making a guess.<\/em><\/td><\/tr>
    Error\/Loss<\/strong><\/td>Compares its prediction (ypred<\/sub>\u200b) to the true answer (ytrue<\/sub>\u200b) and calculates the error<\/strong> (or “loss”).<\/td>The calculated error value.<\/em><\/td><\/tr>
    Adjustment<\/strong><\/td>Uses an optimization function (like Gradient Descent) to calculate how much<\/em> each weight (wi\u200b<\/sub>) needs to be adjusted to reduce the error.<\/td>New values for w1<\/sub>\u200b,w2<\/sub>\u200b,\u2026,b<\/strong> (The core of the learned model).<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n

    This four-step cycle repeats, potentially thousands or millions of times, using the entire dataset. With each iteration, the weights and bias are slightly adjusted to reduce the overall error.<\/p>\n\n\n\n

    Once the training process is complete (the error is minimized), the algorithm’s job is done. The result is the Model<\/strong>.<\/p>\n\n\n\n

    \n

    Model (The Application):<\/strong> This is the saved set of final, optimized numerical values<\/strong> for all the internal parameters (weights and bias).<\/p>\n<\/blockquote>\n\n\n\n

    The trained PySpark LogisticRegressionModel<\/code>, for example, is just an object that holds an array of final coefficients (weights<\/code>) and an intercept (bias<\/code>).<\/p>\n\n\n\n

    When you use the model to make predictions on new data, it no longer learns<\/strong>. It simply takes the new input data, plugs it into the original formula (the blueprint), and uses the saved, learned weights<\/strong> to calculate the final prediction.<\/p>\n\n\n\n

    In summary, the algorithm is the potential<\/strong> for a mathematical relationship, and the model is the realized version<\/strong> of that relationship, represented by a set of final numbers learned from the data.<\/p>\n\n\n\n

    \"\"<\/figure>\n","protected":false},"excerpt":{"rendered":"

    At the heart of machine learning, algorithms learn from data. The key distinction between supervised and unsupervised learning is the type of data used for training. Supervised Learning: Learning with a Teacher Supervised learning algorithms are trained using a dataset that is labeled. This means for every input, there is a known, correct output (the […]<\/p>\n","protected":false},"author":2,"featured_media":2900,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_exactmetrics_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0,"footnotes":""},"categories":[347,34],"tags":[345,346],"class_list":["post-2898","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ml","category-technical","tag-ai","tag-ml"],"_links":{"self":[{"href":"https:\/\/blog.samarthya.me\/wps\/wp-json\/wp\/v2\/posts\/2898","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.samarthya.me\/wps\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.samarthya.me\/wps\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.samarthya.me\/wps\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.samarthya.me\/wps\/wp-json\/wp\/v2\/comments?post=2898"}],"version-history":[{"count":1,"href":"https:\/\/blog.samarthya.me\/wps\/wp-json\/wp\/v2\/posts\/2898\/revisions"}],"predecessor-version":[{"id":2902,"href":"https:\/\/blog.samarthya.me\/wps\/wp-json\/wp\/v2\/posts\/2898\/revisions\/2902"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blog.samarthya.me\/wps\/wp-json\/wp\/v2\/media\/2900"}],"wp:attachment":[{"href":"https:\/\/blog.samarthya.me\/wps\/wp-json\/wp\/v2\/media?parent=2898"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.samarthya.me\/wps\/wp-json\/wp\/v2\/categories?post=2898"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.samarthya.me\/wps\/wp-json\/wp\/v2\/tags?post=2898"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}