Skip to main content

When & Where to use Linear or Logistic regression?

Another post starts with you beautiful people!
First of all thank you everyone for visiting my blog and showing your keen interest in Linear and Logistic Regression topics of Machine Learning track!
Since many of you have asked a common but most important question- How to know when and where apply either Linear or Logistic regression?
Therefore I am going to share this post where I will try to resolve your doubt.
Linear and Logistic regressions are usually the first algorithms people learn in predictive modeling.
Each form has its own importance and a specific condition where they are best suited to apply-

What is Regression Analysis?

  • Regression analysis is a form of predictive modelling technique which investigates the relationship between a dependent (target) and independent variable (s) (predictor). 
  • This technique is used for forecasting, time series modelling and finding the causal effect relationship between the variables. 
  • For example, relationship between rash driving and number of road accidents by a driver is best studied through regression.
  • Regression analysis is an important tool for modelling and analyzing data. 
  • Here, we fit a curve / line to the data points, in such a manner that the differences between the distances of data points from the curve or line is minimized.

Why do we use Regression Analysis?
As mentioned above, regression analysis estimates the relationship between two or more variables. Let’s understand this with an easy example:
Let’s say, you want to estimate growth in sales of a company based on current economic conditions. 
You have the recent company data which indicates that the growth in sales is around two and a half times the growth in the economy. 
Using this insight, we can predict future sales of the company based on current & past information.

There are multiple benefits of using regression analysis. They are as follows:
  • It indicates the significant relationships between dependent variable and independent variable.
  • It indicates the strength of impact of multiple independent variables on a dependent variable.
  • Regression analysis also allows us to compare the effects of variables measured on different scales, such as the effect of price changes and the number of promotional activities. 
  • These benefits help market researchers / data analysts / data scientists to eliminate and evaluate the best set of variables to be used for building predictive .
1. Linear Regression-
  • It is one of the most widely known modeling technique. Linear regression is usually among the first few topics which people pick while learning predictive modeling. 
  • In this technique, the dependent variable is continuous, independent variable(s) can be continuous or discrete, and nature of regression line is linear.
  • Linear Regression establishes a relationship between dependent variable (Y) and one or more independent variables (X) using a best fit straight line (also known as regression line).
  • It is represented by an equation Y=a+b*X + e, where a is intercept, b is slope of the line and e is error term. 
  • This equation can be used to predict the value of target variable based on given predictor variable(s).
  • The difference between simple linear regression and multiple linear regression is that, multiple linear regression has (>1) independent variables, whereas simple linear regression has only 1 independent variable.
How to obtain best fit line (Value of a and b)?
  • This task can be easily accomplished by Least Square Method. It is the most common method used for fitting a regression line. 
  • It calculates the best-fit line for the observed data by minimizing the sum of the squares of the vertical deviations from each data point to the line. Because the deviations are first squared, when added, there is no cancelling out between positive and negative values.
Important Points about linear regression:
  • There must be linear relationship between independent and dependent variables.
  • Multiple regression suffers from multicollinearity, autocorrelation, heteroskedasticity.
  • Linear Regression is very sensitive to Outliers. It can terribly affect the regression line and eventually the forecasted values.
  • Multicollinearity can increase the variance of the coefficient estimates and make the estimates very sensitive to minor changes in the model. 
  • The result is that the coefficient estimates are unstable.
  • In case of multiple independent variables, we can go with forward selection, backward elimination and step wise approach for selection of most significant independent variables.
2. Logistic Regression-
  • Logistic regression is used to find the probability of event=Success and event=Failure. 
  • We should use logistic regression when the dependent variable is binary (0/ 1, True/ False, Yes/ No) in nature.
  • Since we are working here with a binomial distribution (dependent variable), we need to choose a link function which is best suited for this distribution. And, it is Logic Function. 
  • In the equation above, the parameters are chosen to maximize the likelihood of observing the sample values rather than minimizing the sum of squared errors (like in ordinary regression).
Important Points about logistic regression:
  • It is widely used for classification problems.
  • Logistic regression doesn’t require linear relationship between dependent and independent variables.  
  • It can handle various types of relationships because it applies a non-linear log transformation to the predicted odds ratio.
  • To avoid over fitting and under fitting, we should include all significant variables. 
  • A good approach to ensure this practice is to use a step wise method to estimate the logistic regression.
  • It requires large sample sizes because maximum likelihood estimates are less powerful at low sample sizes than ordinary least square.
  • The independent variables should not be correlated with each other i.e. no multi collinearity.  However, we have the options to include interaction effects of categorical variables in the analysis and in the model.
  • If the values of dependent variable is ordinal, then it is called as Ordinal Logistic regression.
  • If dependent variable is multi class then it is known as Multinomial Logistic regression.
I hope after reading and understanding the above key points you will get it easily!
Take a print out of this post and hold it infront of your desk so that whenever you are going to work on predicting a model, you have no doubt at all to choose between linear and logistic regression.
Labels:

Comments

  1. AnonymousOctober 5, 2020 at 10:32 PM

    very well explained. I would like to thank you for the efforts you had made for writing this awesome article. This article inspired me to read more. keep it up.
    Logistic Regression explained
    Correlation vs Covariance
    Simple Linear Regression

    ReplyDelete

Post a Comment

Popular posts from this blog

Learn the fastest way to build data apps

Another post starts with you beautiful people! I hope you have enjoyed and learned something new from my previous three posts about machine learning model deployment. In one post we have learned  How to deploy a model as FastAPI?  I n the second post, we have learned  How to deploy a deep learning model as RestAPI ? and in the third post, we have also learned  How to scale your deep learning model API?   If you are following my blog posts, you have seen how easily you have transit yourselves from aspiring to a mature data scientist. In this new post, I am going to share a new framework-  Streamlit which will help you to easily create a beautiful app with Python only. I will show here how had I used the Streamlit framework to create an app for my YOLOv3 custom model. What is Streamlit? Streamlit’s open-source app framework is the easiest way for data scientists and machine learning engineers to create beautiful, performant apps in only a few hours!...
49 comments
Read more

Machine Learning-Decision Trees and Random Forests

Another post starts with you beautiful people! I hope after reading my previous post about  Linear and Logistic Regression   your confidence level is up and you are now ready to move one step ahead in Machine Learning arena. In this post we will be going over Decision Trees and Random Forests . In order for you to understand this exercise completely there is some required reading. I suggests you to please read following blog post before going further- A Must Read! After reading the blog post you should have a basic layman's (or laywoman!) understanding of how decision trees and random forests work.  A quick intro is as below- Decision Trees (DTs) are a non-parametric supervised learning method used for classification and regression. The goal is to create a model that predicts the value of a target variable by learning simple decision rules inferred from the data features . For instance, in the example below, decision trees learn from data to approximate a sin...
12 comments
Read more

Generative AI: Retrieval Augmented Generation(RAG)

  Another blog post starts with you beautiful people👦. I hope you have explored my  last blog post  about 2x faster fine-tuning of Mistral 7b model on a custom dataset👈. In this blog post, we are going to learn an essential technique in Generative AI: Retrieval Augmented Generation (RAG). What is RAG? Retrieval Augmented Generation (RAG) is an innovative approach that melds generative models, like transformers, with a retrieval mechanism. By tapping into existing knowledge, RAG retrieves pertinent information from expansive external datasets or knowledge bases to enhance the generation process, thereby elevating the model's content relevance and factual accuracy💪. This versatility renders RAG particularly beneficial for tasks demanding the assimilation of external knowledge, such as question answering or content creation. Upon receiving input, RAG actively searches for relevant documents from specified sources (e.g., Wikipedia, company knowledge base, etc.). It th...
8 comments
Read more