Skip to main content

Principal Component Analysis or PCA in machine learning


Another post starts with you beautiful people!
I hope you have enjoyed and must learn something from my previous post about Cross Validation & ROC.
In this post we are going to learn Principal Component Analysis or POC.
Principal Component Analysis (PCA) is a simple yet popular and useful linear transformation technique that is used in numerous applications, such as stock market predictions, the analysis of gene expression data, and many more.
The main idea of principal component analysis (PCA) is to reduce the dimensionality of a data set consisting of many variables correlated with each other, either heavily or lightly, while retaining the variation present in the dataset, up to the maximum extent.

Importantly, the dataset on which PCA technique is to be used must be scaled. The results are also sensitive to the relative scaling. As a layman, it is a method of summarizing data.
Imagine some wine bottles on a dining table. Each wine is described by its attributes like colour, strength, age, etc. But redundancy will arise because many of them will measure related properties. So what PCA will do in this case is summarize each wine in the stock with less characteristics.

In other words, Principal component analysis is a statistical procedure that uses an orthogonal transformation to convert a set of observations of possibly correlated variables into a set of values of linearly uncorrelated variables called principal components.

  • With PCA we can reduce the dimentions with out losing much information
  • PCA also helps to remove the multicollinearity between the variables

Dataset information:-
We will use SPECTF dataset from UCI machine learning repository[download here]
The dataset describes diagnosing of cardiac Single Proton Emission Computed Tomography (SPECT) images. Each of the patients is classified into two categories: normal and abnormal. 
The database of 267 SPECT image sets (patients) was processed to extract features that summarize the original SPECT images. As a result, 44 continuous feature pattern was created for each patient.





Let's train a LogisticRegression model and record the time taken to train before applying PCA:-

Standardising the variables:-


Result:-


This cumilative explained variance graph helps us to choose the number of desired principal components.
90% variation in the data is explaining by the first 15 principal components.

Result:-

PCA transforms a set of correlated variables into a set of linearly uncorrelated variables called principal components, we can check the correlarion with a heat map of correlation matrix.



Check the performance after considering the first 15 principal components:-

We can conclude that the computational time is reduced by several times after applying PCA and selecting 15 principal components, And the variables are transformed to a new set of linearly uncorrelated variables.

Comments

  1. meenatiNovember 1, 2019 at 8:05 PM

    I like your blog, I read this blog please update more content on hacking,Nice post
    Data Science online Training

    ReplyDelete
  2. propidduozoMay 21, 2022 at 12:34 PM

    propidduozo Shannon Romero download
    wirkruptbunsper

    ReplyDelete

Post a Comment

Popular posts from this blog

How to install and compile YOLO v4 with GPU enable settings in Windows 10?

Another post starts with you beautiful people! Last year I had shared a post about  installing and compiling Darknet YOLOv3   in your Windows machine and also how to detect an object using  YOLOv3 with Keras . This year on April' 2020 the fourth generation of YOLO has arrived and since then I was curious to use this as soon as possible. Due to my project (built on YOLOv3 :)) work I could not find a chance to check this latest release. Today I got some relief and successfully able to install and compile YOLOv4 in my machine. In this post I am going to share a single shot way to do the same in your Windows 10 machine. If your machine does not have GPU then you can follow my  previous post  by just replacing YOLOv3 related files with YOLOv4 files. For GPU having Windows machine, follow my steps to avoid any issue while building the Darknet repository. My machine has following configurations: Windows 10 64 bit Intel Core i7 16 GB RAM NVIDIA GeForce G...
46 comments
Read more

How to CORRECTLY convert Darknet/Yolov4 weights to TFLite format?

  Greetings, computer vision practitioners👮! I hope you're all doing well in your projects. Recently, I received a new requirement from a client regarding an object detection solution that had been successfully deployed as a REST API. The solution was seamlessly integrated into both web and mobile applications. However, the client encountered network reliability issues, particularly in rural areas where poor connectivity occasionally caused the solution to fail on mobile devices💔. As we know, network disruptions are a common challenge, but they become critical when they impact business operations. My client sought a robust solution to address this issue, ensuring uninterrupted functionality regardless of network conditions. In this blog post, we will explore how I tackled this challenge and implemented an effective solution💓. The Solution: Enabling Offline Object Detection for Mobile Devices Given that network connectivity issues are beyond our control, the most effective soluti...
11 comments
Read more

Generative AI: Faster fine-tuning of a Mistral Model with less memory

  Another blog post starts with you beautiful people💥. I hope you have started your Generative AI learning from my  last post  and if you have not, then I recommend you read that one before proceeding to this blog.  In this post, we will delve into the transformative power of generative AI, specifically exploring the cutting-edge Mistral 7B model fine-tuning🚀. This revolutionary technology has not only redefined the boundaries of artificial intelligence but also sparked a paradigm shift in how we approach data generation and creativity. Join me on a journey through the fascinating intersection of machine learning and creativity, where Mistral 7B stands as a beacon of innovation, pushing the boundaries of what's possible in the realm of generative AI👈. Fine-tuning a large language model(LLM) refers to the process of training the model on a specific, smaller dataset to adapt it to a particular task or domain. Large language models, like ChatGPT, are pre-trained on ...
4 comments
Read more