Exploring The File Import- Continue

Another post starts with you beautiful people!
I hope you have enjoyed my previous post about Exploring The File Import where we learned about flat file import.
But there are a number of datatypes that cannot be saved easily to flat files, such as lists and dictionaries.
In this exercise we will deal with Pickle,Excel,SAS and HDF5 files.

First we will see how to import the pickle file-
In this exercise, we'll import the pickle package, open a previously pickled data structure from a file and load it.You can find more details about pickle here-what is pickle?

Run the above code snippet in your notebook and discover what Python datatype it yields.

Second we see how to load a excel file-
Whether you like it or not, any working data scientist will need to deal with Excel spreadsheets at some point in time. You won't always want to do so in Excel, however!
Here, we'll learn how to use pandas to import Excel spreadsheets and how to list the names of the sheets in any loaded .xls file.
Specifically, we'll be loading and checking out the spreadsheet 'PRIO Battle Deaths Dataset 3.1.xls', modified from the Peace Research Institute Oslo's (PRIO) dataset.
This data contains age-adjusted mortality rates due to war in various countries over several years.

There may be more than 1 sheet in a xls file so in the below code snippet we learn how to read each sheet-

Third we will see how to load a SAS file and plot the data-
In this exercise, we'll figure out how to import a SAS file as a DataFrame using SAS7BDAT and pandas. Tell me more about sas7bdat

Result-

Next we will learn how to load a HDF5 file-
In this exercise, we'll import it using the h5py library [tell me more]. We'll also print out its datatype to confirm we have imported it correctly.You can find the LIGO data used in on Signal Processing plus loads of documentation and tutorials here-LIGO Dataset

Result-

Extracting data from your HDF5 file-
In this exercise, we'll extract some of the LIGO experiment's actual data from the HDF5 file and we'll visualize it. You can find more about this type of file here- What is HDF5?

Result-

How to load a MATLAB file-
In this exercise, we'll figure out how to load a MATLAB file using scipy.io.loadmat().

This file contains gene expression data from the Albeck Lab at UC Davis. You can find the data and some great documentation here-download dataset

Run the above code in your notebook and discover what Python datatype it yields.

Comments

How to use opencv-python with Darknet's YOLOv4?

Another post starts with you beautiful people 😊 Thank you all for messaging me your doubts about Darknet's YOLOv4. I am very happy to see in a very short amount of time my lovely aspiring data scientists have learned a state of the art object detection and recognition technique. If you are new to my blog and to computer vision then please check my following blog posts one by one- Setup Darknet's YOLOv4 Train custom dataset with YOLOv4 Create production-ready API of YOLOv4 model Create a web app for your YOLOv4 model Since now we have learned to use YOLOv4 built on Darknet's framework. In this post, I am going to share with you how can you use your trained YOLOv4 model with another awesome computer vision and machine learning software library- OpenCV and of course with Python 🐍. Yes, the Python wrapper of OpenCV library has just released it's latest version with support of YOLOv4 which you can install in your system using below command- pip install opencv-pyt...

How can I make a simple ChatBot?

Another post starts with you beautiful people! It has been a long time of posting a new post. But my friends in this period I was not sitting where I got a chance to work with chatbot and classification related machine learning problem. So in this post I am going to share all about chatbot- from where I have learned? What I have learned? And how can you build your first bot? Quite interesting right! Chatbot is a program that can conduct an intelligent conversation based on user's input. Since chatbot is a new thing to me also, I first searched- is there any Python library available to start with this? And like always Python has helped me this time also. There is a Python library available with name as ChatterBot which is nothing but a machine learning conversational dialog engine. And yes that is all I want to start my learning because I always prefer inbuilt Python library to start my learning journey and once I learn this then only I move ahead for another...

Generative AI: Retrieval Augmented Generation(RAG)

Another blog post starts with you beautiful people👦. I hope you have explored my last blog post about 2x faster fine-tuning of Mistral 7b model on a custom dataset👈. In this blog post, we are going to learn an essential technique in Generative AI: Retrieval Augmented Generation (RAG). What is RAG? Retrieval Augmented Generation (RAG) is an innovative approach that melds generative models, like transformers, with a retrieval mechanism. By tapping into existing knowledge, RAG retrieves pertinent information from expansive external datasets or knowledge bases to enhance the generation process, thereby elevating the model's content relevance and factual accuracy💪. This versatility renders RAG particularly beneficial for tasks demanding the assimilation of external knowledge, such as question answering or content creation. Upon receiving input, RAG actively searches for relevant documents from specified sources (e.g., Wikipedia, company knowledge base, etc.). It th...

Learn Data Science using Python

Search This Blog