Exploring The File Import- Continue

Another post starts with you beautiful people!
I hope you have enjoyed my previous post about Exploring The File Import where we learned about flat file import.
But there are a number of datatypes that cannot be saved easily to flat files, such as lists and dictionaries.
In this exercise we will deal with Pickle,Excel,SAS and HDF5 files.

First we will see how to import the pickle file-
In this exercise, we'll import the pickle package, open a previously pickled data structure from a file and load it.You can find more details about pickle here-what is pickle?

Run the above code snippet in your notebook and discover what Python datatype it yields.

Second we see how to load a excel file-
Whether you like it or not, any working data scientist will need to deal with Excel spreadsheets at some point in time. You won't always want to do so in Excel, however!
Here, we'll learn how to use pandas to import Excel spreadsheets and how to list the names of the sheets in any loaded .xls file.
Specifically, we'll be loading and checking out the spreadsheet 'PRIO Battle Deaths Dataset 3.1.xls', modified from the Peace Research Institute Oslo's (PRIO) dataset.
This data contains age-adjusted mortality rates due to war in various countries over several years.

There may be more than 1 sheet in a xls file so in the below code snippet we learn how to read each sheet-

Third we will see how to load a SAS file and plot the data-
In this exercise, we'll figure out how to import a SAS file as a DataFrame using SAS7BDAT and pandas. Tell me more about sas7bdat

Result-

Next we will learn how to load a HDF5 file-
In this exercise, we'll import it using the h5py library [tell me more]. We'll also print out its datatype to confirm we have imported it correctly.You can find the LIGO data used in on Signal Processing plus loads of documentation and tutorials here-LIGO Dataset

Result-

Extracting data from your HDF5 file-
In this exercise, we'll extract some of the LIGO experiment's actual data from the HDF5 file and we'll visualize it. You can find more about this type of file here- What is HDF5?

Result-

How to load a MATLAB file-
In this exercise, we'll figure out how to load a MATLAB file using scipy.io.loadmat().

This file contains gene expression data from the Albeck Lab at UC Davis. You can find the data and some great documentation here-download dataset

Run the above code in your notebook and discover what Python datatype it yields.

Comments

How to convert your YOLOv4 weights to TensorFlow 2.2.0?

Another post starts with you beautiful people! Thank you all for your overwhelming response in my last two posts about the YOLOv4. It is quite clear that my beloved aspiring data scientists are very much curious to learn state of the art computer vision technique but they were not able to achieve that due to the lack of proper guidance. Now they have learnt exact steps to use a state of the art object detection and recognition technique from my last two posts. If you are new to my blog and want to use YOLOv4 in your project then please follow below two links- How to install and compile Darknet code with GPU? How to train your custom data with YOLOv4? In my last post we have trained our custom dataset to identify eight types of Indian classical dance forms. After the model training we have got the YOLOv4 specific weights file as 'yolo-obj_final.weights'. This YOLOv4 specific weight file cannot be used directly to either with OpenCV or with TensorFlow currently becau...

Machine Learning-Cross Validation & ROC curve

Another post starts with you beautiful people! Hope you enjoyed my previous post about improving your model performance by confusion metrix . Today we will continue our performance improvement journey and will learn about Cross Validation (k-fold cross validation) & ROC in Machine Learning. A common practice in data science competitions is to iterate over various models to find a better performing model. However, it becomes difficult to distinguish whether this improvement in score is coming because we are capturing the relationship better or we are just over-fitting the data. To find the right answer of this question, we use cross validation technique. This method helps us to achieve more generalized relationships. What is Cross Validation? Cross Validation is a technique which involves reserving a particular sample of a data set on which we do not train the model. Later, we test the model on this sample before finalizing the model. Here are the steps involved in...

YOLObile- a new state of the art Real-Time Object Detection model for Mobile Devices

Another post starts with you beautiful people! Thanks for giving so many views on my previous post 👍. I am glad to see my previous posts are helping people to use state of the art object detection and recognition deep learning model in their projects. If you are new to my blog, I recommend seeing once my previous posts, and you will not be disappointed if your goal is to learn applied computer vision free of cost. Continuing my journey of sharing knowledge in this post I am going to share with you a new state of the art framework for object detection on mobile devices- YOLObile 📱 There has been a trade-off between speed and the accuracy of object detections. For example, the state of the art, YOLOv4 model gives us a very accurate detection but its speed is slow if we want to use it on a mobile device. On the other hand, its lighter version YOLOv4-tiny works very fast on a mobile device but its accuracy reduces. For a detailed comparison of FPS vs mAP you can ...

Learn Data Science using Python

Search This Blog