Introduction

So far in the course, we have learnt quite a bit about DataFrames. In particular, we learnt about using various boolean and arithmetic operations on DataFrame columns, and also about indexing to select and modify various subsets of a DataFrame.

In this tutorial we will learn another method for doing operations on and also modifying a DataFrame using DataFrame methods like apply() and applymap(). These methods allow us to apply a function over an entire DataFrame.

Let's get started!

Set up

As in the previous tutorials, let us load the Pandas and Numpy libraries at the beginning and change the Pandas display options.

type=codeblock|id=pd_func_import|autocreate=datascience|show_output=0

Introduction

We have already learned in previous tutorials, how to select rows, columns or a slice of DataFrame with row index, column labels or Boolean indexing.

In this tutorial, we will use these techniques to select a subset of the DataFrame and modify the selected data.

Set-up

As in the previous tutorials, let's start by importing the libraries and loading the dataset.

We will use the same dataset that we used in Indexing and Slicing and Criteria Based Selection tutorials.

type=codeblock|id=pd_modeindex_load_adult|autocreate=datascience|show_output=1
import pandas as pd

Introduction

In the previous tutorial, you have used DataFrame which is a 2-dimensional data structure supported by Pandas, which looks and behaves like a table.

In this tutorial, you will learn about Series which is a 1-dimensional data structure supported by Pandas. In fact, in Pandas, each column of a DataFrame is a Series.

We will also learn in depth about creating DataFrames, adding rows to a DataFrame and converting DataFrames to other formats, such as NumPy arrays or Python lists and dictionaries.

Imports

Before we start, let's import the Pandas and NumPy libraries, as we will need them throughout the tutorial.

type=codeblock|id=pd_ce_import_all|autocreate=datascience|show_output=0

Introduction

In this tutorial, we will learn how we can handle datetime data with pandas.

We will also use our previous knowledge in indexing DataFrames to handle time-series data specifically.

Data structures in Pandas for Date and Time

We will first look at the basic data structures available for dealing with date and time in Pandas.

1. Timestamp - These refer to particular instance of time. Eg: '2018-05-31 12:30:45'. This is similar to Python's datetime. It is based on NumPy's data type datetime64.
2. Period - These refer to standard time inte...

Introduction

Once we start working with actual datasets, it might be very rare to find all the data in one place in the same file organised neatly. From time to time we will need to extract data from different sources, combine them in suitable form and only then can we proceed to do some work with the data.

In this tutorial we will learn about two operations — merge and concatenate — that can help us with combining DataFrames in different ways.

Merge

There are various ways in which DataFrames can be merged, depending on our needs. Let us take an example and explore before we learn the different ways.

Suppose we have two Datasets: Left-Dataset and Right...

Introduction

You have already learnt about basics of the Pandas DataFrame, and how to select various subsets of the DataFrame using indexing and slicing.

In this tutorial we will learn how to select a subset of DataFrame which matches some criteria. For example, we might want to select the rows of a DataFrame based on the values in each row (say age >= 30). Or, we might even want to select a subset of the DataFrame based on the datatypes.

Let's get started!

Set-up

As in the previous tutorial, let's start by importing the libraries and loading the dataset.

type=codeblock|id=village_relationship|autocreate=datascience|show_output=1

Link to Problem: Problem GCJ101BB

Clearer problem statement:

There are N chickens.

Each chicken has a starting position X[i], and a speed V[i]. They start at their starting location, and are running at the speed of V[i] towards the right.

You want K chickens to reach the barn before time T. The barn is located at location B.

The problem is that if a chicken has a slower chicken in front of it, it is not able to overtake it unless you do an "allow overtake" (in the problem, this is called "swapping").

You want to minimize the total number of "allow overtake" / "swaps" so that at-least K chickens reach the barn before time T.

Introduction

Pandas is an open-source Python library which provides a 2-dimensional data structure called a DataFrame.

A DataFrame is the Python equivalent of a spreadsheet or SQL table, and is the most commonly used Pandas object. It provides a lot of useful functionality to work with tabular data, which is something we do regularly in data science and machine learning.

In this tutorial, we will be learning about the pandas DataFrame and explore some of its useful methods.

Importing

We will need the Pandas and NumPy libraries for this tutorial, so let us load them at the beginning.

type=codeblock|id=pd_rc_import|show_output=0|autocreate=datascience