How to do R-like data manipulation using Pandas?


R and Python play a crucial role in data processing and manipulation. Many beginners find it difficult to switch from Python to R or vice versa under such requirements. But you have to understand how common the two approaches are. Many data manipulation tasks done in R can also be done using Pandas in python. In this article, we are going to discuss a comparison between data manipulation using R and Pandas based on some of the important functions and features. This will help beginners understand the differences and also help them switch between the two. The main points to be discussed in the article are listed below.


  1. About pandas and R
  2. Compare data operations
  3. R Vs Pandas for data manipulation

About pandas and R

Let’s do a brief introduction to both R and Pandas.

The R programming language

We can think of R as an implementation of the S language which is a language and environment specially designed for statistical and graphical analysis of data. Using the R language, we can use a variety of statistical analysis techniques such as linear or non-linear modeling, testing, clustering, classification, etc. This language also provides various features using which we can perform graphical analysis as well. Using the R language we can produce highly interactive graphs of any data.

In this article, we are going to discuss tools or R language package that can be used for data manipulation.

About pandas

Pandas is a library in python for many data-related tasks such as data manipulation and data conversion. We use data with Pandas in tabular form. Along with these tasks, we can also use Pandas for data warehousing using Pandasql. The function under Pandas can be used to inspect data as we move data in or out of the process.

Looking at the above points we can say that Pandas is a toolkit or library in python and talking about R it is a language in itself and has many toolkits to perform data related tasks . In this article, we will compare R language and Pandas library based on data related tasks.

Let’s start the comparison.

Compare data operations

As data science practitioners, we regularly use Python and R to perform data-related tasks. Using this section of the article, we will learn how we can perform the various toolsets of using R language and Pandas library in python language.

In R, we mainly use dplyr toolbox for querying, filtering and sampling operations. The table below shows different methods we use for the simple operations given above using dplyr and the Pandas toolkit.

R Pandas
sun(data) data.shape
head (data) data.head()
slice(data, 1:10) data.iloc[:9]
filter(data, col1 == 1, col2 == 1) data. query(‘col1 == 1 & col2 == 1’)
Data[data$col1 == 1 & data$col2 == 1,] Data[(data.col1 == 1) & (data.col2 == 1)]
select(data,col1,col2) Data[[‘col1’, ‘col2’]]
select(data, col1:col3) data.loc[:, ‘col1′:’col3’]
distinct(select(data, col1)) Data[[‘col1’]].drop_duplicates()
select(data, -(col1:col3)) data.drop(cols_to_drop, axis=1)
distinct(select(data, col1, col2)) Data[[‘col1’, ‘col2’]].drop_duplicates()
sample_n(data, 10) data.sample(n=10)
sample_frac(data, 0.01) data.sample(frac=0.01)

Let’s see the difference between R(dplyr) and Pandas based on the sort operation.

R Pandas
arrange(data, col1, col2) data. sort_values([‘col1’, ‘col2’])
arrange(data, desc(col1)) data.sort_values(‘col1’, ascending=False)

Let’s see the difference between R(dplyr) and Pandas based on the transform operation.

R Pandas
select(data, col_one = col1) data.rename(columns={‘col1’: ‘col_one’})[‘col_one’]
mutate(data, c=ab) data.assign(c=data[‘a’]-Data[‘b’])
rename(data, col_one = col1) data.rename(columns={‘col1’: ‘col_one’})

Let’s see the difference between R(dplyr) and Pandas based on the grouping and summarizing operation.

R Pandas
summary (data) data.describe()
gdata gdata = data.groupby(‘col1’)
summary(gdata, avg=mean(col1, na.rm=TRUE)) data.groupby(‘col1’).agg({‘col1’: ‘average’})
summary(gdata, total=sum(col1)) data.groupby(‘col1’).sum()


We can perform slicing operations like selecting columns using the c() function in R. In python we can do this using Pandas. For example, the codes below can be used in R to select and access columns using column name or by integer location.

Use of column name

data <- data.frame(a=rnorm(5), b=rnorm(5), c=rnorm(5), d=rnorm(5), e=rnorm(5))
data[, c("a", "c", "e")]

Using the entire location

data <- data.frame(matrix(rnorm(1000), ncol=100))
data[, c(1:10, 25:30, 40, 50:100)]

In Pandas we can do the same operation using the following lines of codes.

import pandas as pd
import numpy as np
data = pd.DataFrame(np.random.randn(5, 3), columns=columns)


Use of column name

data[["a", "c"]]


Location usage

data.loc[:, ["a", "c"]]



Using the R language, we group by gata to create subsets and calculate the average of each subset using the by1 and by2 functions as follows:

data <- data.frame(
  by1 = c("abc", "bdc", 1, 2, "abc", "bcd", 1, 2, "rfg", 1, "abc", 12),
  by2 = c("bac","cbd",99,95,"bac","xyz",95,99,"abc",99,"abc","abc")
  v1 = c(1,3,5,7,8,3,5,NA,4,5,7,9),
  v2 = c(11,33,55,77,88,33,55,NA,44,55,77,99))
aggregate(x=data[, c("v1", "v2")], by=list(mydata2$by1, mydata2$by2), FUN = mean)

Using Pandas, we can perform such an operation in the following way:

data = pd.DataFrame(
        "by1": ["abc", "bdc", 1, 2, "abc", "bcd", 1, 2, "rfg", 1, 'abc', 12],
        "by2": ["bac","cbd",99,95,"bac","xyz",95,99,"abc",99,"abc",'abc',],
        "v1": [1, 3, 5, 7, 8, 3, 5, np.nan, 4, 5, 7, 9],
        "v2": [11, 33, 55, 77, 88, 33, 55, np.nan, 44, 55, 77, 99],


g = data.groupby(["by1", "by2"])
g[["v1", "v2"]].mean()


Match function

In R language, we can select the data using the %ln% function which can be defined using the match module in the following way:

<- 0:9
s %in% c(4,6)

Using Pandas, we can do this using the isin() function in the following way:

s = pd.Series(np.arange(10), dtype=np.float32)
s.isin([4, 6])


Query function

In R language, we need to use the subset function to perform conditional queries with the dataset. The code below is an example of this function.

data <- data.frame(a=rnorm(15), b=rnorm(15))
subset(data, a >= b)
data[data$a >= data$b,]

Where we extract rows where the value of column a is less than and equal to column b.

Using Pandas, we can do this using the query function.

data = pd.DataFrame({"a": np.random.randn(15), "b": np.random.randn(15)})
data.query("a >= b")


R Vs Pandas for data manipulation

Using the above points, we have discussed how we can perform various data analysis using Pandas in python and R’s toolkits. We have found that in R the packages are split into the language and that we have to install them separately on our local machine. When we use Pandas for similar purposes, we can have all the functions in a managed sense or we can say that these functions are in one place, we don’t need to search for the other tools. One thing that R language is good for data analysis is R’s speed and its interface which is much more user-friendly than Pandas. Regarding the R language, we can say that it is less complex than the Python language. R and Pandas are the best at home.

Last words

Here in this article, we have discussed the comparison between R and Pandas. In conclusion, it can be said that R is a programming language while Pandas is a library. By using the packages of R we can perform different operations where Pandas helps us to perform different operations. This tutorial will help beginners understand the difference between the two and also make the migration easier.

The references:


About Author

Comments are closed.