Julia DataFrames Guide — Tabular Data Manipulation and Analysis

In this tutorial, you will learn about Julia DataFrames Guide. We cover key concepts, practical examples, and best practices to help you master this topic.

Julia DataFrames.jl provides a DataFrame type for tabular data with column-oriented access, select/filter/transform operations, groupby/combine for split-apply-combine, and integration with CSV.jl for reading and writing structured data.

Creating DataFrames

using DataFrames

# From columns
df = DataFrame(
    name = ["Alice", "Bob", "Carol"],
    age  = [25, 30, 35],
    city = ["NYC", "LA", "Chicago"]
)

# From array
df = DataFrame(rand(5, 3), :auto)

Basic Operations

# View
first(df, 3)
last(df, 2)
describe(df)

# Access columns
df.name        # vector
df[!, :name]   # column (no copy)
df[:, :name]   # column (copy)

# Access rows
df[1, :]       # first row
df[1:3, :]     # first 3 rows
df[[1, 3], :]  # specific rows

Selection and Filtering

# Select columns
select(df, :name, :age)
select(df, Not(:city))
select(df, r"^a")  # regex column match

# Filter rows
filter(row -> row.age > 25, df)
filter(:age => >(25), df)

# Transform columns
transform(df, :age => (x -> x .* 2) => :age_double)
transform(df, :name => ByRow(uppercase) => :name_upper)

Grouping and Aggregation

# Group by city
gdf = groupby(df, :city)

# Combine with aggregation
combine(gdf, :age => mean => :avg_age)
combine(gdf, :age => (x -> [mean(x), std(x)]) => [:avg, :std])

# Multiple aggregations
combine(gdf, 
    :age => mean => :avg_age,
    :age => length => :count
)

Joins

df1 = DataFrame(id=[1,2,3], name=["A","B","C"])
df2 = DataFrame(id=[2,3,4], score=[95,87,92])

# Inner join
innerjoin(df1, df2, on=:id)

# Left join
leftjoin(df1, df2, on=:id)

# Outer join
outerjoin(df1, df2, on=:id)

Common Mistakes

1. Modifying without assignment

DataFrame operations often return a new DataFrame. Use select!, filter!, transform! for in-place mutation.

2. Index confusion

df[:, :col] copies. df[!, :col] is a view. Modifying the view modifies the original.

3. Type instability in filters

filter with a lambda can be slower than using ByRow. Prefer filter(:col => >(5), df) over filter(r -> r.col > 5, df).

Practice Questions

1. How do you select specific columns? select(df, :col1, :col2) or df[:, [:col1, :col2]].

2. How do you group and aggregate? combine(groupby(df, :group_col), :value_col => mean => :avg_value).

3. How do you filter rows? filter(:age => >(25), df) or filter(row -> row.age > 25, df).

FAQ

{{< faq question="Is DataFrames.jl faster than pandas?" >}} For many operations, yes. Julia's DataFrames benefits from multiple dispatch, type stability, and compilation. {{< /faq >}}

{{< faq question="How do I handle missing values?" >}} Use missing in Julia. skipmissing(df.col) skips missing. dropmissing(df) removes rows with missing values. {{< /faq >}}

{{< faq question="Can I use DataFrames with plotting?" >}} Yes. plot(df.x, df.y) works directly. Plots.jl accepts arrays from DataFrame columns. {{< /faq >}}

What's Next

Now learn about concurrency in Julia.