Tidy-TS

Filtering Rows

Filtering lets you subset your data based on specific conditions. It supports both synchronous and asynchronous filtering patterns.

Basic Filtering

Filter rows based on simple conditions

The filter() function creates a new DataFrame containing only rows that match your condition. You can use any combination of column values and logical operators.

import { createDataFrame } from "@tidy-ts/dataframe";

const people = createDataFrame([
  { id: 1, name: "Luke", species: "Human", mass: 77, height: 172 },
  { id: 2, name: "C-3PO", species: "Droid", mass: 75, height: 167 },
  { id: 3, name: "R2-D2", species: "Droid", mass: 32, height: 96 },
  { id: 4, name: "Darth Vader", species: "Human", mass: 136, height: 202 },
  { id: 5, name: "Chewbacca", species: "Wookiee", mass: 112, height: 228 },
]);

// Filter by numeric conditions
const tallPeople = people.filter((row) => row.height > 180);
tallPeople.print("People taller than 180cm:");

// Filter by string conditions
const humans = people.filter((row) => row.species === "Human");
humans.print("Only humans:");

// Filter by multiple conditions
const tallHumans = people.filter(
  (row) => row.height > 180 && row.species === "Human"
);
tallHumans.print("Tall humans (height > 180cm AND species = Human):");

import { createDataFrame } from "@tidy-ts/dataframe";

const people = createDataFrame([
  { id: 1, name: "Luke", species: "Human", mass: 77, height: 172 },
  { id: 2, name: "C-3PO", species: "Droid", mass: 75, height: 167 },
  { id: 3, name: "R2-D2", species: "Droid", mass: 32, height: 96 },
  { id: 4, name: "Darth Vader", species: "Human", mass: 136, height: 202 },
  { id: 5, name: "Chewbacca", species: "Wookiee", mass: 112, height: 228 },
]);

// Filter by numeric conditions
const tallPeople = people.filter((row) => row.height > 180);
tallPeople.print("People taller than 180cm:");

// Filter by string conditions
const humans = people.filter((row) => row.species === "Human");
humans.print("Only humans:");

// Filter by multiple conditions
const tallHumans = people.filter(
  (row) => row.height > 180 && row.species === "Human"
);
tallHumans.print("Tall humans (height > 180cm AND species = Human):");

Using (row, index, df) Parameters in Filter

Just like mutate(), filter() also provides three parameters available in filter functions

row: Current row's data | index: Row position (0-based) | df: Entire DataFrame

// Filter functions also provide three parameters:
const withParameters = people
  .filter((row, index, df) => {
    // row: Access current row's values
    const isHeavy = row.mass > 100;
    
    // index: Get the current row's position (0-based)
    const isFirstHalf = index < df.nrows() / 2;
    
    // df: Access the entire DataFrame for relative comparisons
    const isAboveAverage = row.mass > 50; 
    
    // Combine all three for complex filtering
    return isHeavy && isFirstHalf && isAboveAverage;
  });

withParameters.print("Filtered using all three parameters:");

// Filter functions also provide three parameters:
const withParameters = people
  .filter((row, index, df) => {
    // row: Access current row's values
    const isHeavy = row.mass > 100;
    
    // index: Get the current row's position (0-based)
    const isFirstHalf = index < df.nrows() / 2;
    
    // df: Access the entire DataFrame for relative comparisons
    const isAboveAverage = row.mass > 50; 
    
    // Combine all three for complex filtering
    return isHeavy && isFirstHalf && isAboveAverage;
  });

withParameters.print("Filtered using all three parameters:");

Complex Filtering Conditions

Use complex logic for filtering scenarios

You can combine multiple conditions using logical operators (&&, ||, !) and even filter based on calculated values from previous transformations.

// Filter with calculated values
const withCalculations = people
  .mutate({
    is_heavy: (row) => row.mass > 100,
  })
  .filter((row) => row.is_heavy);

withCalculations.print("Heavy characters (mass > 100):");

// Filter with calculated values
const withCalculations = people
  .mutate({
    is_heavy: (row) => row.mass > 100,
  })
  .filter((row) => row.is_heavy);

withCalculations.print("Heavy characters (mass > 100):");

Async Filtering

Handle asynchronous operations in filter conditions

Filter functions support async operations for validating data against external APIs, databases, or performing complex async calculations. All async operations are automatically handled with proper concurrency control.

📖 Learn more: See the Async Operations page for async filtering examples and patterns.

Filtering Patterns

Common filtering patterns and best practices

Range Filtering

// Chain multiple filters
const chainedFilter = people
  .filter((row) => row.species === "Human")
  .filter((row) => row.height > 170);

chainedFilter.print("Tall humans (chained filters):");

// Chain multiple filters
const chainedFilter = people
  .filter((row) => row.species === "Human")
  .filter((row) => row.height > 170);

chainedFilter.print("Tall humans (chained filters):");

Null and Missing Value Filtering

import { createDataFrame } from "@tidy-ts/dataframe";

const people = createDataFrame([
  { id: 1, name: "Luke", species: "Human", mass: 77, height: 172 },
  { id: 2, name: "C-3PO", species: "Droid", mass: 75, height: 167 },
  { id: 3, name: "R2-D2", species: "Droid", mass: 32, height: 96 },
  { id: 4, name: "Darth Vader", species: "Human", mass: 136, height: 202 },
  { id: 5, name: "Chewbacca", species: "Wookiee", mass: 112, height: 228 },
]);

// Filter by numeric conditions
const tallPeople = people.filter((row) => row.height > 180);
tallPeople.print("People taller than 180cm:");

// Filter by string conditions
const humans = people.filter((row) => row.species === "Human");
humans.print("Only humans:");

// Filter by multiple conditions
const tallHumans = people.filter(
  (row) => row.height > 180 && row.species === "Human"
);
tallHumans.print("Tall humans (height > 180cm AND species = Human):");

import { createDataFrame } from "@tidy-ts/dataframe";

const people = createDataFrame([
  { id: 1, name: "Luke", species: "Human", mass: 77, height: 172 },
  { id: 2, name: "C-3PO", species: "Droid", mass: 75, height: 167 },
  { id: 3, name: "R2-D2", species: "Droid", mass: 32, height: 96 },
  { id: 4, name: "Darth Vader", species: "Human", mass: 136, height: 202 },
  { id: 5, name: "Chewbacca", species: "Wookiee", mass: 112, height: 228 },
]);

// Filter by numeric conditions
const tallPeople = people.filter((row) => row.height > 180);
tallPeople.print("People taller than 180cm:");

// Filter by string conditions
const humans = people.filter((row) => row.species === "Human");
humans.print("Only humans:");

// Filter by multiple conditions
const tallHumans = people.filter(
  (row) => row.height > 180 && row.species === "Human"
);
tallHumans.print("Tall humans (height > 180cm AND species = Human):");