Pipeline Designer

Data-focused ETL designer — ingest from JSON, CSV or SQL, apply transforms, and push to any destination.

Overview

Pipeline Designer is a specialised ETL (Extract, Transform, Load) tool within the Automation suite. While the Workflow Editor is optimised for AI and event-driven logic, the Pipeline Designer focuses on structured data movement — pulling records from sources, cleaning and reshaping them, and pushing the results to destinations.

Coming Soon

Pipeline Designer is currently in development. The UI is visible in the Automation mode sidebar but pipeline authoring and execution are not yet active.

Planned Features

Source connectors — JSON files, CSV files, REST APIs, SQL databases (PostgreSQL, MySQL, SQLite), and inline data entry.
Transform steps — filter rows by condition, rename columns, parse dates, compute derived columns, join two sources, group and aggregate.
Destination connectors — write to JSON file, CSV file, SQL table (insert or upsert), send as an email attachment, or post to a webhook.
Schema preview — see the column names, types, and a sample of values at each step before running the full pipeline.
Row-count assertions — define minimum and maximum expected row counts; the pipeline fails cleanly if a source returns an unexpected amount of data.
Scheduled runs — integrate with Task Scheduler to run an ETL pipeline on a cron schedule.

When to Use Pipeline Designer vs. Workflow Editor

Use Pipeline Designer when your goal is moving rows of structured data from A to B. Use the Workflow Editor when you need AI generation, multi-step AI chains, or event-driven logic. The two tools are complementary — a Workflow Editor pipeline can call a Pipeline Designer pipeline as a step.

Frequently Asked Questions

Is Pipeline Designer available to use right now?

Not yet. Pipeline Designer is listed in the Automation mode sidebar so you can see where it will live, but pipeline authoring and execution are not active. There is no way to build or run an ETL pipeline in the current release — check back as this page will be updated once the feature ships.

What is the difference between Pipeline Designer and the Workflow Editor?

The Workflow Editor is built for AI-driven and event-driven logic — chaining AI generation steps, reacting to triggers, and orchestrating multi-step automations. Pipeline Designer is purpose-built for structured data movement: extracting rows from a source, transforming them, and loading them into a destination (a classic ETL pattern). Once released, the two will be complementary — a Workflow Editor automation will be able to call a Pipeline Designer pipeline as one of its steps.

What data sources and destinations will Pipeline Designer support?

Planned source connectors include JSON files, CSV files, REST APIs, SQL databases (PostgreSQL, MySQL, SQLite), and inline manual data entry. Planned destinations include writing to a JSON or CSV file, inserting or upserting into a SQL table, sending results as an email attachment, or posting to a webhook. These are the currently planned connectors and are subject to change before release.

Will Pipeline Designer support scheduled or recurring runs?

Yes, that is part of the plan. Pipeline Designer is expected to integrate with the Automation suite's Task Scheduler so an ETL pipeline can be triggered on a cron schedule, in addition to being run manually or called as a step from the Workflow Editor.