Replies: 34 comments 46 replies
-
|
Here are some example ideas:
|
Beta Was this translation helpful? Give feedback.
-
|
Floating Agent:
|
Beta Was this translation helpful? Give feedback.
-
|
Beta Was this translation helpful? Give feedback.
-
|
Link to PR: #5093 |
Beta Was this translation helpful? Give feedback.
-
|
Workflow Performance Profiler:
PR #5098 |
Beta Was this translation helpful? Give feedback.
-
|
I like this one. You can even take a step further to consider wrap this profiler as a tool that can be used by the |
Beta Was this translation helpful? Give feedback.
-
|
Mobile App for Texera. |
Beta Was this translation helpful? Give feedback.
-
|
PR #5083 |
Beta Was this translation helpful? Give feedback.
-
|
UDF Copilot: a schema-aware AI inside the Python UDF editor Themes: Human-Agent Collaboration · Productivity Enhancement The gap. Today, The idea. Bring a Cursor-style assistant into the Monaco UDF editor, with one thing GitHub Copilot can't have: Texera context — the upstream operator's output schema, a real sample row, the downstream consumer, and the UDF's API contract ( Four interaction modes sharing that context:
Why it matters. A large share of Texera's audience are domain experts (biology, social science, medicine) who write UDFs but aren't full-time programmers. A schema-aware copilot collapses the learning curve from "read pytexera docs first" to "just describe what you want, with real column names autocompleting." Implementation sketch. Frontend: extend Demo. Open a Python UDF after a CSVScan → ghost text suggests real columns → Cmd+K rewrites a loop as vectorized pandas → force a schema typo → one-click Fix → side chat generates pytest. ~3 minutes end-to-end. |
Beta Was this translation helpful? Give feedback.
-
|
Does this context selection idea have to be limited in the Python code editor? Can it be generalized to apply for the whole workflow workspace ? |
Beta Was this translation helpful? Give feedback.
-
|
Great point — the schema-aware context can definitely extend to other operators. For this hackathon I'm focusing on Python UDFs since the 4 interaction modes need polish together, but the context plumbing is operator-agnostic so it should generalize. |
Beta Was this translation helpful? Give feedback.
-
|
PR: #5081 |
Beta Was this translation helpful? Give feedback.
-
AI-Augmented Macro Operators for TexeraProblemTexera workflows grow into 20–50+ operator DAGs with no encapsulation. Users copy-paste the same subgraphs across projects, and pipelines run slower than they need to because of inter-operator serialization. What we'll build
Why it fits the Agent Hackathon
Demo (~3 min)
Stretch
|
Beta Was this translation helpful? Give feedback.
-
|
Taken by me. Starting to work on this idea. |
Beta Was this translation helpful? Give feedback.
-
|
PR: #5115 |
Beta Was this translation helpful? Give feedback.
-
AI Workflows in Texera: LLM, RAG & Agentic DAGPitch. Today Texera can run ML inference but has no first-class LLM, RAG, or agentic-DAG operators — the only AI surface is the workflow-editing chat in What we'll ship
Why it's a good hackathon fit
Demo storyboard
Impact. Turns Texera into a viable visual environment for RAG pipelines and agent prototypes — opening data-science / education use cases (UCI courses, research demos) that currently need LangChain glue code. |
Beta Was this translation helpful? Give feedback.
-
Texera Workflow Completion ForecastingWhat exists today. Each operator box already displays cumulative processed input and produced output tuple counts (rendered by The actual gap. What they cannot see:
Proposal. Turn the existing tuple counters into a forecasting layer. Build on the per-operator Scope (four small, independently shippable phases):
Honesty principle. Operators rooted in unbounded sources keep showing counts + rate only — no fabricated forecast. Where the system genuinely cannot predict, it stays silent rather than misleading. Deliverables.
Impact. The existing counters answer what happened. This adds the answers users actually ask out loud: how fast is it going, how much is left, when will it finish — while being explicit when prediction is genuinely impossible. |
Beta Was this translation helpful? Give feedback.
-
|
Idea 1: AI-Guided Tutorial for New Users
Idea 2: AI Workflow Doc Generator
|
Beta Was this translation helpful? Give feedback.
-
|
idea 2: I am very interested in this document. if we can make it embed with the workflow and result, it self can be a data science report!! |
Beta Was this translation helpful? Give feedback.
-
|
we had video tutorials before but it was not very easy to use. it will be great if agent can adapt easily on the current system and introduce! |
Beta Was this translation helpful? Give feedback.
-
|
PR #5104 |
Beta Was this translation helpful? Give feedback.
-
Workflow Auto-Documententation AgentA new tool that automatically generates documentation for non-trivial workflows using AI. ProblemAs workflows grow in size and complexity, their intent becomes increasingly opaque to collaborators and even to their original authors. A pipeline of twenty or more operators offers no concise, human-readable explanation of what it does, what data it consumes, or what it produces. Comment boxes provide a partial remedy but rely on author discipline and are rarely kept current. The platform currently offers no mechanism for summarizing a workflow's purpose and structure at a glance. Proposed SolutionIntroduce an on-demand documentation action available from the workflow editor. When invoked, the system inspects the current workflow's logical structure — its operators, their configurations, and the connections between them — and produces a concise, structured narrative describing the workflow. The generated documentation includes:
The result is rendered alongside the canvas as a readable artifact that can be reviewed, copied, or shared. Design Goals
|
Beta Was this translation helpful? Give feedback.
-
|
glad to see your demos yesterday! I think it is already awesome! keep the good work. I want to reiterate my idea of two-directional binding. now we use data workflows to represent a task, your documentation/summary could be a text version describing the task, it can be coming form the same source data, and modify any (workflow or text) would also update the other one. don't be limited by my thoughts! |
Beta Was this translation helpful? Give feedback.
-
|
PR #5088 |
Beta Was this translation helpful? Give feedback.
-
Streaming Texera: Real-Time Workflows on AmberProblemTexera's Amber engine is micro-batch and bounded-input today: every workflow assumes its sources finish. Live data — sockets, WebSockets, sensor feeds, chat streams, agent event logs — cannot be expressed as a Texera workflow without polling hacks. IdeaAdd a first-class streaming mode to Texera workflows. Sources can produce data forever; downstream operators react to it in real time; the workflow runs indefinitely until the user stops it. What users get
Why it matters
DemoA WebSocket feed of agent tool-call events → tumbling 1-minute window → live "tool usage by agent" chart in the Texera UI, updating as events arrive. |
Beta Was this translation helpful? Give feedback.
-
|
related #5059 (comment) |
Beta Was this translation helpful? Give feedback.
-
Idea 1 — Persistent Memory (a Texera agent that grows with you)Today the agent forgets you between sessions. What if it didn't? A persistent profile that observes your work across every workflow you build — the operators you reach for, your fix patterns, your domain expertise, how you've grown. The agent uses it to pre-seed new workflows in your style ("you usually filter nulls before joining, want me to scaffold that?") and references your past work when you hit familiar errors. You see everything that's remembered in a "Your Data Science Self" panel and can edit or delete any of it. Not just chat memory — closer to personal informatics for data scientists. Idea 2 — Workflow Theater (watch your data flow)Workflow execution today is progress bars and status icons. What if instead, you watched actual data tuples flow through your DAG as animated particles? Bottlenecks visible as traffic jams. Filters showing tuples dropped to the floor. Joins showing two streams merging. Click any tuple mid-flight to inspect it. Scrub a timeline to rewind execution. Builds on the time-travel debugging work in IcedTea — adds the visual layer that makes it legible to humans. |
Beta Was this translation helpful? Give feedback.
-
|
I love both ideas! for idea 1, maybe we can combine memory and style, you can give more personality to the agent as well. |
Beta Was this translation helpful? Give feedback.
-
|
Idea 2 shipped — submitted as Workflow Theater for the Best Data Experience track. PR: galenjauss#1 The visualization is built on the WebSocket events Texera already emits ( 2-minute demo video link to follow. |
Beta Was this translation helpful? Give feedback.
-
|
@galenjauss did you submit your video? we could not find it. |
Beta Was this translation helpful? Give feedback.
-
|
Compiled C++ UDF Operator Category: Productivity Enhancement / Innovation I want to build a Compiled C++ UDF operator for Texera. The idea is to let users write small performance-critical UDFs in C++ directly inside a workflow, compile them from the UI, and run them on incoming data. This would be useful when Python UDFs are easy to write but too slow for CPU-heavy tasks like numeric transformations, feature extraction, custom parsing, or scientific preprocessing. MVP:
Example: double process(double age, double income) { |
Beta Was this translation helpful? Give feedback.
-
|
Implemented a prototype Compiled C++ UDF Operator for Texera. The operator lets users write C++ directly inside the workflow UI, configure input/output columns, compile the code, and run it as a workflow operator. The MVP supports typed tuple/batch/table-style APIs, compiler errors, retained input columns, configurable compiler flags, timeout, and batching. To demonstrate why this is useful, I benchmarked the same deterministic CPU-heavy matrix multiplication workload across C++, Java UDF, and Python UDF. Each runtime receives its own independent source with the same seed/workload. Benchmark Results
This shows the motivation clearly: Python is great for usability, but compiled C++ can provide major speedups for CPU-heavy UDFs while staying inside Texera’s visual workflow model. C++ UDF#include <chrono>
class MatrixMultiplyOperator : public texera::UDFOperator {
public:
texera::TupleOutput process_tuple(const texera::Tuple& tuple, int port) override {
int trial = tuple.get("trial").as_int();
int n = tuple.get("matrix_size").as_int();
long long seed = tuple.get("seed").as_long();
auto start = std::chrono::high_resolution_clock::now();
double checksum = 0.0;
for (int i = 0; i < n; i++) {
for (int j = 0; j < n; j++) {
double cell = 0.0;
for (int k = 0; k < n; k++) {
double a = ((seed + trial * 97LL + i * 31LL + k * 17LL) % 1000LL) / 1000.0;
double b = ((seed + trial * 53LL + k * 13LL + j * 29LL) % 1000LL) / 1000.0;
cell += a * b;
}
checksum += cell;
}
}
auto end = std::chrono::high_resolution_clock::now();
double elapsed_ms = std::chrono::duration<double, std::milli>(end - start).count();
return { texera::TupleLike{
texera::Value::string_value("cpp"),
texera::Value::double_value(checksum),
texera::Value::double_value(elapsed_ms)
}};
}
};
using TexeraUDFOperator = MatrixMultiplyOperator;Java UDFpublic TupleLike processTuple(Tuple tuple) {
int trial = ((Number) tuple.getField("trial")).intValue();
int n = ((Number) tuple.getField("matrix_size")).intValue();
long seed = ((Number) tuple.getField("seed")).longValue();
long start = System.nanoTime();
double checksum = 0.0;
for (int i = 0; i < n; i++) {
for (int j = 0; j < n; j++) {
double cell = 0.0;
for (int k = 0; k < n; k++) {
double a = ((seed + trial * 97L + i * 31L + k * 17L) % 1000L) / 1000.0;
double b = ((seed + trial * 53L + k * 13L + j * 29L) % 1000L) / 1000.0;
cell += a * b;
}
checksum += cell;
}
}
double elapsedMs = (System.nanoTime() - start) / 1_000_000.0;
Object[] inputFields = tuple.getFields();
Object[] outputFields = Arrays.copyOf(inputFields, inputFields.length + 3);
outputFields[inputFields.length] = "java";
outputFields[inputFields.length + 1] = checksum;
outputFields[inputFields.length + 2] = elapsedMs;
return TupleLike$.MODULE$.apply(Arrays.asList(outputFields));
}Python UDFfrom pytexera import *
import time
class ProcessTupleOperator(UDFOperatorV2):
@overrides
def process_tuple(self, tuple_: Tuple, port: int) -> Iterator[Optional[TupleLike]]:
trial = int(tuple_["trial"])
n = int(tuple_["matrix_size"])
seed = int(tuple_["seed"])
start = time.perf_counter()
checksum = 0.0
for i in range(n):
for j in range(n):
cell = 0.0
for k in range(n):
a = ((seed + trial * 97 + i * 31 + k * 17) % 1000) / 1000.0
b = ((seed + trial * 53 + k * 13 + j * 29) % 1000) / 1000.0
cell += a * b
checksum += cell
elapsed_ms = (time.perf_counter() - start) * 1000.0
output = tuple_.as_dict()
output["runtime"] = "python"
output["checksum"] = checksum
output["elapsed_ms"] = elapsed_ms
yield outputTexera.-.Google.Chrome.2026-05-14.23-49-35.github-under-10mb-final-no-audio.mp4 |
Beta Was this translation helpful? Give feedback.
-
|
@carloea2 nice results! however let's keep this post brief for idea discussions, if you went ahead implemented it out, feel free to include detailed plans/results on a separate PR! back to the topic, one quick question: can we try rust as well? it is also very trendy nowadays |
Beta Was this translation helpful? Give feedback.
-
|
Yes we could try rust as well, thanks |
Beta Was this translation helpful? Give feedback.
-
Beta Was this translation helpful? Give feedback.
-
Live Canvas Analytics for TexeraMotivation
The headline: semantic zoom
Supporting features
Why it fits the Agent Hackathon
|
Beta Was this translation helpful? Give feedback.
-
|
Side-by-Side Workflow Version Comparison Adding support for side-by-side workflow version execution and comparison within Texera. The idea is to allow users to run two versions of the same workflow in parallel and automatically detect where their behavior or outputs begin to diverge. By comparing intermediate results at each operator, the system could identify the exact stage where differences are introduced. This feature would help users debug workflow changes, validate new optimizations, and safely iterate on workflows by providing clear visibility into the impact of modifications between versions. |
Beta Was this translation helpful? Give feedback.
-
Beta Was this translation helpful? Give feedback.
-
|
Idea: Smart File Source for Texera Texera currently exposes file input through separate operators such as CSV, JSONL, Arrow, and generic file scan. For new users, this creates friction: before building a workflow, they must already know which scan operator to choose and how to configure it. Real datasets are also messy: CSV vs TSV, missing headers, wrong delimiters, JSONL, Excel sheets, Parquet exports, and files with unclear schemas. I want to build a Smart File Source operator that lets the user select a file once and lets Texera infer how to read it. |
Beta Was this translation helpful? Give feedback.
-
|
it would be great to have it! also consider support folders? (we have a lot of use cases of reading a folder of images, etc). I actually had another direction in mind before: have a LLM operator that reads a sample (or a part) of a file, and create an operator on the fly to read/parse the entire file into table format. then run that operator as a source. so it is not a pre-designed generic operator that supports all kinds of files, but a dynamically generated operator designed specifically for that single file (or a folder of similar files). One use case: I have a business report in pdf which embeds some tables, or other information inside of it, read it out would be very useful. But all pdfs are having different structures which we may not be able to know before seeing the file. same source of the pdfs may share similar sturcutre (e.g., business report generated by the same company across different months). |
Beta Was this translation helpful? Give feedback.
-
|
Divided this idea into 3 features with smart operator, visual workflow and llm parser |
Beta Was this translation helpful? Give feedback.
-
Idea 1A: Plan-then-Build: draft before buildingFrustration: The agent jumps straight to creating operators. When it guesses wrong, you watch 6+ wrong operators appear, pay tokens for each, and regenerate the workflow or clean up manually. Fix: Insert a text-only plan phase before any mutation.
The agent emits a short plan (operators, params, assumptions, open questions) and waits. User refines in plain English ("skip step 3"). Nothing touches the canvas until Approve. Why this matters?
Idea 1B: Sample-then-Scale: try it on 100 rows firstFrustration: "I waited 8 minutes for it to crash at op12. Now I wait 8 more." Most errors (schema, join keys, filters) show up instantly on a sample. Fix: Default the first run to 100 rows per source. User reviews, tweaks, then clicks Run on full data.
Why it matters?
Idea 1 catches intent errors before any work happens. Idea 2 catches data errors before any waiting happens. |
Beta Was this translation helpful? Give feedback.
-
|
I want to encourage you to utilize our workflow canvas: it is more intuitive than "text" interface! for example for planning, instead of showing in "Request → Clarify questions → Markdown plan → Approve → Build" text, just render temporary operators (without detailed properties, just a skeleton operator) on the canvas, would be much more visual and intuitive! for sample, that's a great idea! many other systems have this, always want to see it for texera! |
Beta Was this translation helpful? Give feedback.
-
Idea: Operator Stash — Git Stash for PropertiesFrustration: Every property edit is destructive. The agent overwrites your config and you can't get it back. You want to just test two settings but there's no place to hold one aside. Undo only goes back in a straight line. Fix: Auto-save the previous version of an operator's properties on every change. A tiny stash icon on the node lets you toggle or keep/drop. Scoped per-operator, not global. Why it matters?
|
Beta Was this translation helpful? Give feedback.
-
|
I think we have versioning already! checkout version history of workflow, it also covers individual operator version as well. But love the idea for accept edits, just like what we have in IDEs for accept code change! can have auto mode/ auto accept mode for ppl who don't want to accept manually as well |
Beta Was this translation helpful? Give feedback.
-
|
Very simple but added themes beyond light and dark mode. You can edit font, font size, theme color etc. Applies to all areas of the screen. |
Beta Was this translation helpful? Give feedback.
-
|
YES! I would love to see more themes that are concise across entire site! |
Beta Was this translation helpful? Give feedback.
-
|
@Yicong-Huang see #5084 |
Beta Was this translation helpful? Give feedback.
-
|
A sports-betting pipeline for Valorant kill prop bets on Underdog Fantasy, where Texera does the data work and a machine learning model makes the actual predictions. Texera cleans up raw match data, scrapes daily betting lines, and feeds everything into the model in a repeatable, visual way. The followings are some examples of how this can help: Clean data, ready for the model: Texera handles the messy work — combining stats from multiple sources, weighting recent games more heavily, and packaging it all into the format the model expects. |
Beta Was this translation helpful? Give feedback.
-
|
Texera also supports ML operators, especially useful for inference. I think this can be an App idea as I mentioned before. you can even design a new interface for Apps, and user can optionally click into details to see the underlying workflow. I hope to see this came out true! |
Beta Was this translation helpful? Give feedback.
-
|
I have it working in the terminal right now. Will look into making an interface today. Thank you for the advice. If anyone is curious about the results: Gave the model $30 currently at $632.37 after 1 week. |
Beta Was this translation helpful? Give feedback.
-
|
Explainable Result Pane for Texera I propose upgrading the result pane into an Interactive and Explainable Result Pane that combines two improvements:
This would help users not only see their data, but also understand their workflow. Instead of repeatedly switching between operators and manually inspecting outputs, users could explore, compare, and debug transformations from one place. |
Beta Was this translation helpful? Give feedback.
-
|
it would be a great project if you can polish the result table!!! one concern to keep in mind is about size: size of data could potentially be much larger than frontend's memory. and tables can be very wide. we could also do some frontend views of the same backend result: temporary reorder result, filter rows, hide columns, etc. don't let me restrict your imagination though! |
Beta Was this translation helpful? Give feedback.
-
Beta Was this translation helpful? Give feedback.
-
|
My idea is to implement Operators to allow for reading data from a news api in real time through web sockets, or polled at intervals. Depending on the daily sentiment, it will make a decision, and send a POST request to an external API that outputs its decisions. I'll have it make buy/sell decisions of stocks, or will have it make bets based on Kalshi. Key operators: something that polls from an API endpoint at fixed intervals. another operator that makes POST requests to an endpoint. Might make it poll at fixed intervals? Might just have an operator that makes requests, and instead of polling an endpoint, and just make a timer operator that repeats it while the workflow is active? Not too sure on this but will clarify as I develop more. |
Beta Was this translation helpful? Give feedback.
-
|
we do have some of those operators, checkout source ops. One old example is twitter source reading from tweets. But would be nice to have a general streaming operator that is always running, to accept events, it can serve the purpose for ingestion. I would recommend don't limit to adding an operator. make your project bigger scope that includes those operators, make a use case out of them, so that they are justified! |
Beta Was this translation helpful? Give feedback.
-
|
AI Agent as an operator. You send each row + system prompt into an LLM, which returns text and puts it into a new column, This is useful for classification, NLP, and much more! |
Beta Was this translation helpful? Give feedback.
-
|
Very good idea. Here are some related work in the recent literatures: https://github.com/lotus-data/lotus https://arxiv.org/abs/2407.11418 |
Beta Was this translation helpful? Give feedback.
-
|
Submitted! |
Beta Was this translation helpful? Give feedback.
-
|
Idea: Python UDF to a sub-DAG |
Beta Was this translation helpful? Give feedback.
-
|
Simple start stop commans to launch dev environment. |
Beta Was this translation helpful? Give feedback.
-
|
Added an AI Workflow Wizard to Texera that guides users through four steps: (1) define the analysis goal, (2) select the data source, (3) choose a scientific framework, and (4) set guardrails and constraints. After these steps, the wizard automatically generates a Texera workflow using the existing AI services and lets users review any missing required settings before applying it. The feature was built entirely on the frontend without changing Texera’s core backend code. |
Beta Was this translation helpful? Give feedback.
-
|
PR #5110 |
Beta Was this translation helpful? Give feedback.
-
|
Improve the dataset previewer to show more metadata of each file to get a better insight on the dataset. |
Beta Was this translation helpful? Give feedback.
-
Beta Was this translation helpful? Give feedback.
-
|
BioFlow Genesis: AI guesses what you want to do, from the dataset itself. Drop a CSV. Genesis profiles the data, column types, target candidates, value distributions, and proposes 4 domain-appropriate analyses as cards. A medical dataset gets classification cards. A numerical dataset gets regression cards. A text-heavy one gets exploration. The AI infers intent from the data, not from the user. One click on a card builds a wired Texera workflow with sklearn training and an AI Insight node that writes a 5-section interpretation from the results. |
Beta Was this translation helpful? Give feedback.
-
Beta Was this translation helpful? Give feedback.
-
|
The NL-to-workflow and "explain this workflow" ideas in this thread are especially interesting because they turn the agent service into a repeated workflow assistant, not just a one-off demo. I would meter agent calls by workflow, operator, model, and user action. That makes it easier to see which ideas are practical at scale and which need cheaper routing or caching before they become production features. I am testing an OpenAI-compatible multi-model API layer, and agentic data workflows are a strong use case for cost-aware routing. Would Texera's agent calls run centrally, or would users bring their own model/API configuration? |
Beta Was this translation helpful? Give feedback.
-
|
@bobbai00 Can you chime in? |
Beta Was this translation helpful? Give feedback.
and others
Uh oh!
There was an error while loading. Please reload this page.
-
Please use this post to share ideas you want to pursue in the dkNET-Texera hackathon (https://texera.io/hackathon/) this week at UCI. They can be about all kinds of topics such as
You are welcome to be creative :-)
Beta Was this translation helpful? Give feedback.
All reactions