filmov
tv
MCTS Enhanced AI AGENTS: SELA (Stanford, UC Berkeley)
Показать описание
NEW AI research presents Tree-Search Enhanced LLM Agents (SELA), a novel framework that combines Large Language Model (LLM) agents with Monte Carlo Tree Search (MCTS) to improve Automated Machine Learning (AutoML) processes.
Traditional AutoML systems often rely on fixed pipelines and predefined search spaces, focusing mainly on hyperparameter optimization and model ensembling, which limits their adaptability to diverse datasets and complex tasks. LLM-based agents have attempted to automate machine learning tasks by generating code from natural language prompts, but they frequently produce low-diversity and suboptimal solutions due to a lack of iterative refinement and limited search strategies.
SELA addresses these limitations by representing machine learning pipeline configurations as a hierarchical tree, where each node corresponds to a specific action or method at a particular stage of the pipeline (such as data preprocessing, feature engineering, or model training). By employing MCTS, SELA systematically explores this tree-structured search space, intelligently balancing exploration of new configurations with exploitation of known promising strategies.
The core innovation of SELA lies in its integration of MCTS with LLM agents to iteratively refine machine learning pipelines based on experimental feedback. The LLM serves both as an insight proposer—generating diverse methods for each stage of the machine learning process based on the problem description and dataset—and as an experiment executor, planning, coding, and executing the pipelines.
During each iteration, MCTS selects a path through the tree representing a specific pipeline configuration, which the LLM agent then simulates by generating and executing the corresponding code to obtain performance metrics. These metrics are fed back into the MCTS algorithm during the backpropagation step, updating the node values and influencing future selections. This iterative, feedback-driven process allows SELA to mimic the problem-solving approach of human experts, systematically conducting experiments and refining strategies to discover high-performance pipelines.
Evaluations on 20 datasets from the AutoML Benchmark demonstrate that SELA consistently outperforms traditional AutoML systems and existing agent-based approaches, achieving a win rate of 65% to 80% against each baseline, thus validating its effectiveness and adaptability in automating complex machine learning tasks.
All rights w/ authors:
SELA: TREE-SEARCH ENHANCED LLM AGENTS FOR
AUTOMATED MACHINE LEARNING
by Yizhou Chi, Yizhang Lin, Sirui Hong, Duyi Pan, Yaying Fei, Guanghao Mei, Bangbang Liu, Tianqi Pang, Jacky Kwok, Ceyao Zhang, Bang Liu, Chenglin Wu
#airesearch
#aiagents
#artificialintelligence
#ai
Traditional AutoML systems often rely on fixed pipelines and predefined search spaces, focusing mainly on hyperparameter optimization and model ensembling, which limits their adaptability to diverse datasets and complex tasks. LLM-based agents have attempted to automate machine learning tasks by generating code from natural language prompts, but they frequently produce low-diversity and suboptimal solutions due to a lack of iterative refinement and limited search strategies.
SELA addresses these limitations by representing machine learning pipeline configurations as a hierarchical tree, where each node corresponds to a specific action or method at a particular stage of the pipeline (such as data preprocessing, feature engineering, or model training). By employing MCTS, SELA systematically explores this tree-structured search space, intelligently balancing exploration of new configurations with exploitation of known promising strategies.
The core innovation of SELA lies in its integration of MCTS with LLM agents to iteratively refine machine learning pipelines based on experimental feedback. The LLM serves both as an insight proposer—generating diverse methods for each stage of the machine learning process based on the problem description and dataset—and as an experiment executor, planning, coding, and executing the pipelines.
During each iteration, MCTS selects a path through the tree representing a specific pipeline configuration, which the LLM agent then simulates by generating and executing the corresponding code to obtain performance metrics. These metrics are fed back into the MCTS algorithm during the backpropagation step, updating the node values and influencing future selections. This iterative, feedback-driven process allows SELA to mimic the problem-solving approach of human experts, systematically conducting experiments and refining strategies to discover high-performance pipelines.
Evaluations on 20 datasets from the AutoML Benchmark demonstrate that SELA consistently outperforms traditional AutoML systems and existing agent-based approaches, achieving a win rate of 65% to 80% against each baseline, thus validating its effectiveness and adaptability in automating complex machine learning tasks.
All rights w/ authors:
SELA: TREE-SEARCH ENHANCED LLM AGENTS FOR
AUTOMATED MACHINE LEARNING
by Yizhou Chi, Yizhang Lin, Sirui Hong, Duyi Pan, Yaying Fei, Guanghao Mei, Bangbang Liu, Tianqi Pang, Jacky Kwok, Ceyao Zhang, Bang Liu, Chenglin Wu
#airesearch
#aiagents
#artificialintelligence
#ai
0:28:05
MCTS Enhanced AI AGENTS: SELA (Stanford, UC Berkeley)
1:10:21
Synapse + Power BI - Jesús Gil y Marco Tulio Gómez - SOLCOMP
Комментарии