August 29, 2024
Editors' notes
This article has been reviewed according to Science X's editorial process and policies. Editors have highlighted the following attributes while ensuring the content's credibility:
fact-checked
trusted source
proofread
Simulating learning methodology: An approach to machine learning automation
As a fundamental technology of artificial intelligence, existing machine learning (ML) methods often rely on extensive human intervention and manually presetting, like manually collecting, selecting, and annotating data, manually constructing the fundamental architecture of deep neural networks, and determining the algorithm types and their hyperparameters of the optimization algorithms, etc. These limitations hamper the ability of ML to effectively deal with complex data and varying multi-tasks environments in the real world.
To address these existing challenges in current ML, the research team from Xian Jiaotong University has developed a novel approach called simulating learning methodology (SLeM). The core concept of SLeM is to simulate and extract the learning methodology of ML traditionally set by humans, transforming it into an automated learning process. Essentially, the SLeM framework represents an ML for ML paradigm, where ML tools are employed to design and optimize MLs fundamental components.
The team has developed a series of algorithms for ML automation based on the SLeM framework, demonstrating its effectiveness in enhancing the adaptive learning capabilities of existing ML methods.
"Recently, many AutoML methods have been proposed to achieve ML automation. However, most existing AutoML methods are heuristic in nature, making it difficult to establish a solid theoretical foundation. In contrast, the SLeM framework offers a unified mathematical formulation for ML automation and provides theoretical insights into the task transfer generalization capabilities of SLeM," says Prof. Zongben Xu, the lead author of this paper and an academician of the Chinese Academy of Sciences.
The development of advanced large language models (LLMs) has become a cornerstone of artificial intelligence, significantly expanding the capacity to solve a wide range of applications and tasks. However, the underlying theoretical evidence for LLMs' remarkable task generalization abilities has not been fully addressed by the ML community. The novel SLeM approach provides a promising perspective and tool for advancing the study and understanding of task generalization capabilities in large language models (LLMs).
More information: Zongben Xu et al, Simulating learning methodology (SLeM): An approach to machine learning automation, National Science Review (2024). DOI: 10.1093/nsr/nwae277
Provided by Science China Press Citation: Simulating learning methodology: An approach to machine learning automation (2024, August 29) retrieved 29 August 2024 from https://techxplore.com/news/2024-08-simulating-methodology-approach-machine-automation.html This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.
Explore further
In-depth analysis: Automated machine learning from the perspective of bilevel optimization 14 shares
Feedback to editors
