aster.cloud aster.cloud
  • /
  • Platforms
    • Public Cloud
    • On-Premise
    • Hybrid Cloud
    • Data
  • Architecture
    • Design
    • Solutions
    • Enterprise
  • Engineering
    • Automation
    • Software Engineering
    • Project Management
    • DevOps
  • Programming
    • Learning
  • Tools
  • About
  • /
  • Platforms
    • Public Cloud
    • On-Premise
    • Hybrid Cloud
    • Data
  • Architecture
    • Design
    • Solutions
    • Enterprise
  • Engineering
    • Automation
    • Software Engineering
    • Project Management
    • DevOps
  • Programming
    • Learning
  • Tools
  • About
aster.cloud aster.cloud
  • /
  • Platforms
    • Public Cloud
    • On-Premise
    • Hybrid Cloud
    • Data
  • Architecture
    • Design
    • Solutions
    • Enterprise
  • Engineering
    • Automation
    • Software Engineering
    • Project Management
    • DevOps
  • Programming
    • Learning
  • Tools
  • About
  • Hybrid Cloud

IBM’s CodeFlare Significantly Cuts The Time To Automate Transfer Learning Tasks For Foundation Models

  • aster.cloud
  • December 22, 2021
  • 2 minute read

Earlier this year, we presented CodeFlare at the Ray Summit. Our open-source tool promised data scientists that it would reduce the time it took to set up, run, and scale machine learning (ML) models. The goal of the tool was to reduce the amount of time scientists spent configuring their experiments, allowing them to spend more time diving into the results of their research. And back then, users applied our framework to analyze and optimize hundreds of thousands of pipelines for training ML models — with CodeFlare cutting the training time for many from hours to minutes.

Just six months later, we’ve refined CodeFlare so that it can take on foundation models,1 and it’s now available as open-source software. This effectively takes CodeFlare from an exploratory tool for data science researchers to a tool that can automate AI and ML workflows on IBM’s hybrid cloud.


Partner with aster.cloud
for your next big idea.
Let us know here.



From our partners:

CITI.IO :: Business. Institutions. Society. Global Political Economy.
CYBERPOGO.COM :: For the Arts, Sciences, and Technology.
DADAHACKS.COM :: Parenting For The Rest Of Us.
ZEDISTA.COM :: Entertainment. Sports. Culture. Escape.
TAKUMAKU.COM :: For The Hearth And Home.
ASTER.CLOUD :: From The Cloud And Beyond.
LIWAIWAI.COM :: Intelligence, Inside and Outside.
GLOBALCLOUDPLATFORMS.COM :: For The World's Computing Needs.
FIREGULAMAN.COM :: For The Fire In The Belly Of The Coder.
ASTERCASTER.COM :: Supra Astra. Beyond The Stars.
BARTDAY.COM :: Prosperity For Everyone.

Faster, automated foundation model training

Businesses use foundation models for a multitude of tasks. For example, a financial-services company could customize a foundation model just for sentiment analysis. Today, this takes an incredible amount of time, sometimes even weeks, to gather and train an AI model on the right body of data, combined with multiple upstream and downstream tasks — each often written by different, siloed teams.

Our goal is to make the generation of downstream models ‘one-click’ easy for data scientists.

CodeFlare simplifies this entire process using a Python-based interface for these foundation model “pipelines” — a sequence of multiple, often heterogeneous steps — making it simpler to integrate, parallelize, and share data. These tasks of preprocessing, validating, and adapting foundation models for business use cases are now fully automated on a hybrid cloud platform.

Read More  Vertex AI Foundations For Secure And Compliant ML/AI Deployment

How CodeFlare works

In the sentiment analysis case, CodeFlare starts off by cleaning up the input data, including de-duplication, and removing unsafe or biased content. Then, it tunes a foundation model for all of the specific tasks needed for the organization’s sentiment analysis. With just a few lines of code, a data scientist can operationalize hundreds of such pipelines and automate these tasks whenever they need to make any changes.

Our goal is to make generating downstream models as easy for data scientists as a single mouse click. Today, organizations can use CodeFlare on their own datasets on our hybrid cloud platform, without the data leaving their environment. Financial sentiment analysis is a motivating use case for our team: Internally, we’re using it to show business users how CodeFlare automates our validation of foundation language models on various state-of-the-art benchmarks.

This is just one of the many use cases CodeFlare could help simplify. You can download and experiment with CodeFlare on your own processes today.

The authors would like to acknowledge the entire CodeFlare team involved in advancing this work, including Edward Epstein, Linsong Chu, Masayasu Muraoka, Marquita Ellis, Rong Zhang.

References

  1. Awasthy, P., Bhattacharjee, B., Kender, J., Florian, R. Predictive Model Selection for Transfer Learning in Sequence Labeling Tasks. Association for Computational Linguistics. p. 113-118. (2020).

 

 

By Bishwaranjan Bhattacharjee | Raghu Ganti | Carlos Costa | Mudhakar Srivatsa | Nick Fuller
Source IBM


For enquiries, product placements, sponsorships, and collaborations, connect with us at [email protected]. We'd love to hear from you!

Our humans need coffee too! Your support is highly appreciated, thank you!

aster.cloud

Related Topics
  • CodeFlare
  • IBM
  • Machine Learning
  • Ray Summit
You May Also Like
oracle-ibm
View Post
  • Hybrid Cloud
  • Technology

IBM and Oracle Expand Partnership to Advance Agentic AI and Hybrid Cloud

  • May 6, 2025
View Post
  • Hybrid Cloud
  • Technology

IBM Completes Acquisition of HashiCorp, Creates Comprehensive, End-to-End Hybrid Cloud Platform

  • February 27, 2025
Vehicle Manufacturing
View Post
  • Hybrid Cloud
  • Public Cloud

Toyota shifts into overdrive: Developing an AI platform for enhanced manufacturing efficiency

  • December 10, 2024
Cloud computing concept image double exposure Digitally Enhanced Smart City Concept with Cloud Computing
View Post
  • Cloud-Native
  • Computing
  • Hybrid Cloud
  • Multi-Cloud
  • Public Cloud

Make Your Business Resilient By Integrating These Best Practices Into Your Cloud Architecture

  • July 29, 2024
IBM
View Post
  • Hybrid Cloud
  • Platforms

IBM To Acquire HashiCorp, Inc. Creating A Comprehensive End-to-End Hybrid Cloud Platform

  • April 24, 2024
Clouds
View Post
  • Computing
  • Hybrid Cloud

Research Finds IT Leaders Are Choosing Hybrid Cloud Strategies Due To Flexibility, Cost-Effectiveness, And Security

  • February 14, 2024
View Post
  • Hybrid Cloud
  • Platforms

New IBM LinuxONE 4 Express To Offer Cost Savings And Client Value Through A Cyber Resilient Hybrid Cloud And AI Platform

  • February 6, 2024
Red Hat
View Post
  • Hybrid Cloud

DGB Chooses Red Hat OpenShift As Hybrid Cloud Infrastructure Foundation

  • January 18, 2024

Stay Connected!
LATEST
  • Camping 1
    The Summer Adventures : Camping Essentials
    • June 27, 2025
  • Host a static website on AWS with Amazon S3 and Route 53
    • June 27, 2025
  • Prioritize security from the edge to the cloud
    • June 25, 2025
  • 6 edge monitoring best practices in the cloud
    • June 25, 2025
  • Genome 5
    AlphaGenome: AI for better understanding the genome
    • June 25, 2025
  • 6
    Pure Accelerate 2025: All the news and updates live from Las Vegas
    • June 18, 2025
  • 7
    ‘This was a very purposeful strategy’: Pure Storage unveils Enterprise Data Cloud in bid to unify data storage, management
    • June 18, 2025
  • What is cloud bursting?
    • June 18, 2025
  • 9
    There’s a ‘cloud reset’ underway, and VMware Cloud Foundation 9.0 is a chance for Broadcom to pounce on it
    • June 17, 2025
  • What is confidential computing?
    • June 17, 2025
about
Hello World!

We are aster.cloud. We’re created by programmers for programmers.

Our site aims to provide guides, programming tips, reviews, and interesting materials for tech people and those who want to learn in general.

We would like to hear from you.

If you have any feedback, enquiries, or sponsorship request, kindly reach out to us at:

[email protected]
Most Popular
  • Oracle adds xAI Grok models to OCI
    • June 17, 2025
  • Fine-tune your storage-as-a-service approach
    • June 16, 2025
  • 3
    Advanced audio dialog and generation with Gemini 2.5
    • June 15, 2025
  • Google Cloud, Cloudflare struck by widespread outages
    • June 12, 2025
  • 5
    Global cloud spending might be booming, but AWS is trailing Microsoft and Google
    • June 13, 2025
  • /
  • Technology
  • Tools
  • About
  • Contact Us

Input your search keywords and press Enter.