![]() Using data pipelines, Astronomer extends data orchestration capabilities to machine learning operations, according to Fletcher, who said that Apache Airflow comes in handy. (* Disclosure below.) Joining the data orchestration and MLOps dots They discussed how Astronomer uses Apache Airflow to enhance data orchestration and MLOps. Hillion and Jeff Fletcher (right), director of field engineering and machine learning at Astronomer, spoke with theCUBE industry analyst Lisa Martin at the AWS Startup Showcase: “Top Startups Building Generative AI on AWS” event, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. “We run something like a million data operations every month within my team … the ability to spin up new production workflows essentially in a single day, you go from an idea in the morning to a new dashboard or a new model in the afternoon. “We went from at the beginning of last year, about 500 data tasks that we were running on a daily basis to about 15,000 every day,” he said. uses it to remove friction when operationalizing machine learning and data workflows, according to Steven Hillion (pictured, left), chief data officer of Astronomer. Since Apache Airflow has emerged as the de facto standard to orchestrate data pipelines, Astronomer Inc. And doing so effectively depends on a reliable, scalable and easy way to develop and run data workflows. Access to real-time data is no longer a nice-to-have for organizations it’s an imperative.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |