mage-ai/mage-ai
Fork: 774 Star: 7969 (更新于 2024-11-20 09:55:35)
license: Apache-2.0
Language: Python .
🧙 Build, run, and manage data pipelines for integrating and transforming data.
最后发布版本： 0.9.74 ( 2024-09-20 02:55:47)
Mage AI
Give your data team magical powers.
Mage is a hybrid framework for transforming and integrating data. It combines the best of both worlds: the flexibility of notebooks with the rigor of modular code.
- Extract and synchronize data from 3rd party sources.
- Transform data with real-time and batch pipelines using Python, SQL, and R.
- Load data into your data warehouse or data lake using our pre-built connectors.
- Run, monitor, and orchestrate thousands of pipelines without losing sleep.
Plus hundreds of enterprise-class features, infrastructure innovations, and magical surprises.
Available in two spellbinding versions
For teams. Fully managed platform
for integrating and transforming data.
|
Self-hosted. System to build, run, and
manage data pipelines.
|
It’s magic.
For documentation on getting started, how to develop, and how to deploy to production check out the live
Developer documentation portal.
🏃♀️ Install
The recommended way to install the latest version of Mage is through Docker with the following command:
docker pull mageai/mageai:latest
You can also install Mage using pip or conda, though this may cause dependency issues without the proper environment.
pip install mage-ai
conda install -c conda-forge mage-ai
Looking for help? The fastest way to get started is by checking out our documentation here.
Looking for quick examples? Open a demo project right in your browser or check out our guides.
🎮 Demo
Live demo
Build and run a data pipeline with our demo app.
WARNING
The live demo is public to everyone, please don’t save anything sensitive (e.g. passwords, secrets, etc).
Demo video (5 min)
Click the image to play video
🔮 Features
| 🎶 | Orchestration | Schedule and manage data pipelines with observability. |
| 📓 | Notebook | Interactive Python, SQL, & R editor for coding data pipelines. |
| 🏗️ | Data integrations | Synchronize data from 3rd party sources to your internal destinations. |
| 🚰 | Streaming pipelines | Ingest and transform real-time data. |
| ❎ | dbt | Build, run, and manage your dbt models with Mage. |
A sample data pipeline defined across 3 files ➝
- Load data ➝
@data_loader def load_csv_from_file() -> pl.DataFrame: return pl.read_csv('default_repo/titanic.csv') - Transform data ➝
@transformer def select_columns_from_df(df: pl.DataFrame, *args) -> pl.DataFrame: return df[['Age', 'Fare', 'Survived']] - Export data ➝
@data_exporter def export_titanic_data_to_disk(df: pl.DataFrame) -> None: df.to_csv('default_repo/titanic_transformed.csv')
最近版本更新:(数据更新于 2024-10-05 11:40:29)
2024-09-20 02:55:47 0.9.74
2024-08-02 22:59:23 0.9.73
2024-06-21 22:32:03 0.9.72
2024-06-01 06:55:11 0.9.71
2024-04-27 07:53:46 0.9.70
2024-03-28 23:35:49 0.9.68
2024-03-14 08:08:00 0.9.66
2024-02-29 07:59:09 0.9.65
2024-02-16 10:43:43 0.9.64
2024-02-08 12:06:30 0.9.63
主题(topics):
artificial-intelligence, data, data-engineering, data-integration, data-pipelines, data-science, dbt, elt, etl, machine-learning, orchestration, pipeline, pipelines, python, reverse-etl, spark, sql, transformation
mage-ai/mage-ai同语言 Python最近更新仓库
2024-11-22 02:39:01 goauthentik/authentik
2024-11-22 00:03:47 comfyanonymous/ComfyUI
2024-11-21 22:06:18 rashevskyv/dbi
2024-11-21 21:09:02 xtekky/gpt4free
2024-11-21 20:03:58 ultralytics/ultralytics
2024-11-21 00:54:04 hect0x7/JMComic-Crawler-Python