What's Changed

🎉 Exciting New Features

📡 Add Streaming Oracle Destination

This feature adds a new custom exporter option for OracleDB allowing developers to easily export their streaming data to Oracle DB.

by @matrixstone in https://github.com/mage-ai/mage-ai/pull/4896

👤 Multi Project Platform Support

This feature adds support for each user to have their own active project within a multi project platform that keeps track of every project for every user.

by @dy46 in https://github.com/mage-ai/mage-ai/pull/4865

⚙️ Support Job Parameters in K8s Executor

Adds support to help manage environment variables, and control job execution and cleanup policies within the Kubernetes cluster.

k8s_executor_config:
  container_config:
    image: mageai/mageai:0.9.7
    env:
    - name: USER_CODE_PATH
      value: /home/src/k8s_project
  job:
    active_deadline_seconds: 120
    backoff_limit: 3
    ttl_seconds_after_finished: 86400

by @artche in https://github.com/mage-ai/mage-ai/pull/5044

🔑 Support OIDC_ROLES_MAPPING and user_roles

This feature is used to map roles obtained from an OpenID Connect (OIDC) identity provider to roles used within your application allowing for a standardized way to manage user roles across different identity providers.

by @mihaivint in https://github.com/mage-ai/mage-ai/pull/5053 and https://github.com/mage-ai/mage-ai/pull/4899

💅 Enhancements & Polish

• Optimize CPU and memory usage
  • [xy] Disable autoreload in non-dev environments to reduce CPU usage. by @wangxiaoyou1993 in https://github.com/mage-ai/mage-ai/pull/5136
  • [xy] Reduce CPU usage in scheduler. by @wangxiaoyou1993 in https://github.com/mage-ai/mage-ai/pull/5137
  • [td] Memory upgrades by @tommydangerous in https://github.com/mage-ai/mage-ai/pull/5092
  • [td] Track resource usage and run block execution function in a context by @tommydangerous in https://github.com/mage-ai/mage-ai/pull/5105
• Improve page performance
  • [xy] Some improvements on multi project by @wangxiaoyou1993 in https://github.com/mage-ai/mage-ai/pull/5063
  • [xy] Cache project platform settings data in API calls. by @wangxiaoyou1993 in https://github.com/mage-ai/mage-ai/pull/5079
  • [xy] Speed up pipeline update endpoint. by @wangxiaoyou1993 in https://github.com/mage-ai/mage-ai/pull/5125
• Improve dynamic blocks
  • [td] Dynamic blocks use the new memory enhancements by @tommydangerous in https://github.com/mage-ai/mage-ai/pull/5106
  • [td] Dynamic block stream mode by @tommydangerous in https://github.com/mage-ai/mage-ai/pull/5121
  • [td] Dynamic block resilience by @tommydangerous in https://github.com/mage-ai/mage-ai/pull/5113
• [xy] Support configuring logs_dir_path in environment variable. by @wangxiaoyou1993 in https://github.com/mage-ai/mage-ai/pull/5018
• [dy] Add prune option for fetch by @dy46 in https://github.com/mage-ai/mage-ai/pull/5014
• [xy] Include server and scheduler process id in status response. by @wangxiaoyou1993 in https://github.com/mage-ai/mage-ai/pull/5027
• [xy] Add pipeline_run_id to kwargs. by @wangxiaoyou1993 in https://github.com/mage-ai/mage-ai/pull/5030
• [xy] Allow configuring authentication mode in mssql config. by @wangxiaoyou1993 in https://github.com/mage-ai/mage-ai/pull/5051
• Reload libraries when running blocks from edit pipeline by @hugabora in https://github.com/mage-ai/mage-ai/pull/4953
• [xy] Support configuring concurrency config via env vars. by @wangxiaoyou1993 in https://github.com/mage-ai/mage-ai/pull/5057
• [jk] Improve visibility of variable/secret names by @johnson-mage in https://github.com/mage-ai/mage-ai/pull/5072
• relax pytz version requirements by @alex-hunsaker in https://github.com/mage-ai/mage-ai/pull/5006
• upgrade deltalake for tableMerger by @oonyoontong in https://github.com/mage-ai/mage-ai/pull/5020
• [td] Turn off unoptimized images by @tommydangerous in https://github.com/mage-ai/mage-ai/pull/5082
• Add pipeline_schedule_description variable to notification sender by @hjhdaniel in https://github.com/mage-ai/mage-ai/pull/5118

🐛 Bug Fixes

• [xy] Exclude dropped columns from the postgres schema discovery query. by @wangxiaoyou1993 in https://github.com/mage-ai/mage-ai/pull/5002
• [xy] Fix None Bigquery database error. by @wangxiaoyou1993 in https://github.com/mage-ai/mage-ai/pull/5005
• [xy] Fix extracting table name from INSERT statement in SQL block. by @wangxiaoyou1993 in https://github.com/mage-ai/mage-ai/pull/5008
• [xy] Catch yaml interpolation error. by @wangxiaoyou1993 in https://github.com/mage-ai/mage-ai/pull/5012
• [xy] Not run insert query for empty dataframe. by @wangxiaoyou1993 in https://github.com/mage-ai/mage-ai/pull/5017
• [dy] Fix git and seed issues by @dy46 in https://github.com/mage-ai/mage-ai/pull/5025
• [xy] Fix custom template issue on server start. by @wangxiaoyou1993 in https://github.com/mage-ai/mage-ai/pull/5038
• [xy] Comment out flaky unit tests by @wangxiaoyou1993 in https://github.com/mage-ai/mage-ai/pull/5046
• [jk] Allow files to be selected for pipeline zip import by @johnson-mage in https://github.com/mage-ai/mage-ai/pull/5042
• [dy] Update git email field by @dy46 in https://github.com/mage-ai/mage-ai/pull/5045
• [dy] Fix circular dependency by @dy46 in https://github.com/mage-ai/mage-ai/pull/5050
• [td] Update charts and make a ton of bug fixes by @tommydangerous in https://github.com/mage-ai/mage-ai/pull/5075
• [td] Fix IDE outputs by @tommydangerous in https://github.com/mage-ai/mage-ai/pull/5088
• [td] Fix git issue by @tommydangerous in https://github.com/mage-ai/mage-ai/pull/5089
• Td fix git2 by @tommydangerous in https://github.com/mage-ai/mage-ai/pull/5090
• [td] Upgrade deltalake because of Docker image build error by @tommydangerous in https://github.com/mage-ai/mage-ai/pull/5091
• [xy] Fix websocket authentication. by @wangxiaoyou1993 in https://github.com/mage-ai/mage-ai/pull/5094
• [td] Fix interpolating dictionaries when adding custom code to ide execution by @tommydangerous in https://github.com/mage-ai/mage-ai/pull/5095
• [xy] Fix switching active kernel by @wangxiaoyou1993 in https://github.com/mage-ai/mage-ai/pull/5098
• [xy] Fix retry in streaming pipeline. by @wangxiaoyou1993 in https://github.com/mage-ai/mage-ai/pull/5099
• [xy] Fix performance issue of fetching variables. by @wangxiaoyou1993 in https://github.com/mage-ai/mage-ai/pull/5108
• [jk] Fix max concurrent runs value for backfill by @johnson-mage in https://github.com/mage-ai/mage-ai/pull/5115
• [xy] Fix circular dependency. by @wangxiaoyou1993 in https://github.com/mage-ai/mage-ai/pull/5122
• [td] Don’t raise error if can’t load by @tommydangerous in https://github.com/mage-ai/mage-ai/pull/5132
• [xy] Fix custom template creation. by @wangxiaoyou1993 in https://github.com/mage-ai/mage-ai/pull/5134
• [xy] Fix variables API and BaseEnum by @wangxiaoyou1993 in https://github.com/mage-ai/mage-ai/pull/5135
• [td] Fix after on pipeline dashboard by @tommydangerous in https://github.com/mage-ai/mage-ai/pull/5142
• [td] Fix after panel showing up everywhere by @tommydangerous in https://github.com/mage-ai/mage-ai/pull/5143
• [td] Fix charts when using Spark by @tommydangerous in https://github.com/mage-ai/mage-ai/pull/5028
• [xy] Not show locals in CLI exception. by @wangxiaoyou1993 in https://github.com/mage-ai/mage-ai/pull/5009
• [dy] Remove error pop up by @dy46 in https://github.com/mage-ai/mage-ai/pull/5015
• [td] Fix interactions, triggers, dynamic blocks, charts, and more by @tommydangerous in https://github.com/mage-ai/mage-ai/pull/5064
• [td] Fix infer variables by @tommydangerous in https://github.com/mage-ai/mage-ai/pull/5102
• Escaping Backslashes when interpolating code content by @jimilp7 in https://github.com/mage-ai/mage-ai/pull/5112

✨ New Contributors

• @alex-hunsaker made their first contribution in https://github.com/mage-ai/mage-ai/pull/5006
• @mihaivint made their first contribution in https://github.com/mage-ai/mage-ai/pull/4899
• @stefaan1o made their first contribution in https://github.com/mage-ai/mage-ai/pull/5034
• @richardlhughes made their first contribution in https://github.com/mage-ai/mage-ai/pull/5033
• @artche made their first contribution in https://github.com/mage-ai/mage-ai/pull/5044
• @CLHdevOps made their first contribution in https://github.com/mage-ai/mage-ai/pull/5077
• @jimilp7 made their first contribution in https://github.com/mage-ai/mage-ai/pull/5112

Full Changelog: https://github.com/mage-ai/mage-ai/compare/0.9.70...0.9.71