Release Highlights

Notice: This PR includes a fix for Single Sign-On (OIDC) that was introduced in the previous release, v0.9.4.

Important Release Notes

With this release, if you are using Neo4J as your graph implementation, you need to set: GRAPH_SERVICE_DIFF_MODE_ENABLED=false

For GMS (or MAE Consumer for standalone mode).

User Experience

• Manual Lineage is LIVE! You can now add and remove lineage between entities in the Lineage Visualization screen, making it easier than ever to manage the complex relationships between your data resources.

• Our new Views feature makes it easy to create curated sets of Entities within DataHub. This is a great way to start to isolate the entities that matter most, and provide your DataHub end-users with a streamlined view of the assets that are relevant to their use cases. See the original demo video.

• In-App Product Tours are here! When logging into DataHub and/or visiting a new page type for the first time, new users will be prompted with a helpful walkthrough of core functionality to get them familiar with the platform. We’ll continue to add modules as we roll out new features!

• Automatically send updates to Slack and/or Microsoft Teams when changes are made within DataHub by leveraging our the new Slack and Teams Actions.

Metadata Ingestion

We’re continuing to improve the user experience for UI-based ingestion for the following sources:

• DataBricks Unity Catalog
• dbt Cloud
• MySQL
• Trino/Presto
• Microsoft SQL Server
• MariaDB

If you’re just getting started with UI-based Ingestion, check out our new BigQuery & Snowflake guides.

Stateful ingestion is now supported for Iceberg (thanks for the contrib, @cccs-Dustin!) and LDAP (thanks for the contrib, @bda618!)

What's Changed

• feat(ingest): add failure/warning counts to ingest_stats by @hsheth2 in https://github.com/datahub-project/datahub/pull/6823
• refactor(ingest): clean up pipeline init error handling by @hsheth2 in https://github.com/datahub-project/datahub/pull/6817
• fix(ingest): exclude ztsd from uber jar to prevent jni conflicts with spark by @danielli-ziprecruiter in https://github.com/datahub-project/datahub/pull/6787
• feat(ingest/bigquery): add option to enable/disable legacy sharded table support by @treff7es in https://github.com/datahub-project/datahub/pull/6822
• fix(ingest): support patches in auto_status_aspect by @hsheth2 in https://github.com/datahub-project/datahub/pull/6827
• fix(ci): reduce flakiness views select test by @anshbansal in https://github.com/datahub-project/datahub/pull/6821
• refactor(ingest): clean up exception types by @hsheth2 in https://github.com/datahub-project/datahub/pull/6818
• fix(ingest): fixed snowflake oauth ingestion not using role attribute… by @daguito81 in https://github.com/datahub-project/datahub/pull/6825
• refactor(ingestion): Browse Paths Upgrade V2 Feast & Sagemaker by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/6002
• fix(lineage) Fix lineage viz with multiple siblings by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/6826
• fix(pac4j-oidc): add verifier parameter by @david-leifker in https://github.com/datahub-project/datahub/pull/6835
• feat(ingest): extract kafka topic config properties as customProperties by @mayurinehate in https://github.com/datahub-project/datahub/pull/6783
• docs: Incorrect import statement fixed in example by @mirac-cisco in https://github.com/datahub-project/datahub/pull/6838
• feat(ingestion): support lineage for delta lake writes by @danielli-ziprecruiter in https://github.com/datahub-project/datahub/pull/6834
• feat(ui): Support defining the ID for Glossary Term and Glossary Term Group in UI by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/6830
• feat(ci): add cypress test ui based ingestion by @anshbansal in https://github.com/datahub-project/datahub/pull/6769
• feat(ui): sortable domain list by @looppi in https://github.com/datahub-project/datahub/pull/6736
• fix(ci): add labels based on more folders by @anshbansal in https://github.com/datahub-project/datahub/pull/6840
• fix(ingest): kafka ingest task hand up with error bootstrap server by @wangsaisai in https://github.com/datahub-project/datahub/pull/6820
• fix(ingest): Fixing Kafka source linting by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/6844
• fix(ingestion) Inject pipeline_name into recipes at runtime by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/6833
• feat(ingest): add db/schema properties hook to SQL common by @hsheth2 in https://github.com/datahub-project/datahub/pull/6847
• fix(oidc): fix oidc authentication loop by @david-leifker in https://github.com/datahub-project/datahub/pull/6848
• docs(confluent): add details for actions pod for confluent by @RyanHolstien in https://github.com/datahub-project/datahub/pull/6810
• feat(ingestion): Business Glossary# Add domain support in GlossaryTerm ingestion by @mohdsiddique in https://github.com/datahub-project/datahub/pull/6829
• fix(ingest/looker): handle missing label fields by @hsheth2 in https://github.com/datahub-project/datahub/pull/6849
• refactor(ui): Misc domains improvements by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/6850

New Contributors

• @daguito81 made their first contribution in https://github.com/datahub-project/datahub/pull/6825
• @mirac-cisco made their first contribution in https://github.com/datahub-project/datahub/pull/6838

Full Changelog: https://github.com/datahub-project/datahub/compare/v0.9.4...v0.9.5