Release Highlights

Known Issues

When using stand-alone MAE consumers (mae-consumer-job) this release will not work; this has been resolved in v0.8.40.

User Experience

• NEW: support for surfacing outcomes of dbt Tests in dataset entity pages (see it in action here)
• NEW: Improved navigation of dbt resources: dbt models and their associated warehouse tables are now merged into a unified entity (see it here). This will automatically be enabled for all newly ingested entities. To view this for entities you have already ingested, you will need to run a restore indices job.
• Improvement to Impact Analysis: When looking at the Lineage tab, you can now easily toggle between “Upstream” and “Downstream” entities (try it out here)

Developer Experience

• NEW: Java Kafka Emitter – Use this when you want to decouple your metadata producer from the uptime of your datahub metadata server by utilizing Kafka as a highly available message bus

Metadata Ingestion

• NEW: Make bulk edits to your metadata via CSV (read more)
• Snowflake ingestion improvements: configure profiling to run only if they have been updated within the prior N days
• Managed ingestion update: removed need for sink block

What's Changed

• fix(ui-ingestion): update looker ingestion warning banner by @aditya-radhakrishnan in https://github.com/datahub-project/datahub/pull/5142
• chore: Bump Default UI Ingestion Version 0.8.38 by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/5145
• feat(schema): support rendering schemas with . in field names by @gabe-lyons in https://github.com/datahub-project/datahub/pull/5141
• feat(dbt): Platform instances for target platform by @skrydal in https://github.com/datahub-project/datahub/pull/5129
• feat(ingest): snowflake profile tables only if they have been updates… by @mayurinehate in https://github.com/datahub-project/datahub/pull/5132
• fix(airflow): fixes DeprecationWarning with hook-class-names by @sayakmaity in https://github.com/datahub-project/datahub/pull/5143
• feat(frontend): Parse JWT access token claims by @chen4119 in https://github.com/datahub-project/datahub/pull/5138
• fix(tokens): Using keyword search filters for ListAccessTokensResolver by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/5154
• feat(ui) Update the max text length of Terms/Term Groups by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/5162
• docs(policies): add info about Manage User Credentials by @aditya-radhakrishnan in https://github.com/datahub-project/datahub/pull/5157
• fix(restore-indices): Do not fail on MAE row count diff by @dexter-mh-lee in https://github.com/datahub-project/datahub/pull/5165
• fix(Kafka-setup): Make sure it doesn't fail when the new envs are not set by @dexter-mh-lee in https://github.com/datahub-project/datahub/pull/5168
• chore(deps): Bump Nimbus Jose JWT dependency by @pedro93 in https://github.com/datahub-project/datahub/pull/5158
• fix(recs): Verify that an entity exists before recommending by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/5163
• fix(business glossary): setting properties to be empty if the node has no properties aspect by @gabe-lyons in https://github.com/datahub-project/datahub/pull/5166
• refactor(ui): Misc improvements to Dataset Assertions UI by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/5155
• chore(guava): force version of guava in client jars per #5134 by @RyanHolstien in https://github.com/datahub-project/datahub/pull/5153
• feat(boot): Make Glossary Term Upgrade Async by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/5164
• fix(frontend): Add iam auth jar to frontend by @dexter-mh-lee in https://github.com/datahub-project/datahub/pull/5171
• docs(features): update & clean up Features page by @maggiehays in https://github.com/datahub-project/datahub/pull/5175
• fix(glue): fix glue profiling config option by @kangseonghyun in https://github.com/datahub-project/datahub/pull/5178
• feat(upgrade) Check version when determining to run RestoreGlossaryIndices step by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/5182
• fix(jaas): fixed auth.jaas.enabled option parsing by @alexey-kravtsov in https://github.com/datahub-project/datahub/pull/5179
• feat(ingestion): bigquery - Option to send usage queries as well as Operational metadata by @treff7es in https://github.com/datahub-project/datahub/pull/5151
• feat(build): changes to decrease build time, cancel runs in case of multiple commits by @anshbansal in https://github.com/datahub-project/datahub/pull/5187
• refactor(docs): Update Metadata Events Docs by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/5173
• fix(ingest): If there is no manager for a LDAP user (example: system account) by @bda618 in https://github.com/datahub-project/datahub/pull/5180
• bug(ingest): correct case of sys views for mssql description populati… by @BALyons in https://github.com/datahub-project/datahub/pull/5186
• refactor(configs): Simplify Kafka Topic name configurations + docs by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/5198
• feat(ingest): dbt - adding support for dbt tests by @shirshanka in https://github.com/datahub-project/datahub/pull/5201
• fix(cli): correct handling of env variables by @anshbansal in https://github.com/datahub-project/datahub/pull/5203
• feat(ci): split integration tests to reduce run time by @anshbansal in https://github.com/datahub-project/datahub/pull/5205
• feat(datahub-client): add java kafka emitter by @MugdhaHardikar-GSLab in https://github.com/datahub-project/datahub/pull/5074
• feat(graphql): add metrics capturing for graphql latency by @RyanHolstien in https://github.com/datahub-project/datahub/pull/5200
• test(ingestion): bigquery-usage - Adding tests for bigquery usage filters by @treff7es in https://github.com/datahub-project/datahub/pull/5195
• fix(ui): load monaco-editor as a dependency and not from a third party CDN by @Masterchen09 in https://github.com/datahub-project/datahub/pull/5189
• feat(cli): Add token parameter for sample ingestion by @pedro93 in https://github.com/datahub-project/datahub/pull/5160
• feat(lineage) Update Lineage tab and Impact Analysis feature by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/5121
• fix(ingest): add missing ownership types by @afghori in https://github.com/datahub-project/datahub/pull/5209
• feat(ingestion) ldap: make ldap attrs keys configurable by @atulsaurav in https://github.com/datahub-project/datahub/pull/4682
• Remove unnecessary space from application.yml of GMS by @mmmeeedddsss in https://github.com/datahub-project/datahub/pull/5216
• fix(upgrade): fix upgrade when s3 path has = by @RyanHolstien in https://github.com/datahub-project/datahub/pull/5220
• feat(docs) Add and update docs for the new Glossary experience by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/5211
• feat(glossary) Add empty state for the Business Glossary home page by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/5217
• feat(bootstrap): add bootstrap step to clear out unknown aspect rows from the database by @RyanHolstien in https://github.com/datahub-project/datahub/pull/5148
• feat(ingest): adds csv enricher ingestion source by @aditya-radhakrishnan in https://github.com/datahub-project/datahub/pull/5221
• fix(build): pin confluent kafka dependency by @anshbansal in https://github.com/datahub-project/datahub/pull/5224
• fix(ingest): databricks - ingest structs correctly through hive by @shirshanka in https://github.com/datahub-project/datahub/pull/5223
• feat(dbt): add sibling association logic to associate dbt elements with their target systems by @gabe-lyons in https://github.com/datahub-project/datahub/pull/5190
• feat(tableau): use pagination for all connection queries by @mayurinehate in https://github.com/datahub-project/datahub/pull/5204
• Handling 404 page not found by @Ankit-Keshari-Vituity in https://github.com/datahub-project/datahub/pull/5227
• refactor(UI): Refactor Dataset Health Status by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/5222
• fix(dbt-test): Inconsistency in assertions by @Santhin in https://github.com/datahub-project/datahub/pull/5214
• feat(ingest): remove need for sink block in UI based ingestion by @anshbansal in https://github.com/datahub-project/datahub/pull/5208
• fix(ingest): bigquery - Grouping date named tables at bigquery by @treff7es in https://github.com/datahub-project/datahub/pull/5230
• Add check for 0 rows when profiling datasets from s3 by @Jiafi in https://github.com/datahub-project/datahub/pull/5219
• [bug fix]: disabled create buttons by @xiphl in https://github.com/datahub-project/datahub/pull/5234
• fix(ingest): bigquery - Handling gracefully sql parser error in bq lineage by @treff7es in https://github.com/datahub-project/datahub/pull/5238
• fix(ingest): do not dump password by @anshbansal in https://github.com/datahub-project/datahub/pull/5235
• feat(ingest): dbt - improving dbt_meta mapping by @shirshanka in https://github.com/datahub-project/datahub/pull/5237
• fix(siblings): Force merged urn to viewed entity urn by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/5247
• refactor(docs): Move CLI docs to root level by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/5246

New Contributors

• @skrydal made their first contribution in https://github.com/datahub-project/datahub/pull/5129
• @sayakmaity made their first contribution in https://github.com/datahub-project/datahub/pull/5143
• @kangseonghyun made their first contribution in https://github.com/datahub-project/datahub/pull/5178
• @alexey-kravtsov made their first contribution in https://github.com/datahub-project/datahub/pull/5179
• @bda618 made their first contribution in https://github.com/datahub-project/datahub/pull/5180
• @BALyons made their first contribution in https://github.com/datahub-project/datahub/pull/5186
• @afghori made their first contribution in https://github.com/datahub-project/datahub/pull/5209
• @Santhin made their first contribution in https://github.com/datahub-project/datahub/pull/5214

Full Changelog: https://github.com/datahub-project/datahub/compare/v0.8.38...v0.8.39