v0.13.1

datahub-project/datahub

版本发布时间: 2024-04-03 03:40:00

datahub-project/datahub最新发布版本:v0.13.3(2024-05-24 07:11:13)

What's Changed

bump(kafka-setup): client version bump by @david-leifker in https://github.com/datahub-project/datahub/pull/9962
feat(ingest): throw codegen error on duplicate class names by @hsheth2 in https://github.com/datahub-project/datahub/pull/9960
feat(docker): respect pip mirrors with uv by @hsheth2 in https://github.com/datahub-project/datahub/pull/9963
Openlineage endpoint and Spark Lineage Beta Plugin by @treff7es in https://github.com/datahub-project/datahub/pull/9870
fix(ingest/json-schema): adding support descriptions for array by @AvaniSiddhapuraAPT in https://github.com/datahub-project/datahub/pull/9757
fix(ingest/redshift): fix bug in lineage v2 table renames by @hsheth2 in https://github.com/datahub-project/datahub/pull/9967
feat(ingest): speed up to_obj() and validate() by @hsheth2 in https://github.com/datahub-project/datahub/pull/9969
feat(ingest): fix fspath lint error by @hsheth2 in https://github.com/datahub-project/datahub/pull/9976
docs: archive old version before 0.12.0 & fix broken links by @yoonhyejin in https://github.com/datahub-project/datahub/pull/9957
fix(ui/markdown-editor): arrows change field when editing description… by @gaurav2733 in https://github.com/datahub-project/datahub/pull/9949
feat(ui/policies): add filter for Active/Inactive/All on policy page by @gaurav2733 in https://github.com/datahub-project/datahub/pull/9958
feat(ui): add option to add picture link for groups by @akarsh991 in https://github.com/datahub-project/datahub/pull/9882
feat(ingest): add Looks subtype + stop reemitting browsePathV2 by @hsheth2 in https://github.com/datahub-project/datahub/pull/9978
fix(ingest/bigquery): escape special characters for table descriptions by @AvaniSiddhapuraAPT in https://github.com/datahub-project/datahub/pull/9932
feat(ui): add loading spin to access management table by @filipe-caetano-ovo in https://github.com/datahub-project/datahub/pull/9974
fix(ingestion/fivetran): Fix fivetran get connector jobs bug by @shubhamjagtap639 in https://github.com/datahub-project/datahub/pull/9975
feat(ingest/dbt): generate CLL for all node types by @hsheth2 in https://github.com/datahub-project/datahub/pull/9964
chore(search): bump OpenSearch image version to 2.11.0 by @darnaut in https://github.com/datahub-project/datahub/pull/9984
feat(ingest): enable stateful_ingestion by default for DataHub rest sink by @shubhamjagtap639 in https://github.com/datahub-project/datahub/pull/9934
feat(ingestion/cli): Adding check option to validate allow/deny and path_specs by @treff7es in https://github.com/datahub-project/datahub/pull/9983
fix(ingest): only import PathSpec when necessary by @hsheth2 in https://github.com/datahub-project/datahub/pull/9989
feat(config): add configuration to reprocess UI sourced events by @RyanHolstien in https://github.com/datahub-project/datahub/pull/9988
feat(pluginRegistry): add configuration to reduce runnable frequency by @RyanHolstien in https://github.com/datahub-project/datahub/pull/9990
build(react): Fix typescript errors in test files by @sumitappt in https://github.com/datahub-project/datahub/pull/9982
feat(docs): disable last update timestamps by @hsheth2 in https://github.com/datahub-project/datahub/pull/9987
feat: add versioned content for 0.12.1 by @yoonhyejin in https://github.com/datahub-project/datahub/pull/9944
doc: add version 0.13.0 by @yoonhyejin in https://github.com/datahub-project/datahub/pull/9991
fix: fix mobile view and subtitles on slack/calendar page by @yoonhyejin in https://github.com/datahub-project/datahub/pull/9822
fix(ingest/redshift): fix stl scan lineage for lineage v2 by @hsheth2 in https://github.com/datahub-project/datahub/pull/9986
fix(ingest/delta-lake): support parsing nested types correctly by @dushayntAW in https://github.com/datahub-project/datahub/pull/9862
fix(test): nested domains by @david-leifker in https://github.com/datahub-project/datahub/pull/9993
fix(ci): refactor build-and-test command by @hsheth2 in https://github.com/datahub-project/datahub/pull/9999
feat(ingest/snowflake): generate query nodes for snowflake by @mayurinehate in https://github.com/datahub-project/datahub/pull/9966
fix(ingest/unity): creating group urn in case of group by @dushayntAW in https://github.com/datahub-project/datahub/pull/9951
fix(ui/left-side-bar): hide data products option in left side bar by @gaurav2733 in https://github.com/datahub-project/datahub/pull/10001
feat(ingest/redshift): make query generation configurable by @hsheth2 in https://github.com/datahub-project/datahub/pull/10000
fix(opensearch): Rollover usage events at a file size rather than time-based manner by @darnaut in https://github.com/datahub-project/datahub/pull/10006
chore(java): bump java dependency versions by @david-leifker in https://github.com/datahub-project/datahub/pull/10009
ci(react): Update package.json to enable lint check by @sumitappt in https://github.com/datahub-project/datahub/pull/10011
fix(ui/ingest): trim leading and trailing whitespaces from the text f… by @gaurav2733 in https://github.com/datahub-project/datahub/pull/10012
fix(policy-backfull): fix policy backfill job by @david-leifker in https://github.com/datahub-project/datahub/pull/10016
feat(opensearch): support for updating ISM policy used for usage events by @darnaut in https://github.com/datahub-project/datahub/pull/10018
refactor(react): Provide option to skip importing theme in CustomThemeProvider; rearrange toplevel components by @asikowitz in https://github.com/datahub-project/datahub/pull/9940
fix(openapi): fix openapi openlineage endpoint by @david-leifker in https://github.com/datahub-project/datahub/pull/10019
feat(ingest): update sqlglot fork by @hsheth2 in https://github.com/datahub-project/datahub/pull/10022
feat(ingest/superset): map awsathena platform name to athena by @LePuppy in https://github.com/datahub-project/datahub/pull/10005
fix(ingest/redshift): patch instead of replace redshift custom properties by @ethan-cartwright in https://github.com/datahub-project/datahub/pull/9293
fix(ingest/slack): tweak docs for slack source by @hsheth2 in https://github.com/datahub-project/datahub/pull/10007
fix(ingest): use contextvar for cooperative timeout by @hsheth2 in https://github.com/datahub-project/datahub/pull/10021
feat(ingest): improve custom package metadata by @hsheth2 in https://github.com/datahub-project/datahub/pull/9985
feat(docs): build website using swc-loader instead of babel by @hsheth2 in https://github.com/datahub-project/datahub/pull/9977
feat(ingest): add query formatting to sql aggregator by @hsheth2 in https://github.com/datahub-project/datahub/pull/10025
feat(ingest): add DataHubGraph.emit_all method by @hsheth2 in https://github.com/datahub-project/datahub/pull/10002
feat(ingestion): Support for Server-less Redshift by @skrydal in https://github.com/datahub-project/datahub/pull/9998
fix(ingest/teradata): small teradata improvements by @treff7es in https://github.com/datahub-project/datahub/pull/9953
feat(ingest): add classification for sql sources by @mayurinehate in https://github.com/datahub-project/datahub/pull/10013
docs(monitoring): add health check endpoint by @kopax-polyconseil in https://github.com/datahub-project/datahub/pull/10033
feat(ingest/dbt): capture both raw and compiled code by @hsheth2 in https://github.com/datahub-project/datahub/pull/10026
fix(ingest/redshift): Temp table lineage fix by @treff7es in https://github.com/datahub-project/datahub/pull/10008
feat(ingest): utilities for query logs by @hsheth2 in https://github.com/datahub-project/datahub/pull/10036
docs: add missing api sample docs by @yoonhyejin in https://github.com/datahub-project/datahub/pull/9869
feat(gms): add aspect name to siblings hook log by @hsheth2 in https://github.com/datahub-project/datahub/pull/10044
feat(ingest): add classification to bigquery, redshift by @mayurinehate in https://github.com/datahub-project/datahub/pull/10031
fix(ui/lineage): show data is too large error when limitation exceeds by @gaurav2733 in https://github.com/datahub-project/datahub/pull/10038
feat(ci): exempt more names from community by @mayurinehate in https://github.com/datahub-project/datahub/pull/10039
docs: improve versiondropdown design & set docs main to /features by @yoonhyejin in https://github.com/datahub-project/datahub/pull/9994
fix(ingest/redshift): tweak lineage v2 queries by @hsheth2 in https://github.com/datahub-project/datahub/pull/10045
chore(aws-msk-iam-auth): bump dependency version by @darnaut in https://github.com/datahub-project/datahub/pull/10063
feat(lineage): add priority to via node by @RyanHolstien in https://github.com/datahub-project/datahub/pull/10034
docs(acryl-cloud): notes for 0.2.16 by @anshbansal in https://github.com/datahub-project/datahub/pull/10069
fix(ingest/unity-catalog): generate sibling and lineage by @dushayntAW in https://github.com/datahub-project/datahub/pull/9894
fix(ingest): only auto-enable stateful ingestion if pipeline name is set by @hsheth2 in https://github.com/datahub-project/datahub/pull/10075
feat(ingest/s3): set default spark version by @hsheth2 in https://github.com/datahub-project/datahub/pull/10057
feat(ingest): better rest emitter error message by @hsheth2 in https://github.com/datahub-project/datahub/pull/10073
docs(sdk): Update API guide with example for Acryl by @gabe-lyons in https://github.com/datahub-project/datahub/pull/10072
feat(ingest): check for private import path usages by @hsheth2 in https://github.com/datahub-project/datahub/pull/10059
feat(ingest): add sql formatter utility by @hsheth2 in https://github.com/datahub-project/datahub/pull/10064
feat(ingest): refactor LineageConfig class by @hsheth2 in https://github.com/datahub-project/datahub/pull/10074
feat(ingest/dbt): point dbt assertions at dbt nodes by @hsheth2 in https://github.com/datahub-project/datahub/pull/10055
feat(dbt): show source and compiled code in the UI by @hsheth2 in https://github.com/datahub-project/datahub/pull/10028
feat(ui/ingest): ingestion form for Okta and AzureAD by @gaurav2733 in https://github.com/datahub-project/datahub/pull/9829
Update domains docs to include nested domains by @eboneil in https://github.com/datahub-project/datahub/pull/9890
fix(ingestion): Handle Redshift string length limit in Serverless mode by @skrydal in https://github.com/datahub-project/datahub/pull/10051
build(deps): bump follow-redirects from 1.15.4 to 1.15.6 in /docs-website by @dependabot in https://github.com/datahub-project/datahub/pull/10060
build(deps): bump es5-ext from 0.10.62 to 0.10.63 in /docs-website by @dependabot in https://github.com/datahub-project/datahub/pull/9927
fix(lineage): fix array out of bounds error by @david-leifker in https://github.com/datahub-project/datahub/pull/10081
Add owners, tags, glossary terms to dataset yaml loader by @eboneil in https://github.com/datahub-project/datahub/pull/9859
Add rate limiting to slack source by @eboneil in https://github.com/datahub-project/datahub/pull/10082
fix(metadata-ingestion)glue connector failure when Optional field Type of PartitionKey is absent for a Table by @siladitya2 in https://github.com/datahub-project/datahub/pull/10052
feat(redshift): adds flag to skip all external tables by @sgomezvillamor in https://github.com/datahub-project/datahub/pull/10040
feat(models) : Joins (Datasets) schema, resolvers and UI by @poorvi767 in https://github.com/datahub-project/datahub/pull/8325
feat(properties) Add upsertStructuredProperties graphql endpoint for assets by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/9906
Clean up logic for dataset.py yaml loader by @eboneil in https://github.com/datahub-project/datahub/pull/10089
feat(ingest/dbt): add option to skip sources by @hsheth2 in https://github.com/datahub-project/datahub/pull/10077
feat(ingest): support incremental column-level lineage by @hsheth2 in https://github.com/datahub-project/datahub/pull/10090
feat(ingest/powerbi): add chart subtypes by @hsheth2 in https://github.com/datahub-project/datahub/pull/10076
fix(ingest/metabase): Use connect_uri instead of display_uri to query Metabase API by @diegmonti in https://github.com/datahub-project/datahub/pull/9996
feat(tableau): ability to force extraction of table/column level linage from SQL queries by @alexs-101 in https://github.com/datahub-project/datahub/pull/9838
feat(ingest/datahub-gc): gc source to cleanup things by @anshbansal in https://github.com/datahub-project/datahub/pull/10085
docs(acryl-cloud): fix year in notes from 2023 to 2024 by @anshbansal in https://github.com/datahub-project/datahub/pull/10095
feeat(openapi): add batch endpoint to v2 using requestbody by @RyanHolstien in https://github.com/datahub-project/datahub/pull/10100
fix(ingest/dbt): fix config validator for skip_sources_in_lineage by @hsheth2 in https://github.com/datahub-project/datahub/pull/10098
docs: add gtm tag by @yoonhyejin in https://github.com/datahub-project/datahub/pull/10083
docs: add doc for assertions & data contracts by @yoonhyejin in https://github.com/datahub-project/datahub/pull/10029
test(ingest/mssql): use non-ephemeral mapping port by @hsheth2 in https://github.com/datahub-project/datahub/pull/10104
fix(ingestion/unity-catalog): patch owners and properties by @dushayntAW in https://github.com/datahub-project/datahub/pull/10086
fix(ingestion/transformer): added new transformer to cleanup suffix/prefix in owner URN by @dushayntAW in https://github.com/datahub-project/datahub/pull/10067
fix(ui/user-group): add non existent entity page for user by @gaurav2733 in https://github.com/datahub-project/datahub/pull/10004
fix(resolver): Allow users to add/remove related terms for children glossary terms by @pinakipb2 in https://github.com/datahub-project/datahub/pull/9895
Increase role member count in listRoles query to 20 from 10 by @jayasimhankv in https://github.com/datahub-project/datahub/pull/10020
fix(frontend): exclude plugins/frontend/auth/user.props config does not exist warnings from log by @Masterchen09 in https://github.com/datahub-project/datahub/pull/10043
fix(ui): show dataset display name in browse paths v2 by @Masterchen09 in https://github.com/datahub-project/datahub/pull/10054
fix(metrics): get fieldName for GraphQL Mutation queries by @trialiya in https://github.com/datahub-project/datahub/pull/9972
feat(UI): disable access management ui when no roles are linked to entity by @githendrik in https://github.com/datahub-project/datahub/pull/9610
ci(filters): add graphql code to backend trigger by @david-leifker in https://github.com/datahub-project/datahub/pull/10113
test(urn): add test case by @david-leifker in https://github.com/datahub-project/datahub/pull/10112
fix(ui) Add min width to the usage stats component by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/10056
log(system-update): Update DataHubStartupStep.java by @david-leifker in https://github.com/datahub-project/datahub/pull/9971
fix(usage-stats): usage-stats error handling and filter by @david-leifker in https://github.com/datahub-project/datahub/pull/10105
fix(elasticsearch logging): log how long bulk execution took by @darnaut in https://github.com/datahub-project/datahub/pull/10116
feat(auth): view authorization by @david-leifker in https://github.com/datahub-project/datahub/pull/10066
fix(searchContext): fix search flag immutability by @david-leifker in https://github.com/datahub-project/datahub/pull/10117
fix(ingest/looker): use external_base_url for explore url generation by @k7ragav in https://github.com/datahub-project/datahub/pull/10093
feat(ingest/dagster): Dagster source by @treff7es in https://github.com/datahub-project/datahub/pull/10071
fix(forms) Fix a couple of small inconsistencies with forms by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/9928
fix: exclude Elasticsearch ignore_throttled warnings from log by @Masterchen09 in https://github.com/datahub-project/datahub/pull/10042
Update build-and-test.yml by @david-leifker in https://github.com/datahub-project/datahub/pull/10127
fix(mae-consumer): fix aspect retriever injections mae-consumer by @david-leifker in https://github.com/datahub-project/datahub/pull/10125
fix(docs): fix docs build by @RyanHolstien in https://github.com/datahub-project/datahub/pull/10129
fix(search): respect the search flags term bucket size by @david-leifker in https://github.com/datahub-project/datahub/pull/10130
fix(ingestProposal): fix/handle no-op ingestion by @david-leifker in https://github.com/datahub-project/datahub/pull/10126
fix(ci): simplify python release process by @hsheth2 in https://github.com/datahub-project/datahub/pull/10133
feat(lineage): add a parameter to allow limiting the per hop exploration of lineage search by @RyanHolstien in https://github.com/datahub-project/datahub/pull/10062
feat(ingest/bigquery): Respect dataset and table patterns when ingesting lineage via catalog api by @ANich in https://github.com/datahub-project/datahub/pull/10080
feat(ingest): emit platform for query entities by @hsheth2 in https://github.com/datahub-project/datahub/pull/10103
feat(ingest): loosen pyarrow dep by @hsheth2 in https://github.com/datahub-project/datahub/pull/10141
fix(ingest/dbt): respect convert_column_urns_to_lowercase in mapping CLL by @hsheth2 in https://github.com/datahub-project/datahub/pull/10132
chore(ingestion-base): update base requirements by @david-leifker in https://github.com/datahub-project/datahub/pull/10142
feat(ingest/dbt): dbt model performance by @hsheth2 in https://github.com/datahub-project/datahub/pull/9992
fix(ingest/databricks): support hive metastore schemas with special char by @mayurinehate in https://github.com/datahub-project/datahub/pull/10049
feat(ui): sort partition keys to the top of the table for better visibility by @ngamanda in https://github.com/datahub-project/datahub/pull/9959
fix: OBS-729 | Filters: Fix alignment on nested dropdown by @sumitappt in https://github.com/datahub-project/datahub/pull/10140
feat(ingest/dynamodb): add support for classification by @mayurinehate in https://github.com/datahub-project/datahub/pull/10138
feat(incidents) incident resolution note more clearly displayed by @jayacryl in https://github.com/datahub-project/datahub/pull/10151
fix(entity-client): fix entity client cache and test by @david-leifker in https://github.com/datahub-project/datahub/pull/10149
chore(ingest): update doc & log detail by @HuanjieGuo in https://github.com/datahub-project/datahub/pull/10139
feat(ingest): loosen airflow plugin dependencies requirements by @hsheth2 in https://github.com/datahub-project/datahub/pull/10106
feat(ingest): fix validators by @hsheth2 in https://github.com/datahub-project/datahub/pull/10115
feat(ingest/bigquery): improve debug logs by @hsheth2 in https://github.com/datahub-project/datahub/pull/10101
fix(graphQL): Ignore soft-deleted assertions in UI calls by @pedro93 in https://github.com/datahub-project/datahub/pull/10148
fix(openapi): fix system-metadata response by @david-leifker in https://github.com/datahub-project/datahub/pull/10155
docs: update markprompt project key by @yoonhyejin in https://github.com/datahub-project/datahub/pull/10134
add row type for athena types by @rae89 in https://github.com/datahub-project/datahub/pull/10131
fix(setup): fix postgres setup to create temp table with no data by @trialiya in https://github.com/datahub-project/datahub/pull/10154
feat(ingest/looker): update browse paths to align with looker UI by @mayurinehate in https://github.com/datahub-project/datahub/pull/10147
feat(ingest/airflow): allow plugin to load on listener exception by @hsheth2 in https://github.com/datahub-project/datahub/pull/10152
feat(ingestion/bigquery): BigQuery Owner Label to Datahub Ownership by @shubhamjagtap639 in https://github.com/datahub-project/datahub/pull/10047
feat(ingest): bump sqlglot dep by @hsheth2 in https://github.com/datahub-project/datahub/pull/10144
docs(website): tweak eyebrow copy by @hsheth2 in https://github.com/datahub-project/datahub/pull/10143
docs: upgrade markprompt version by @yoonhyejin in https://github.com/datahub-project/datahub/pull/10159
fix(openapi): fix index out of bounds for sort order by @RyanHolstien in https://github.com/datahub-project/datahub/pull/10168
fix(search): fix field name in api by @RyanHolstien in https://github.com/datahub-project/datahub/pull/10170
build(docker): prefix pr on pr sha tags by @david-leifker in https://github.com/datahub-project/datahub/pull/10171
Revert docker helper changes by @david-leifker in https://github.com/datahub-project/datahub/pull/10172
feat(metadata-jobs): improve consumer logging by @darnaut in https://github.com/datahub-project/datahub/pull/10173
test(graph): refactor graph test by @david-leifker in https://github.com/datahub-project/datahub/pull/10175
fix(ingest/tableau) Fix Tableau lineage ingestion from Clickhouse by @valeral in https://github.com/datahub-project/datahub/pull/10167
[oracle ingestion]: get database name when using service by @Nelvin73 in https://github.com/datahub-project/datahub/pull/10158
fix(docker): fix versioning for compose file post release by @RyanHolstien in https://github.com/datahub-project/datahub/pull/10176
fix(restoreIndices): batchSize vs limit by @david-leifker in https://github.com/datahub-project/datahub/pull/10178
feat(ui): show classification in test connection by @hsheth2 in https://github.com/datahub-project/datahub/pull/10156
fix(ingest): add classification dep for dynamodb by @hsheth2 in https://github.com/datahub-project/datahub/pull/10162
feat(ingest/dbt): enable model performance and compiled code by default by @hsheth2 in https://github.com/datahub-project/datahub/pull/10164
refactor(docker): move to acryldata repo for all images by @david-leifker in https://github.com/datahub-project/datahub/pull/9459
fix(github): fix docker publish by @david-leifker in https://github.com/datahub-project/datahub/pull/10186
feat(lineage): mark nodes as explored by @RyanHolstien in https://github.com/datahub-project/datahub/pull/10180
feat(ingest/gc): add index truncation logic by @anshbansal in https://github.com/datahub-project/datahub/pull/10099
fix(entity-service): fix findFirst when already present by @david-leifker in https://github.com/datahub-project/datahub/pull/10187
fix(ingestion/salesforce): fixed the issue by escaping the markdown string by @dushayntAW in https://github.com/datahub-project/datahub/pull/10157

New Contributors

@AvaniSiddhapuraAPT made their first contribution in https://github.com/datahub-project/datahub/pull/9757
@akarsh991 made their first contribution in https://github.com/datahub-project/datahub/pull/9882
@filipe-caetano-ovo made their first contribution in https://github.com/datahub-project/datahub/pull/9974
@dushayntAW made their first contribution in https://github.com/datahub-project/datahub/pull/9862
@LePuppy made their first contribution in https://github.com/datahub-project/datahub/pull/10005
@kopax-polyconseil made their first contribution in https://github.com/datahub-project/datahub/pull/10033
@poorvi767 made their first contribution in https://github.com/datahub-project/datahub/pull/8325
@diegmonti made their first contribution in https://github.com/datahub-project/datahub/pull/9996
@alexs-101 made their first contribution in https://github.com/datahub-project/datahub/pull/9838
@pinakipb2 made their first contribution in https://github.com/datahub-project/datahub/pull/9895
@trialiya made their first contribution in https://github.com/datahub-project/datahub/pull/9972
@k7ragav made their first contribution in https://github.com/datahub-project/datahub/pull/10093
@jayacryl made their first contribution in https://github.com/datahub-project/datahub/pull/10151
@HuanjieGuo made their first contribution in https://github.com/datahub-project/datahub/pull/10139
@rae89 made their first contribution in https://github.com/datahub-project/datahub/pull/10131
@valeral made their first contribution in https://github.com/datahub-project/datahub/pull/10167
@Nelvin73 made their first contribution in https://github.com/datahub-project/datahub/pull/10158

Full Changelog: https://github.com/datahub-project/datahub/compare/v0.13.0...v0.13.1

相关地址：原始地址下载(tar) 下载(zip)

查看：2024-04-03发行的版本