v0.12.0 Release Highlights

User Experience

Nested Domains

Nested Domains are here! This provides flexibility in organizing your entities within Domains to match the unique organizational structure of your company.

DataHub Chrome Extension Improvements

The Acryl DataHub Chome extension now supports PowerBI! This is a super powerful way for your business users to gain DataHub-specific insights directly in the BI tools they use most. Additionally, we now support making edits back to DataHub Entities directly from the Chrome extension.

Access Management Tab for Datasets

Shoutout to @Ramendra761 from the PayPal Team for contributing a new Access Management tab in Dataset Entity pages! The aim of this feature is to enable users to view the required roles for accessing the Dataset, as defined by Roles and/or Policies in the organization’s Access Management System. It also introduces the ability to request access directly from the page.

Metadata Ingestion

Miscellaneous Improvements

• Sampling-Based Profiling: You can now configure sampling-based profiling to address query performance concerns in Snowflake and BigQuery
• Kafka Connect > Snowflake: We now support automatically defining lineage between the two platforms
• Athena: Support for complex and nested schemas

Column-Level Lineage

We are incubating CLL support for the following:

• Airflow plugin v2 now supports automatic extraction of CLL for certain operators, removing the need to annotate DAGs
• dbt
• Redshift
• PowerBI (support for Column-Level Lineage for M-Query)

Incubating Sources

• MLflow
• Teradata
• Unity Catalog Notebooks
• DynamoDB

Developer Experience

• Data Contracts: v0.12.0 introduces underlying models and CLI; UI support to follow
• We now support creating custom models without requiring a fork of the main DataHub project
• Updates to support OpenSearch 2.x and alternate Postgres db in postgres-setup

Other Notable Changes

• Session token configuration has changed, all previously created session tokens will be invalid and users will be prompted to log in. Expiration time has also been shortened which may result in more login prompts with the default settings. There should be no other interruption due to this change.

Breaking Changes

Find full details here

• #9044 - GraphQL APIs for adding ownership now expect either an ownershipTypeUrn referencing a customer ownership type or a (deprecated) type. Where before adding an ownership without a concrete type was allowed, this is no longer the case. For simplicity you can use the type parameter which will get translated to a custom ownership type internally if one exists for the type being added.
• #9010 - In Redshift source's config incremental_lineage is set default to off.
• #8810 - Removed support for SQLAlchemy 1.3.x. Only SQLAlchemy 1.4.x is supported now.
• #8942 - Removed urn:li:corpuser:datahub owner for the Measure, Dimension and Temporal tags emitted by Looker and LookML source connectors.
• #8853 - The Airflow plugin no longer supports Airflow 2.0.x or Python 3.7. See the docs for more details.
• #8853 - Introduced the Airflow plugin v2. If you're using Airflow 2.3+, the v2 plugin will be enabled by default, and so you'll need to switch your requirements to include pip install 'acryl-datahub-airflow-plugin[plugin-v2]'. To continue using the v1 plugin, set the DATAHUB_AIRFLOW_PLUGIN_USE_V1_PLUGIN environment variable to true.
• #8943 - The Unity Catalog ingestion source has a new option include_metastore, which will cause all urns to be changed when disabled. This is currently enabled by default to preserve compatibility, but will be disabled by default and then removed in the future. If stateful ingestion is enabled, simply setting include_metastore: false will perform all required cleanup. Otherwise, we recommend soft deleting all databricks data via the DataHub CLI: datahub delete --platform databricks --soft and then reingesting with include_metastore: false.
• #8846 - Changed enum values in resource filters used by policies. RESOURCE_TYPE became TYPE and RESOURCE_URN became URN. Any existing policies using these filters (i.e. defined for particular urns or types such as dataset) need to be upgraded manually, for example by retrieving their respective dataHubPolicyInfo aspect and changing part using filter i.e.

   "resources": {
     "filter": {
       "criteria": [
         {
           "field": "RESOURCE_TYPE",
           "condition": "EQUALS",
           "values": [
             "dataset"
           ]
         }
       ]
     }

into

   "resources": {
     "filter": {
       "criteria": [
         {
           "field": "TYPE",
           "condition": "EQUALS",
           "values": [
             "dataset"
           ]
         }
       ]
     }

for example, using datahub put command. Policies can also be removed and re-created via UI.

• #9077 - The BigQuery ingestion source by default sets match_fully_qualified_names: true. This means that any dataset_pattern or schema_pattern specified will be matched on the fully qualified dataset name, i.e. <project_name>.<dataset_name>. We attempt to support the old pattern format by prepending .*\\. to dataset patterns lacking a period, so in most cases this should not cause any issues. However, if you have a complex dataset pattern, we recommend you manually convert it to the fully qualified format to avoid any potential issues.

What's Changed

• feat(UI): AccessManagement UI to access the role metadata for a dataset by @Ramendra761 in https://github.com/datahub-project/datahub/pull/8541
• Glossary Navigation Cypress test by @kkorchak in https://github.com/datahub-project/datahub/pull/8804
• ci: upgrade python to 3.10 for builds by @hsheth2 in https://github.com/datahub-project/datahub/pull/8808
• feat(ingestion/looker): Add view file-path as option in view_naming_pattern config by @siddiquebagwan-gslab in https://github.com/datahub-project/datahub/pull/8713
• feat(upgrade): add ability to provide a startingOffset for RestoreIndices by @ukayani in https://github.com/datahub-project/datahub/pull/8539
• fix(index): Do not override the search analyzer for ngram fields by @iprentic in https://github.com/datahub-project/datahub/pull/8818
• test(managed_ingestion): fix managed ingestion test by fixing actions… by @david-leifker in https://github.com/datahub-project/datahub/pull/8820
• docs: add 0.11 docs to docs site by @hsheth2 in https://github.com/datahub-project/datahub/pull/8813
• docs(release): Update updating-datahub.md for 0.11.0 release by @iprentic in https://github.com/datahub-project/datahub/pull/8821
• fix(ingest/mssql): Add UNIQUEIDENTIFIER data type as String by @cjm98332 in https://github.com/datahub-project/datahub/pull/8642
• build(ingest): upgrade to sqlalchemy 1.4, drop 1.3 support by @mayurinehate in https://github.com/datahub-project/datahub/pull/8810
• fix(ingest): use epoch 1 for dev build versions by @hsheth2 in https://github.com/datahub-project/datahub/pull/8824
• ci: make wheel builds more robust by @hsheth2 in https://github.com/datahub-project/datahub/pull/8815
• feat(cli): fix upload ingest cli endpoint by @pedro93 in https://github.com/datahub-project/datahub/pull/8826
• docs(transformer): fix names in sample code of 'pattern_add_dataset_domain' by @Starkie in https://github.com/datahub-project/datahub/pull/8755
• fix(siblingsHook): check number of dbtUpstreams instead of all upStreams by @ethan-cartwright in https://github.com/datahub-project/datahub/pull/8817
• fix(java) Update DataProductMapper to always return a name by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/8832
• build(ingest): Bump jsonschema for Python >= 3.8 by @asikowitz in https://github.com/datahub-project/datahub/pull/8836
• feat(ingest/rest-emitter): Do not raise error on retry failure to get better error messages by @asikowitz in https://github.com/datahub-project/datahub/pull/8837
• ci: add markdown-link-check by @yoonhyejin in https://github.com/datahub-project/datahub/pull/8771
• docs(managed datahub): release notes 0.2.11 by @anshbansal in https://github.com/datahub-project/datahub/pull/8830
• build(ingest): Remove constraint on jsonschema for Python >= 3.8 by @asikowitz in https://github.com/datahub-project/datahub/pull/8842
• fix(build): clean task cleanup generated src by @anshbansal in https://github.com/datahub-project/datahub/pull/8844
• feat(ci): disable ingestion smoke build by @anshbansal in https://github.com/datahub-project/datahub/pull/8845
• fix: fix quickstart page by @yoonhyejin in https://github.com/datahub-project/datahub/pull/8784
• feat(bigquery): add better timers around every API call by @mayurinehate in https://github.com/datahub-project/datahub/pull/8626
• feat(ingestion/dynamodb): Add DynamoDB as new metadata ingestion source by @TonyOuyangGit in https://github.com/datahub-project/datahub/pull/8768
• feat(ingest/bigquery): support bigquery profiling with sampling by @mayurinehate in https://github.com/datahub-project/datahub/pull/8794
• Fix for edit_documentation and glossary_navigation cypress tests by @kkorchak in https://github.com/datahub-project/datahub/pull/8838
• feat(ui/java) Update domains to be nested by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/8841
• dcs(ml-models): enhancing ml model documentation by @gabe-lyons in https://github.com/datahub-project/datahub/pull/8848
• logging(lineage): adding some lineage explorer and impact analysis logging by @gabe-lyons in https://github.com/datahub-project/datahub/pull/8849
• fix(gms): lower telemetry error log level by @hsheth2 in https://github.com/datahub-project/datahub/pull/8860
• fix(datahub-gms) usage stats queryRange API's Authorization error for Dataset Owners by @siladitya2 in https://github.com/datahub-project/datahub/pull/8819
• docs(observability): Add Custom Assertion user guide by @zmcnellis in https://github.com/datahub-project/datahub/pull/8854
• fix(airflow): fix provider loading exception by @hsheth2 in https://github.com/datahub-project/datahub/pull/8861
• Fix glossary_navigation.js by @kkorchak in https://github.com/datahub-project/datahub/pull/8864
• Managing Secrets Cypress test by @kkorchak in https://github.com/datahub-project/datahub/pull/8863
• feat(ui) Make certain things disabled if read only mode is enabled by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/8870
• fix(ingest): fix mode lint error by @mayurinehate in https://github.com/datahub-project/datahub/pull/8875
• feat(search): update to support OpenSearch 2.x by @david-leifker in https://github.com/datahub-project/datahub/pull/8852
• docs(observability): Custom Assertion user guide updates by @zmcnellis in https://github.com/datahub-project/datahub/pull/8878
• feat(ingest): bump acryl-sqlglot by @hsheth2 in https://github.com/datahub-project/datahub/pull/8882
• feat(ingest): bulk fetch schema info for schema resolver by @mayurinehate in https://github.com/datahub-project/datahub/pull/8865
• fix(docs): remove link-checker from CI by @hsheth2 in https://github.com/datahub-project/datahub/pull/8883
• feat(entity-client): enable client side cache for entity-client and usage-client by @david-leifker in https://github.com/datahub-project/datahub/pull/8877
• docs: add homepage ctas by @jeffmerrick in https://github.com/datahub-project/datahub/pull/8866
• fix(ingest/bigquery): show report in output by @hsheth2 in https://github.com/datahub-project/datahub/pull/8867
• fix(docker): support alternate postgres db in postgres-setup by @hsheth2 in https://github.com/datahub-project/datahub/pull/8800
• feat(python): support custom models without forking by @hsheth2 in https://github.com/datahub-project/datahub/pull/8774
• fix(docs): fixes link to developers guides by @sgomezvillamor in https://github.com/datahub-project/datahub/pull/8809
• docs(authorization): correct policies example by @siladitya2 in https://github.com/datahub-project/datahub/pull/8833
• fix(report): too long report causes MSG_SIZE_TOO_LARGE in kafka by @sgomezvillamor in https://github.com/datahub-project/datahub/pull/8857
• docs(ingest/lookml): add guide on debugging lkml parse errors by @hsheth2 in https://github.com/datahub-project/datahub/pull/8890
• feat(ingest/kafka): support metadata mapping from kafka avro schemas by @mayurinehate in https://github.com/datahub-project/datahub/pull/8825
• feat(ingest/kafka-connect): Lineage for Kafka Connect > Snowflake by @shubhamjagtap639 in https://github.com/datahub-project/datahub/pull/8811
• fix(test): fix test execution by @david-leifker in https://github.com/datahub-project/datahub/pull/8889
• feat(ingest/snowflake): allow shares config without platform instance by @mayurinehate in https://github.com/datahub-project/datahub/pull/8803
• fix(ingest): bound types-requests by @hsheth2 in https://github.com/datahub-project/datahub/pull/8895
• fix(build): run codegen when building datahub-ingestion image by @hsheth2 in https://github.com/datahub-project/datahub/pull/8869
• fix(ingest/s3): Converting windows style path to posix one on local fs by @treff7es in https://github.com/datahub-project/datahub/pull/8757
• fix(docs): Rebranding custom to custom SQL by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/8896
• docs(observability): Freshness Assertion Operation Types by @zmcnellis in https://github.com/datahub-project/datahub/pull/8907
• doc(ingestion): looker & lookml ingestion guide by @siddiquebagwan in https://github.com/datahub-project/datahub/pull/8006
• fix(ingest): bump typing-extensions by @hsheth2 in https://github.com/datahub-project/datahub/pull/8897
• feat(metadata-ingestion): implement mlflow source by @hariishaa in https://github.com/datahub-project/datahub/pull/7971
• feat(docs): Update ownership-types image urls by @pedro93 in https://github.com/datahub-project/datahub/pull/8905
• docs(website): style tweaks for readability and more open spacing by @jeffmerrick in https://github.com/datahub-project/datahub/pull/8876
• build(ingest/databricks): Relax databricks-sdk pin by @asikowitz in https://github.com/datahub-project/datahub/pull/8855
• test(ingest/delta-lake): Fix minio test for new version of delta-lake by @asikowitz in https://github.com/datahub-project/datahub/pull/8914
• doc: fix title of the ui ingestion guide & remove browse.md by @yoonhyejin in https://github.com/datahub-project/datahub/pull/8916
• refactor(ingest/bigquery): Clarify table / view queries by @asikowitz in https://github.com/datahub-project/datahub/pull/8913
• refactor(ingest/graph): Factor out filter logic by @asikowitz in https://github.com/datahub-project/datahub/pull/8888
• fix(docker): move base image to -base tag, full image to head by @david-leifker in https://github.com/datahub-project/datahub/pull/8919
• fix(docker): slim tags by @david-leifker in https://github.com/datahub-project/datahub/pull/8922
• ci: Docker slim tag fix by @david-leifker in https://github.com/datahub-project/datahub/pull/8925
• refactor(misc): testngJava fix, systemrestli client, cache key fix, e… by @david-leifker in https://github.com/datahub-project/datahub/pull/8926
• feat(openapi): openapi v2 updates by @david-leifker in https://github.com/datahub-project/datahub/pull/8927
• fix(data-product): show data product card on home page by @Endtry in https://github.com/datahub-project/datahub/pull/8924
• fix(graphql): support additional types in scrollAcrossEntities by @hsheth2 in https://github.com/datahub-project/datahub/pull/8891
• docs: update cta links for acryl by @hsheth2 in https://github.com/datahub-project/datahub/pull/8908
• feat(docs): Corrects release version for custom ownership types. by @pedro93 in https://github.com/datahub-project/datahub/pull/8847
• docs: fix typo in impact-analysis.md by @Erik-McKelvey in https://github.com/datahub-project/datahub/pull/8915
• feat(chrom-ext-editable): set readOnly to false so that side navigati… by @Endtry in https://github.com/datahub-project/datahub/pull/8930
• fix(client): use value for RelationshipDirection by @eboneil in https://github.com/datahub-project/datahub/pull/8912
• fix(fine-grained lineage) CLL for datajob downstreams by @eboneil in https://github.com/datahub-project/datahub/pull/8937
• fix(ingest): refactor test markers + fix disk space issues in CI by @hsheth2 in https://github.com/datahub-project/datahub/pull/8938
• fix(cli): make quickstart docker compose up command more robust by @hsheth2 in https://github.com/datahub-project/datahub/pull/8929
• feat(transfomer): add transformer to get ownership from tags by @anshbansal in https://github.com/datahub-project/datahub/pull/8748
• docs(lineage): Lineage docs refactoring by @yoonhyejin in https://github.com/datahub-project/datahub/pull/8899
• feat(ingestion/powerbi): column level lineage extraction for M-Query by @siddiquebagwan-gslab in https://github.com/datahub-project/datahub/pull/8796
• feat(ingest/airflow): airflow plugin v2 by @hsheth2 in https://github.com/datahub-project/datahub/pull/8853
• feat(ingest/snowflake): initialize schema resolver from datahub for l… by @mayurinehate in https://github.com/datahub-project/datahub/pull/8903
• feat(bigquery): excluding projects without any datasets from ingestion by @upendrao in https://github.com/datahub-project/datahub/pull/8535
• feat(ingest/unity): Ingest notebooks and their lineage by @asikowitz in https://github.com/datahub-project/datahub/pull/8940
• test(ingest/unity): Add Unity Catalog memory performance testing by @asikowitz in https://github.com/datahub-project/datahub/pull/8932
• doc: DataHubUpgradeHistory_v1 by @sgomezvillamor in https://github.com/datahub-project/datahub/pull/8918
• fix: fix typo on aws guide by @yoonhyejin in https://github.com/datahub-project/datahub/pull/8944
• feat(dbt-ingestion): add documentation link from dbt source to institutionalMemory by @ethan-cartwright in https://github.com/datahub-project/datahub/pull/8686
• refactor(style): Improve search bar input focus + styling by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/8955
• feat: data contracts models + CLI by @hsheth2 in https://github.com/datahub-project/datahub/pull/8923
• ci: tweak ci runs to decrease wait time of devs by @anshbansal in https://github.com/datahub-project/datahub/pull/8945
• docs(ingest): add permissions required for athena ingestion by @mayurinehate in https://github.com/datahub-project/datahub/pull/8948
• feat(ingestion/dynamodb): implement pagination for list_tables by @jinlintt in https://github.com/datahub-project/datahub/pull/8910
• feat(ci): enable ci to run on PR-s targeting all branches by @shirshanka in https://github.com/datahub-project/datahub/pull/8933
• feat(ingest/dbt): support use_compiled_code and test_warnings_are_errors by @hsheth2 in https://github.com/datahub-project/datahub/pull/8956
• refactor(boot): increases wait timeout for servlets initialization by @PatrickfBraz in https://github.com/datahub-project/datahub/pull/8947
• fix(ingest/unity): Remove metastore from ingestion and urns; standardize platform instance; add notebook filter by @asikowitz in https://github.com/datahub-project/datahub/pull/8943
• fix: add retry for fetch_url by @yoonhyejin in https://github.com/datahub-project/datahub/pull/8958
• feat(ingest/unity): Use ThreadPoolExecutor for CLL by @asikowitz in https://github.com/datahub-project/datahub/pull/8952
• feat(ingest/snowflake): support profiling with sampling by @mayurinehate in https://github.com/datahub-project/datahub/pull/8902
• Manage Access Tokens Cypress test by @kkorchak in https://github.com/datahub-project/datahub/pull/8936
• Nested domains cypress test by @kkorchak in https://github.com/datahub-project/datahub/pull/8879
• feat(models/assertion): Add SQL Assertions by @asikowitz in https://github.com/datahub-project/datahub/pull/8969
• feat(ingest): incremental lineage source helper by @mayurinehate in https://github.com/datahub-project/datahub/pull/8941
• feat(ingest): refactor + simplify incremental lineage helper by @mayurinehate in https://github.com/datahub-project/datahub/pull/8976
• fix(lint): run black, isort by @anshbansal in https://github.com/datahub-project/datahub/pull/8978
• fix(setup): drop older table if exists by @anshbansal in https://github.com/datahub-project/datahub/pull/8979
• feat(ingest/tableau): Allow parsing of database name from fullName by @asikowitz in https://github.com/datahub-project/datahub/pull/8981
• feat(auth): add data platform instance field resolver provider by @amanda-her in https://github.com/datahub-project/datahub/pull/8828
• feat(graphql): Added datafetcher for DataPlatformInstance entity by @siladitya2 in https://github.com/datahub-project/datahub/pull/8935
• feat(config): configurable bootstrap policies file by @sgomezvillamor in https://github.com/datahub-project/datahub/pull/8812
• feat(ingestion/redshift): CLL support in redshift by @siddiquebagwan-gslab in https://github.com/datahub-project/datahub/pull/8921
• fix(ingest): Fix postgres lineage within views by @harsha-mandadi-4026 in https://github.com/datahub-project/datahub/pull/8906
• refactor(ingest/dbt): move dbt tests logic to dedicated file by @hsheth2 in https://github.com/datahub-project/datahub/pull/8984
• fix(ingest/snowflake): fix sample fraction for very large tables by @mayurinehate in https://github.com/datahub-project/datahub/pull/8988
• fix: Display generic not found page for corp groups that do not exist by @jayasimhankv in https://github.com/datahub-project/datahub/pull/8880
• fix(ingest/looker): stop emitting tag owner by @sgomezvillamor in https://github.com/datahub-project/datahub/pull/8942
• feat(ingest): add output schema inference for sql parser by @hsheth2 in https://github.com/datahub-project/datahub/pull/8989
• fix(ingest/bigquery): Fix shard regexp to match without underscore as well by @treff7es in https://github.com/datahub-project/datahub/pull/8934
• feat(ingestion): Adding config option to auto lowercase dataset urns by @treff7es in https://github.com/datahub-project/datahub/pull/8928
• feat(ingest/s3): support .gzip and fix decompression bug by @hsheth2 in https://github.com/datahub-project/datahub/pull/8990
• feat(ingestion): Adds support for memory profiling by @pedro93 in https://github.com/datahub-project/datahub/pull/8856
• feat(auth): add group membership field resolver provider by @amanda-her in https://github.com/datahub-project/datahub/pull/8846
• Query plus filter search test by @kkorchak in https://github.com/datahub-project/datahub/pull/8993
• feat(ingest/teradata): Teradata source by @treff7es in https://github.com/datahub-project/datahub/pull/8977
• ci(ingest): update base requirements by @anshbansal in https://github.com/datahub-project/datahub/pull/8995
• docs(Acryl DataHub): release notes for 0.2.12 by @anshbansal in https://github.com/datahub-project/datahub/pull/9006
• feat(cli/datacontract): Add data quality assertion support by @asikowitz in https://github.com/datahub-project/datahub/pull/8968
• feat(ingest/teradata): view parsing by @treff7es in https://github.com/datahub-project/datahub/pull/9005
• Adding missing sqlparser libs to setup.py by @treff7es in https://github.com/datahub-project/datahub/pull/9015
• feat(graphql): support filtering based on greater than/less than criteria by @iprentic in https://github.com/datahub-project/datahub/pull/9001
• build(ingest): remove ratelimiter dependency by @mayurinehate in https://github.com/datahub-project/datahub/pull/9008
• build(ingest/redshift): Add sqlglot dependency by @asikowitz in https://github.com/datahub-project/datahub/pull/9021
• feat(ingest/teradata): Add option to not use file backed dict for view definitions by @asikowitz in https://github.com/datahub-project/datahub/pull/9024
• feat(ingest/unity-catalog): Support external S3 lineage by @asikowitz in https://github.com/datahub-project/datahub/pull/9025
• fix(ingest) - Fix file backed collection temp directory removal by @treff7es in https://github.com/datahub-project/datahub/pull/9027
• add dependency level to scrollAcrossLineage search results by @ethan-cartwright in https://github.com/datahub-project/datahub/pull/9016
• add create dataproduct example by @ethan-cartwright in https://github.com/datahub-project/datahub/pull/9009
• Download Lineage Results Cypress Test by @kkorchak in https://github.com/datahub-project/datahub/pull/9017
• fix(ingest/bigquery): Remove table name restrictions (allow $ and @) by @asikowitz in https://github.com/datahub-project/datahub/pull/9030
• chore(docker): update base images to alpine 3.18 by @RyanHolstien in https://github.com/datahub-project/datahub/pull/8967
• fix(frontend): update cookie module by @RyanHolstien in https://github.com/datahub-project/datahub/pull/8862
• docs(datahub-lite): Fix recipe by @asikowitz in https://github.com/datahub-project/datahub/pull/9023
• fix(ingest): fix typo in parsing list of groups by @mayurinehate in https://github.com/datahub-project/datahub/pull/9037
• feat(ingestion/Vertica): Fixed vertica integration test Updated vertica dialect by @vishalkSimplify in https://github.com/datahub-project/datahub/pull/9011
• fix(ingest/sqlalchemy): Fix URL parsing when sqlalchemy_uri provided by @asikowitz in https://github.com/datahub-project/datahub/pull/9032
• feature(ingest/athena): introduce support for complex and nested schemas in Athena by @bossenti in https://github.com/datahub-project/datahub/pull/8137
• docs: adding documentation for deployment of DataHub on Azure by @Saketh-Mahesh in https://github.com/datahub-project/datahub/pull/8612
• feat(frontend/ingestion): Support flagged / warning / connection failure statuses; add recipe by @asikowitz in https://github.com/datahub-project/datahub/pull/8920
• feat(avro): upgrade avro to 1.11 by @RyanHolstien in https://github.com/datahub-project/datahub/pull/9031
• fix(search): Detect field type for use in defining the sort order by @iprentic in https://github.com/datahub-project/datahub/pull/8992
• fix(api): Add preceding / to get index sizes path by @iprentic in https://github.com/datahub-project/datahub/pull/9043
• fix(search): Apply SearchFlags passed in through to scroll queries by @iprentic in https://github.com/datahub-project/datahub/pull/9041
• fix(ownership): Corrects validation of ownership type and makes it consistent across graphQL calls by @pedro93 in https://github.com/datahub-project/datahub/pull/9044
• docs(protobuf) Update messaging around nesting messages by @eboneil in https://github.com/datahub-project/datahub/pull/9048
• Use data-testids for glossary_navigation and dataset_ownership tests by @kkorchak in https://github.com/datahub-project/datahub/pull/9033
• test(ingest/delta-lake): Fix integration tests by @asikowitz in https://github.com/datahub-project/datahub/pull/9056
• Ingestion source creation cypress test by @kkorchak in https://github.com/datahub-project/datahub/pull/8850
• docs: fix lineage capability annotations by @hsheth2 in https://github.com/datahub-project/datahub/pull/8954
• Added more data-testid usage for edit_documentation and managing_secr… by @kkorchak in https://github.com/datahub-project/datahub/pull/9060
• fix(search): fix mapping builder bug by @david-leifker in https://github.com/datahub-project/datahub/pull/9062
• feat(ingestion): Adds more advanced configurations for runtime debugging by @pedro93 in https://github.com/datahub-project/datahub/pull/8998
• feat(ingest/s3): S3 add partition to schema by @treff7es in https://github.com/datahub-project/datahub/pull/8900
• feat(frontend): Remove debug flag from start script by @pedro93 in https://github.com/datahub-project/datahub/pull/9075
• feat(sqlparser): parse create DDL statements by @hsheth2 in https://github.com/datahub-project/datahub/pull/9002
• docs(ingest): update to get_workunits_internal by @eboneil in https://github.com/datahub-project/datahub/pull/9054
• Column level lineage and path test by @kkorchak in https://github.com/datahub-project/datahub/pull/8822
• refactor(ingest): Move sqlalchemy import out of sql_types.py by @asikowitz in https://github.com/datahub-project/datahub/pull/9065
• fix(ingest): add releases link by @hsheth2 in https://github.com/datahub-project/datahub/pull/9014
• fix(ingest/bigquery): Correctly apply table pattern to read events; fix end time calculation; deprecate match_fully_qualified_names by @asikowitz in https://github.com/datahub-project/datahub/pull/9077
• feat(sqlparser): extract CLL from updates by @hsheth2 in https://github.com/datahub-project/datahub/pull/9078
• fix(ui): Fixes handling of resources filters in UI by @skrydal in https://github.com/datahub-project/datahub/pull/9087
• docs(ingest/bigquery): Add docs for breaking change: match_fully_qualified_names by @asikowitz in https://github.com/datahub-project/datahub/pull/9094
• docs(update): Added info on breaking change for policies by @skrydal in https://github.com/datahub-project/datahub/pull/9093
• docs: add luckyorange script to head by @yoonhyejin in https://github.com/datahub-project/datahub/pull/9080
• design: refactor docs navbar by @yoonhyejin in https://github.com/datahub-project/datahub/pull/8975
• fix(ingest): update athena type mapping by @hsheth2 in https://github.com/datahub-project/datahub/pull/9061
• feat(ingest/datahub-source): Allow ingesting aspects from the entitiesV2 API by @asikowitz in https://github.com/datahub-project/datahub/pull/9089
• feat(ingestion/redshift): support auto_incremental_lineage by @siddiquebagwan-gslab in https://github.com/datahub-project/datahub/pull/9010
• feat(auth): Add backwards compatible field resolver by @pedro93 in https://github.com/datahub-project/datahub/pull/9096
• build(gradle): Support IntelliJ 2023.2.3 by @asikowitz in https://github.com/datahub-project/datahub/pull/9034
• build(ingest): Bump avro pin: security vulnerability by @asikowitz in https://github.com/datahub-project/datahub/pull/9042
• fix(ingestion/redshift): fix schema field data type mappings by @siddiquebagwan-gslab in https://github.com/datahub-project/datahub/pull/9053
• fix(datahub-protobuf): add check if nested field is reserved by @dyhn78 in https://github.com/datahub-project/datahub/pull/9058
• fix(ingest): better handling around sink errors by @hsheth2 in https://github.com/datahub-project/datahub/pull/9003
• feat(ingest/bigquery): Attempt to support raw dataset pattern by @asikowitz in https://github.com/datahub-project/datahub/pull/9109
• docs(observability): Column Assertion user guide by @zmcnellis in https://github.com/datahub-project/datahub/pull/9106

New Contributors

• @Ramendra761 made their first contribution in https://github.com/datahub-project/datahub/pull/8541
• @ukayani made their first contribution in https://github.com/datahub-project/datahub/pull/8539
• @cjm98332 made their first contribution in https://github.com/datahub-project/datahub/pull/8642
• @ethan-cartwright made their first contribution in https://github.com/datahub-project/datahub/pull/8817
• @hariishaa made their first contribution in https://github.com/datahub-project/datahub/pull/7971
• @Endtry made their first contribution in https://github.com/datahub-project/datahub/pull/8924
• @Erik-McKelvey made their first contribution in https://github.com/datahub-project/datahub/pull/8915
• @upendrao made their first contribution in https://github.com/datahub-project/datahub/pull/8535
• @jayasimhankv made their first contribution in https://github.com/datahub-project/datahub/pull/8880
• @Saketh-Mahesh made their first contribution in https://github.com/datahub-project/datahub/pull/8612
• @dyhn78 made their first contribution in https://github.com/datahub-project/datahub/pull/9058

Full Changelog: https://github.com/datahub-project/datahub/compare/v0.11.0...v0.12.0