Introduction

Dinky is a data development platform based on Apache Flink, which enables agile data development and deployment.

Upgrade instructions

Dinky 1.0 is a refactored version that restructures existing functions, adds several enterprise-level functions, and fixes some limitations of 0.7. Currently, it is not possible to directly upgrade from 0.7 to 1.0. An upgrade plan will be provided in the future.

Function

Its main functions are as follows:

• FlinkSQL data development: automatic prompt completion, syntax highlighting, statement beautification, syntax verification, execution plan, MetaStore, lineage analysis, version comparison, etc.
• Support FlinkSQL multi-version development and multiple execution modes: Local, Standalone, Yarn/Kubernetes Session, Yarn Per-Job, Yarn/Kubernetes Application
• Support Apache Flink ecosystem: Connector, FlinkCDC, Paimon, etc.
• Support FlinkSQL syntax enhancement: whole database synchronization, execution environment, global variables, statement merging, table value aggregation function, loading dependencies, row-level permissions, Jar submission, etc.
• Support FlinkCDC real-time warehousing of the entire database into the lake: multi-database output, automatic table creation, model evolution, sub-database and sub-table
• Supports SQL job development and metadata browsing: ClickHouse, Doris, Hive, Mysql, Oracle, Phoenix, PostgreSql, Presto, SqlServer, StarRocks, etc.
• Support Flink real-time online debugging preview TableData, ChangeLog, Operator, Catalog
• Support Flink job custom monitoring statistical analysis and custom alarm rules.
• Support real-time task operation and maintenance: online and offline, job information (supports obtaining checkpoint), job log, version information, job snapshot, monitoring, SQL lineage, alarm record, etc.
• Support real-time job alarms and alarm groups: DingTalk, WeChat business account, Feishu, email, SMS, etc.
• Supports automatically hosted SavePoint/CheckPoint recovery and triggering mechanisms: latest, earliest, specified, etc.
• Supports multiple resource management: cluster instances, cluster configurations, data sources, alarms, documents, global variables, Git projects, UDFs, system configurations, etc.
• Support enterprise-level management: tenants, users, roles, menus, tokens, data permissions

New Feature

• Added new homepage signboard
• Data development supports code tips
• Supports real-time printing of Flink table data
• The console supports real-time printing task submission log
• Support Flink CDC 3.0 entire database synchronization
• Support customized alarm rules and customized alarm information templates
• Comprehensive revision of the operation and maintenance center
• k8s and k8s operator support
• Support proxy Flink webui access
• Support Flink task monitoring
• Support Dinky jvm monitoring
• New resource center function and expanded rs protocol
• New Git UDF/JAR project hosting and overall construction process
• Supports full-mode application mode custom jar submission
• openapi supports custom parameter submission
• Permission system upgrade, supporting tenants, roles, tokens, and menu permissions
• LDAP authentication support
• New widget function on data development page
• Support pushing dependent tasks to DolphinScheduler