Linkis-1.0.1 includes all of Project Linkis 1.0.1. This release mainly fixes performance bugs in high concurrency scenarios, optimizes the resource management module, and optimizes the Spark engine to support the use of FATE (Federated Machine Learning), and fixes multiple discovered bugs. Abbreviations: CGS: Computation Governance Services PES: Public Enhancement Services MGS: Microservice Governance Services

Enhancement

• Linkis-906 [Commons] HTTPClient supports setting contentType when uploading files.
• Linkis-897 [Commons] HttpClient supports Response whose content is empty.
• Linkis-877 [Commons] HDFS FileSystem object supports caching.
• Linkis-862 [Commons] Optimize ShutdownHook to be compatible with windows operating system.
• Linkis-889 [Orchestrator] ioClient supports isolating a separate Orchestrator.
• Linkis-880 [PES-JobHistory] JobHistory supports recording the original code of the task submitted by the user.
• Linkis-879 [PES-InstanceLabel] InstanceLabel module support to modify and remove instance label information.
• Linkis-864 [PES-ErrorCode] Optimize the initialization sql of the error code, remove the default value of the ID field.
• Linkis-872 [PES-Configuration] Management console configuration supports real-time modification of cached values.
• Linkis-844 [PES-Configuration] The management console supports setting memory unit, the default unit is GB.
• Linkis-869 [CG-Entrance] Optimize task submission record client IP for link analysis.
• Linkis-848 [CG-LimkisMnagager] The tenant label of ECM should be retained to facilitate the restart.
• Linkis-863 [EnginePlugin-Spark] Spark EngineConn startup parameters support the setting of native parameters to support the use of Fate, such as setting the python version.

Bugs Fix

• Linkis-900 [Commons] Logging is recommended to be optimized, because the user may pass in Null and thus cause an exception.
• Linkis-851 [Commons] The version of the jackson-core-asl package is incorrect after linkis1.0 is compiled with the CDH version.
• Linkis-896 [PES-UDF] UDFClient has a bug that can only get UDF but cannot get method functions.
• Linkis-873 [PES-BML] Optimize the execution order of the BML EngineConnHook to fix the bug of not being able to read resource files.
• Linkis-850 [PES-Configuration] Failed to save IDE-JDBC parameter configuration in Linkis1.0 management console.
• Linkis-847 [CG-Entrance] When a task contains thousands of SQL, the execution may not be correct.
• Linkis-842 [CG-Entrance] Task execution fails when the sql limit setting is greater than 5000.
• Linkis-841 [CG-Entrance] Log is not fully recorded in the stress test of 10,000 tasks.
• Linkis-840 [CG-Entrance] During the stress test of 10,000 tasks, the task status is in the Inited/Running/Scheduled status and no longer updated.
• Linkis-846 [CG-LinkisMnaager] When the parameter of the number of engine is set to 2 in management console, however, 18 engines is started when schedulis is scheduling.
• Linkis-845 [CG-LinkisMnaager] The value of concurrent label may be null in ECM management .
• Linkis-843 [EnginePlugin-Hive] When killing a complex hive task, Hive MR of Yarn is not simultaneously killed.

Credits

The release of Linkis 1.0.1 is inseparable from the contributors of the Linkis community. Thanks to all the community contributors!

Compiled Version

1. Tencent Cloud:

Linkis-1.0.1 Compiled (.tar.gz)