Following the 0.129.0 release - the largest release to date, 0.130.0 continues to polish qsv as a data-wrangling engine, packing new features, fixes, and improvements, previewing upcoming features in qsv pro 1.0. Here are a few highlights:

Highlights

• Added .ssv (semicolon separated values) automatic support. Semicolon separated values are now automatically detected and supported by qsv. Though not as common as CSV, SSV is used in some regions and industries, so qsv now supports it.
• Added cargo deb compatibility. In preparation for the release of DataPusher+ 1.0, we're now making it easier to upgrade qsvdp so CKAN administrators can install and upgrade it easily using apt-get install qsvdp or apt-get upgrade qsvdp. DP+ is our next-gen, high-speed data ingestion tool for CKAN that uses qsv as its analysis engine. Its not only a robust, fast, validating data pump that guarantees high quality data, it also does extended analysis to infer and automatically derive high-quality metadata - what we call "automagical metadata".
• Upgraded to the latest Polars upstream at the py-polars-1.3.0 tag. Polars tops the TPC-H Benchmark and is several orders of magnitude faster than traditional dataframe libraries (cough - 🐼 pandas). qsv proudly rides the 🐻‍❄️ Polars bear to get subsecond response times even with very large datasets!
• qsv v0.130.0 shell completions files are available for download here. With shell completions, pressing tab in a compatible shell provides suggestions for various qsv commands, subcommands, and options that you can choose from. Supported shells include bash, zsh, powershell, fish, nushell, fig, and elvish. View tips on how to install completions for the bash shell here.

Added

• apply: add base62 encode/decode operations https://github.com/jqnatividad/qsv/pull/2013
• headers: add --just-count option https://github.com/jqnatividad/qsv/pull/2004
• json: add --select option https://github.com/jqnatividad/qsv/pull/1990
• searchset: add --not-one flag by @rzmk in https://github.com/jqnatividad/qsv/pull/1994
• Added .ssv (semicolon separated values) automatic support https://github.com/jqnatividad/qsv/pull/1987
• Added cargo deb compatibility by @tino097 in https://github.com/jqnatividad/qsv/pull/1991
• contrib(completions): add --just-count for headers by @rzmk in https://github.com/jqnatividad/qsv/pull/2006
• contrib(completions): add --select for json by @rzmk in https://github.com/jqnatividad/qsv/pull/1992
• added several benchmarks
• added more tests

Changed

• diff: allow selection of --key and --sort-columns by name, not just by index https://github.com/jqnatividad/qsv/pull/2010
• fetch & fetchpost: replace deprecated Redis execute command https://github.com/jqnatividad/qsv/commit/75cbe2b76426591e4658fdcb7d29287a40a7db36
• stats: more intelligent --infer-lenoption https://github.com/jqnatividad/qsv/commit/c6a0e641cd4c6ef87c070c8944f32a962a11c7e3
• validate: return delimiter detected upon successful CSV validation https://github.com/jqnatividad/qsv/pull/1977
• bump polars to latest upstream at py-polars-1.3.0 tag https://github.com/jqnatividad/qsv/pull/2009
• deps: bump csvs_convert from 0.8.12 to 0.8.13 https://github.com/jqnatividad/qsv/commit/d1d08009deb0579fd4d6fe305097e00e92da4191
• build(deps): bump cached from 0.52.0 to 0.53.0 by @dependabot in https://github.com/jqnatividad/qsv/pull/1983
• build(deps): bump cached from 0.53.0 to 0.53.1 by @dependabot in https://github.com/jqnatividad/qsv/pull/1986
• build(deps): bump postgres from 0.19.7 to 0.19.8 by @dependabot in https://github.com/jqnatividad/qsv/pull/1985
• build(deps): bump pyo3 from 0.22.1 to 0.22.2 by @dependabot in https://github.com/jqnatividad/qsv/pull/1979
• build(deps): bump redis from 0.25.4 to 0.26.0 by @dependabot in https://github.com/jqnatividad/qsv/pull/1995
• build(deps): bump serde_json from 1.0.120 to 1.0.121 by @dependabot in https://github.com/jqnatividad/qsv/pull/2011
• build(deps): bump simple-expand-tilde from 0.1.7 to 0.4.0 by @dependabot in https://github.com/jqnatividad/qsv/pull/1984
• build(deps): bump tokio from 1.38.0 to 1.38.1 by @dependabot in https://github.com/jqnatividad/qsv/pull/1973
• build(deps): bump tokio from 1.38.1 to 1.39.1 by @dependabot in https://github.com/jqnatividad/qsv/pull/1988
• build(deps): bump xxhash-rust from 0.8.11 to 0.8.12 by @dependabot in https://github.com/jqnatividad/qsv/pull/1997
• apply select clippy suggestions
• updated several indirect dependencies
• made various usage text improvements
• pin Rust nightly to 2024-07-26

Fixed

• diff: clarify --key usage examples, resolves #1998 by @rzmk in https://github.com/jqnatividad/qsv/pull/2001
• json: refactored so it didn't need to use threads to spawn qsv select to order the columns. Had to do this as sometimes intermediate output was sent to stdout before the final output was ready https://github.com/jqnatividad/qsv/commit/0f25deff98139b574dfd61c6e9bf58d36ea16618
• py: replace row with col in usage text by @allen-chin in https://github.com/jqnatividad/qsv/pull/2008
• reverse: fix indexed bug https://github.com/jqnatividad/qsv/pull/2007
• validate: properly auto-detect tab delimiter when file extension is TSV or TAB https://github.com/jqnatividad/qsv/pull/1975
• fix panic when process_input helper fn receives unexpected input from stdin https://github.com/jqnatividad/qsv/commit/152fec486c0e7b16242f3967930e9654ff2bdf3c

Removed

• docs: remove *nix only message for foreach by @rzmk in https://github.com/jqnatividad/qsv/pull/1972

New Contributors

• @tino097 made their first contribution in https://github.com/jqnatividad/qsv/pull/1991
• @allen-chin made their first contribution in https://github.com/jqnatividad/qsv/pull/2008

Full Changelog: https://github.com/jqnatividad/qsv/compare/0.129.1...0.130.0

To stay updated with datHere's latest news and updates (including qsv pro, datHere's CKAN DMS, and analyze.dathere.com), subscribe to the newsletter here: dathere.com/newsletter