This is our ~quarterly rollup release, containing many new features, bug fixes, and performance improvements. See the merged PRs for details. Interfaces are not guaranteed stable before 1.0.0.

This release includes dramatic changes to parsing and the query planner. Users running a test server are expected to be unaffected, but integrators building their own backend may need to make changes.

Merged PRs

go-mysql-server

• 1965: Add error test for ambiguous column name query close https://github.com/dolthub/dolt/issues/6395
• 1963: Fixed type hashing for Full-Text Fixes the reopened issue: https://github.com/dolthub/dolt/issues/6543 Not all types were covered by the default case.
• 1961: engine: Make ReadOnly a toggleable atomic.Bool.
• 1960: Fixed Full-Text NULL handling and ALTER TABLE case Fixes https://github.com/dolthub/dolt/issues/6540 and https://github.com/dolthub/dolt/issues/6541
• 1959: Bug fix: Prevent panic when reading non-existent user and system vars Fixes https://github.com/dolthub/dolt/issues/6546 We have enginetests that cover both of these queries, but because we don't run them with the full wire request/response processing code, we didn't catch these bugs during testing. Happy to add more tests here if there are suggestions, but I think the right way to test both of these is to get our existing test suite running over a SQL server connection, instead of just using the internal interfaces to the engine (i.e. https://github.com/dolthub/dolt/issues/3646), which Zach started on last week.
• 1958: Don't parse queries twice Local profile for oltp_point_select (query with smallest time spent in execution) is 5-15% speedup. Impact on queries with longer-runtime will be smaller, proportional to the fraction of time spent in analysis vs execution. results here: https://github.com/dolthub/dolt/pull/6547#issuecomment-1686795603
• 1957: Fixed relevancy ordering for Full-Text Fixes https://github.com/dolthub/dolt/issues/6530
• 1956: Removed unused AutoIncrementGetter interface Related to https://github.com/dolthub/dolt/issues/6543. The AutoIncrementGetter was moved from the editor to the table years ago, however the interface remained. I debated deleting this during my Full-Text implementation, however decided to leave it. Now, we've encountered an error with an integrator making use of the interface, so it has been removed here and the interface was moved into the integrator since it's an internal detail now.
• 1955: adding catalog table function interface
• 1954: Lateral join uses prepend row on RHS
• 1952: GMS tests have to resolve defaults Force GMS to resolve column defaults, fix bugs exposed by additional testing. Dolt enginetests pass locally.
• 1951: Fixed collation display in SHOW CREATE TABLE Originally, we didn't display the collation in SHOW CREATE TABLE when the collation was the default collation. Now, it doesn't display it if it's the same as the table collation, which mimics MySQL's behavior.
• 1950: Honor precision for datetime and timestamp, default to 0 (no fractional seconds)
• 1949: Revert "Merge pull request #1944 from dolthub/zachmu/timestamp" This reverts commit ca69015243946a0a42a0ec6a225da9c4551a0e12, reversing changes made to c11b504a18c48671c29c1494460f662e1f13f712.
• 1948: Name resolution refactor Accumulation of:
  • https://github.com/dolthub/go-mysql-server/pull/1927
  • https://github.com/dolthub/go-mysql-server/pull/1932
  • https://github.com/dolthub/go-mysql-server/pull/1936
  • https://github.com/dolthub/go-mysql-server/pull/1939
  • https://github.com/dolthub/go-mysql-server/pull/1943 Name resolution is interleaved with converting AST->plan nodes. Prepared statements use ParsedQuery and AST round tripping to analyze fresh query strings for every invoke. Various bug fixes.
• 1944: Changed datetime and timestamp types to honor precision and default to 0 digit precision This matches the MySQL behavior. Partial fix for https://github.com/dolthub/dolt/issues/6503
• 1942: Full-Text Fixes Pt. 3
• 1941: Bug fix for JSON_ARRAY function with binary arguments Binary args are now treated as character strings Also added a testing path to ScriptTests that let you inject Vitess bindvars to exercise more server logic Fixes https://github.com/dolthub/go-mysql-server/issues/1855
• 1940: Add SECURITY.md.
• 1938: Create interface for indexible tables in IndexedTableAccess Currently, only ResolvedTables are allowed to have indexes. There exists an interface, sql.IndexAddressable, which any node or table can implement in order to be a candidate for index-based optimization. But in practice, implementing that interface won't actually do anything because the IndexedTableAccess struct explicitly requires a ResolvedTable. This PR replaces the ResolvedTable field in IndexedTableAccess with a new interface tentatively called TableNode, although a more specific name would probably be better. In order for a node to be used for index-based optimization, it must implement this interface, and the table returned by the UnderlyingTable method must implement sql.IndexAddressable
• 1937: Remove do-nothing logic from pushdown.go This code is for an optimization that "pushes" filters deeper into the tree so that they're adjacent to the tables they modify. I've simplified the logic in two ways:
  • Removed handling of IndexedTableAccess nodes. This analysis pass used to run after these nodes were generated, but now runs before, so this code path will never be hit. Removing this logic makes it easier to make future changes to IndexedTableAccess
  • Removed the withTable subfunction, which never actually does anything, because the only table it will attempt to assign to a ResolvedTable is the table already on the node. This was likely leftover from a previous cleanup.
• 1935: Allow timestamps when encoding json As reported on discord, Nautobot, through DJango, puts time stamp data into a json object. This fails because:

  db> select JSON_OBJECT("a", Now());
  unsupported type: time.Time
  

  This change enables the encoding of a time stamp into a string.
• 1933: Fixed case sensitivity bugs in various statements
• 1930: leave aliases in projection We convert expression.Alias into expression.GetField with the name replaced in the top-level projection. When aliasing two different columns with the same name, we fail to properly distinguish them and end up rewriting the GetField indexes for both columns to be the same; this leads to incorrect results. A simple fix appears to be simply allowing the top-level projection to remain as an expression.Alias. This fix does not work for prepared statements in the old name resolution path. fix for: https://github.com/dolthub/dolt/issues/6455
• 1929: Add support for json_contains_path()
• 1928: Generated stored columns prototype Looking for feedback. The approach is a little bit wonky: it uses the same code paths as column defaults since they behave so similarly, but this has some weird consequences:
  • For generated columns, we fill in the column.DefaultValue field with the Generated expression
  • In various places, we now have to consider whether to use either column.DefaultValue, or column.Generated Overall I think I'm favorable on this approach, but keeping the two values more separate might be better for maintainability, not sure. Thoughts?
• 1926: Use EvaluateCondition for conditions in join statements Fixes https://github.com/dolthub/dolt/issues/6412 Use the given EvaluateCondition util function to determine whether a join condition is satisfied, which accounts for truthy integer values as mentioned in the linked issue.
• 1925: Fix load data check constraint indexing bug fix show tests
• 1924: Makes several new kinds of alter table statements involving auto_increment columns work correctly Also: makes several kinds of ALTER TABLE statements with multiple clauses more lenient in their error checking than MySQL. These statements will now succeed instead of being rejected. Fixes https://github.com/dolthub/dolt/issues/6218
• 1922: Render enum/set variables as strings after they are set Fixes https://github.com/dolthub/dolt/issues/6370 Ensure that when getting all session variables, if any of them is of an enum/set type, we convert them back to their string value. The SET command would amend the value in the session variables map to its numerical representation, which we need to translate back.
• 1921: Use a custom iterator when building HashJoin Lookups. Previously we were gathering all the child rows into a CachedResults node, and then iterating over the cached results to build the lookup. Cutting out the CachedResults and filling the lookup as we iterate simplifies the plan tree and also prevents a potentially large slice allocation. Benchmarking this showed no measurable impact on runtime, suggesting that the time taken populating/iterating over the cache was negligible. But the hash join used in the benchmark allocated 20% less memory.
• 1920: Allow AntiHashJoins to distinguish between empty and non-empty secondary tables. AntiHashJoin and LeftOuterHashExcludeNullsJoin should always evaluate at least one secondary row if the secondary table is nonempty. This guarentees the same behavior as MySQL for where x not in (...) expressions when x is nullable. Some joins (AntiJoins and some joins converted from AntiJoins) have an "excludes NULL" property, which means that if a filter condition on the join evaluates to NULL, the corresponding primary row can't be included in the result set. This means that if the primary row has a NULL value used in a filter, we need to know whether or not the secondary table has at least one row. Using a HashJoin makes that impossible without special handling. We had a test designed to catch this, but the test wasn't actually using a HashJoin, so this was missed. This PR also fixes the test to use a hash join.
• 1919: JSON_ARRAY correctly handles CHAR bindvar params The JSON_ARRAY function wasn't able to handle a []byte literal expression in a bind var. I also noticed that although our script tests can specify Bindings, they weren't getting used by our prepared query runners. I also fixed that, so that I could add a new test for this issue. Fixes: https://github.com/dolthub/go-mysql-server/issues/1855 Related Vitess change: https://github.com/dolthub/vitess/pull/261 Dolt CI Checks: https://github.com/dolthub/dolt/pull/6441
• 1917: More name res tests Fix JSON_TABLE and stored procedures on new name resolution path. edit: Added on duplicate update rewrite
• 1915: ANSI_QUOTES support for go-mysql-server Now that our parser has support for parsing in ANSI_QUOTES mode and treating double quotes as identifier quotes, this PR starts using that support when we parse queries. The main changes in this PR are:
  • a new SqlMode type to make it easier to load SQL_MODE and inspect enabled modes.
  • tracks SQL_MODE when a procedure/trigger/view is created.
  • passes ParserOptions to the parser, to enable parsing in ANSI_QUOTES mode.
  • connects the Vitess error logger to GMS' error log, so that errors in Vitess aren't swallowed (currently if a statement in a ComPrepare message doesn't parse, GMS doesn't log anything). Dolt has a similar, corresponding PR: https://github.com/dolthub/dolt/pull/6465 Related to: https://github.com/dolthub/dolt/issues/6305
• 1914: More Full-Text fixes More bugs found and fixed for FULLTEXT indexes. The ordering bug is fairly major, as it could require a drop/readd of the index, so I want to get this out now rather than waiting until I've gotten more tests/fixes in.
• 1913: Adjust range heap coster to ensure we don't choose it over a strong lookup join. This improves the performance for a client query by ensuring that we don't us a HeapRangeJoin when there's a good LookupJoin candidate, but we do use a HeapRangeJoin when there isn't. In larger queries, the time spent generating and evaluating candidate rows dominates. Using seqIOCostFactor feels like a bit of a misnomer, but cpuCostFactor is too low to reflect this behavior, and this is what costInnerJoin uses.
• 1912: Normalized handling of ALTER TABLE statements This PR normalizes the handling of all ALTER TABLE statements to use the same code path. Previously some used convertDDL instead. Also changes DDL processing to generate multiple plan nodes for statements that need them. Currently tested with add column .. unique, but the same approach should also work with auto_increment and other clauses. Relies on changes in https://github.com/dolthub/vitess/pull/260
• 1909: Show variables panic
• 1908: Some Full-Text fixes Adds a few additional tests, along with fixes. This is primarily for Sakila compatibility. More tests and fixes are forthcoming, and will arrive in a second PR.
• 1907: support joins on using syntax Relevant MySQL docs This PR adds an implementation for the USING clause during joins. It is mostly equivalent to creating a conjunction over equality statements over each column in an ON clause. Additionally, this properly supports NATURAL LEFT/RIGHT JOINs. So we now have
  • ... JOIN ... USING (...)
  • ... LEFT JOIN ... USING (...)
  • ... RIGHT JOIN ... USING (...)
  • ... NATURAL JOIN ...
  • ... NATURAL LEFT JOIN ...
  • ... NATURAL RIGHT JOIN ... fix for: https://github.com/dolthub/dolt/issues/5789
• 1906: Fix ErrBinaryCollation in queries with []byte argument Binding arguments of type []byte have been converted to string literals with the default collation (utf8mb4_0900_bin) by types.CreateStringWithDefaults(). I think types.CreateBinary() should be used, which generates literals with binary collation.
• 1905: When estimating the cardinality of an outer join, bound the cardinality below. For instance, left outer joins can never have a smaller cardinality than their left child. This should let us get more accurate estimates on the cardinality of outer joins and choose better plans.
• 1904: sql/mysql_db: Rework how we store and access table data for users, grants and replica_source_info. In general, we keep the same structure of storing the data in structs, keyed in multimaps, and having converters for going to and from sql.Rows. We change the following important things:
  1. We more explicitly model a multimap and an indexed set of entities with multiple keyers.
  2. We add read and write locking around edits and reads of the data table.
  3. We explicitly do not expose the raw indexed set or multimap from the MySQLDb instance itself. The unprincipled access of the various *Data instances in the old implementation was somewhat problematic.
• 1896: Stubbing out new Handler function to prep for supporting ANSI_QUOTES SQL mode PR https://github.com/dolthub/vitess/pull/256 adds support at the Vitess layer for parsing SQL statements using ANSI_QUOTES SQL mode, where double quotes are used as identifier quotes, and not string literal quotes. This change updates the implementations of mysql.Handler for the new function needed to parse prepared statement commands with ANSI_QUOTES mode. A future PR will enable the ability to turn on the ANSI_QUOTES support in Vitess.
• 1892: Fix panic for WHERE filter over SHOW VARIABLES Fix for: https://github.com/dolthub/dolt/issues/6379
• 1891: Left pad odd length hex strings Fixes https://github.com/dolthub/dolt/issues/6351 If hex string v is of odd length, prepend an "0" before passing it to Go's DecodeString. I've changed the implementation to use DecodeString, but if keeping this as a byte array is preferred, I'll switch back.
• 1890: Fix panic when selecting unsigned number columns via driver Calling the reflect.Value.Int() on an unsigned typed value will panic. The reflect.Value.Uint() must be used.
• 1889: Recost LookupJoins to remove special casing for indexes where every column is used in a filter. The only joins that are affected by this recosting are joins that use an index where every column in the index has a key expression, but we can't prove that lookups are constant. We currently special case this, giving these a slightly elevated priority over joins that don't use every column in the index (and we can't prove that lookups are constant.) So for example a lookup with one key expression on an index with one column ends up being preferred over say, a lookup with three key expressions on an index with four columns, even though it looks like in practice the latter is a more efficient join. Removing this special casing resulted in improved plans in specific client queries. Most of the changed plans in this PR are workarounds for https://github.com/dolthub/go-mysql-server/issues/1893, https://github.com/dolthub/go-mysql-server/issues/1894, and https://github.com/dolthub/go-mysql-server/issues/1895. A few are actually better plans than we were previously generating.
• 1887: have plan.EmptyTable implement sql.DeletableTable We throw an error when trying to delete from EmptyTable. This error is partially caused by https://github.com/dolthub/go-mysql-server/pull/1885 The error didn't show up in GMS as there are Exchange nodes that appear in dolt side that don't appear here.
• 1886: Apply index optimization to Offset nodes and drop 0 Offsets This PR applies the index sort optimization to plans that look like LIMIT(OFFSET(SORT())). Additionally, this PR drops OFFSET nodes that start at 0, as they don't change the output. Fix for: https://github.com/dolthub/dolt/issues/6347
• 1885: have plan.EmptyTable implement sql.Updatable interface In order to not throw table doesn't support UPDATE error, the EmptyTable node should implement the Updatable interface. Additionally, there are other private noop iterators to prevent nil panics. Fix for: https://github.com/dolthub/dolt/issues/5397
• 1884: Fix data race in processlist map Do a deep copy of maps and any nested maps to prevent concurrent read/write access. Fix for: https://github.com/dolthub/dolt/issues/6332
• 1882: Add dynamic privilege CLONE_ADMIN
• 1880: Add "Sliding Range Join" execution plan (The original PR was accidentally merged. I fixed the history but there doesn't seem to be a way to "unmerge" the PR) This is the draft implementation of the "Sliding Range Join" execution plan. This allows for more performant joins when the join condition checks that the column on one table is within a range specified by two columns on the other table.
• 1877: Add "Sliding Range Join" execution plan This is the draft implementation of the "Sliding Range Join" execution plan. This allows for more performant joins when the join condition checks that the column on one table is within a range specified by two columns on the other table. TODO: Elaborate in this description before making this PR not a draft.
• 1875: match notation for decimal parsing Follow up to https://github.com/dolthub/go-mysql-server/pull/1874 Turns out it is possible to specify floats using scientific notation, which caused some issues with conversions especially around large decimals.
• 1874: avoid scientific notation for floats/decimals We use some string comparison logic to find precision loss and determine if something should be float/decimal type. When printing very large floats, the %v format option in the fmt packages defaults to scientific notation, so like 1.234e567. This does not work well with our (hacky) string code. The %f option doesn't work very well, as it likes to append .00000. It appears strconv.FormatFloat does what we want, so I just picked that. fmt package docs: https://pkg.go.dev/fmt Format Float docs: https://pkg.go.dev/strconv#FormatFloat fix for: https://github.com/dolthub/dolt/issues/6322
• 1873: Supporting mysql.help_ tables This first pass adds the table schemas to the mysql database for help_keyword, help_category, help_topic, and help_relation. There is no support for data in the tables yet; we're starting with just table schemas to see if that's enough for tool compatibility. Related to: https://github.com/dolthub/dolt/issues/6308 I still need to do acceptance testing with the exact repro in the issue linked above. I'll follow up with Max on that tomorrow to confirm. I'm hoping just supporting the schema will be enough for the FusionAuth tool, but we may end up needing to populate these tables, too.
• 1871: Full-Text Indexes This is a partial implementation of Full-Text indexes. There is still quite a bit to finish on the GMS side (as can be seen from the copious amount of TODOs), but this shows the broad strokes of how it's implemented, along with most of the "difficult" design choices being implemented. The major choice that has not yet been finalized is how to deal with FTS_DOC_ID, as it's an AUTO_INCREMENT column in MySQL, but that would not play well with Dolt merging. I already have ideas on how to handle that (taking into account remotes, etc.), but that would come from a later PR. https://docs.google.com/document/d/1nGyYg461AhxQjFLzhEEj01XMz0VaTBaBaA44WNu0fc4/edit Quite a few things have changed from the initial design doc, mostly based on feedback during the meeting, however some of it was post-meeting. There are three tables instead of 1: Config (stores table-specific information shared across all indexes), WordToPos (maps words to an ID and position, not fully used in the default search), and Count (used to calculate relevancy, also not fully used in the default search). I was planning on converting MATCH ... AGAINST ... to a join between the tables, which would work when fetching results, but MATCH ... AGAINST ... may also be used as a result, which necessitated writing all of the functionality anyway, so the join plan was dropped. Last thing to mention, is that I'm pretty sure that Full-Text indexes actually do a full table scan. It seems weird, but AFAICT the indexes are used to quickly calculate relevancy for each search mode. It seems that, for overly large tables, the search time increases even when other index operations continue to operate nearly instantaneously. I've tagged two people for review to make it a bit easier. Of course, feel free to take a look at more if you desire. @reltuk The sql/fulltext/fulltext.go file is an expansion of the file you've previously reviewed (all still kept to a single file for now). To complement it and see how it'll be implemented on the Dolt side, you can look at memory/table.go. Dolt's table editor will be similar, and the merge paths will only use the FulltextEditor, which special logic to interface with it from those paths. @max-hoffman Take a look at the analyzer changes, along with the sql/plan/ddl.go file. You'll probably need to reference sql/fulltext/fulltext.go as well.
• 1869: Migrate most enginetests for new name resolution This doubles most of the enginetests to add versions with new name resolution. As a result testing takes ~2x as long, temporarily. Gets majority of those enginetests working with a couple bigger exceptions:
  • plans projections are subtly different in a way that should be optimized but is probably not priority
  • stored procedures need their custom resolution logic ported
  • on duplicate update expressions are buggy, going to rewrite those for new format
  • skipping one derived table alias test, where we do not have expression memoization or lateral joins to help us execute a resolved plan (related https://github.com/dolthub/dolt/issues/6407)
  • many tests throw "column not found" instead of "table not found" errors. I tried to bookkeep those with Skips, but the skipped suites may accumulate other differences in the meantime.
  • I'll need to revert our prepared statement logic before the final switch
  • Various validators work a bit differently, might end up skipping some error tests to get the final switch in sooner Other suites:
  • TestJSONTableScripts_Experimental -- json_table still broken
  • TestRenameColumn_Exp -- error test has different error A couple other discoveries:
  • We have to hallucinate unknown table, column, and procedure names in trigger bodies on CREATE, and only fail on execution
  • Column default expressions appear to be resolved at execution time
  • Alter statements are only resolved at execution time
  • The precedence for ASOF in nested views and procedure calls is a bit hairy
  • json_array w/ and w/o distinct appears to be untested
  • ORDER BY in UNION seems pretty flaky and lightly tested, we resolve names from the left Dolt PR: https://github.com/dolthub/dolt/pull/6414
• 1868: Fix bug in OrderedDistinct over Projection A mistype was checking the table name twice for column equality, rather than table and column name. The bug led to using OrderedDistinct in inappropriate cases.
• 1867: Support for USING character set expressions Fixes https://github.com/dolthub/dolt/issues/6291
• 1865: Add CustomValidator interface In many places we expect to see a commit hash in our result. Since the hashes take into account the system time when computing, it is difficult to predict what they will be. This PR adds a new interface that can be implemented on the dolt side to check for commit hashes.
• 1864: No parallelism for children of ordered distinct We permitted parallelism into an OrderedDistinct node, which is a specialized Distinct node that expects results sorted on a specific index key. This change prevents parallelizing children of OrderedDistinct.
• 1863: Bump google.golang.org/grpc from 1.37.0 to 1.53.0 Bumps google.golang.org/grpc from 1.37.0 to 1.53.0.
  Release notes

  Sourced from google.golang.org/grpc's releases.

  Release 1.53.0

  API Changes

  • balancer: support injection of per-call metadata from LB policies (#5853)
  • resolver: remove deprecated field resolver.Target.Endpoint and replace with resolver.Target.Endpoint() (#5852)
    • Special Thanks: @​kylejb

  New Features

  • xds/ringhash: introduce GRPC_RING_HASH_CAP environment variable to override the maximum ring size. (#5884)
  • rls: propagate headers received in RLS response to backends (#5883)

  Bug Fixes

  • transport: drain client transport when streamID approaches MaxStreamID (#5889)
  • server: after GracefulStop, ensure connections are closed when final RPC completes (#5968)
  • server: fix a few issues where grpc server uses RST_STREAM for non-HTTP/2 errors (#5893)
  • xdsclient: fix race which can happen when multiple load reporting calls are made at the same time. (#5927)
  • rls: fix a data race involving the LRU cache (#5925)
  • xds: fix panic involving double close of channel in xDS transport (#5959)
  • gcp/observability: update method name validation (#5951)

  Documentation

  • credentials/oauth: mark NewOauthAccess as deprecated (#5882)
    • Special Thanks: @​buzzsurfr

  Release 1.52.3

  Bug Fixes

  • Fix user-agent version

  Release 1.52.2

  Bug Fixes

  • xds: fix panic involving double close of channel in xDS transport (#5959)

  Release 1.52.1

  Bug Fixes

  • grpclb: rename grpclbstate package back to state (#5963)

  Release 1.52.0

  New Features

  • xdsclient: log node ID with verbosity INFO (#5860)
  • ringhash: impose cap on max_ring_size to reduce possibility of OOMs (#5801)

  Behavior Changes

  ... (truncated)

  Commits

  • dba26e1 Change version to 1.53.0 (#5983)
  • 2a1e934 server: after GracefulStop, ensure connections are closed when final RPC comp...
  • e2d69aa tests: fix spelling of variable (#5966)
  • a6376c9 xds/resolver: cleanup tests to use real xDS client 3/n (#5953)
  • bf8fc46 xds/resolver: cleanup tests to use real xDS client 5/n (#5955)
  • 3930549 resolver: replace resolver.Target.Endpoint field with Endpoint() method (#5852)
  • 894816c grpclb: rename grpclbstate package back to state (#5962)
  • e5a0237 encoding: fix duplicate compressor names (#5958)
  • 4adb2a7 xds/resolver: cleanup tests to use real xDS client 2/n (#5952)
  • 52a8392 gcp/observability: update method name validation (#5951)
  • Additional commits viewable in compare view
  
  [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
  Dependabot commands and options

  
  You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/dolthub/go-mysql-server/network/alerts).
• 1862: implementing lateral joins
• 1861: chore: remove refs to deprecated io/ioutil
• 1850: Name resolution correctness tests This fixes many of the remaining correctness tests for TestSimpleQueries, TestsJoinOps, TestJoinPlanning, TestColumnAliases, TestDerivedTableOuterScopeVisibility, TestAmbiguousColumnResolution, TestReadOnlyVersionedQueries with the new name resolution strategy. Many of the query plans are slightly different but mostly equivalent. Join rearrangements and un-nesting in particular are better after this change, because I needed the transform logic work for both. There are a variety of other bugs the slight plan differences exposed that are fixed now. This does not fix every set of enginetests, there is still a lot to do. But I'm locking in compatibility for most of the core tests to prevent backsliding. The next follow-up is probably replacing the old name resolution. I will need to figure out if triggers, procs, prepared statements need any sort of special treatment.

vitess

• 265: upgraded YAML library
• 264: Revert https://github.com/dolthub/vitess/pull/261 Change in description breaks prepared statements that send a value with character data above the max size of a CHAR column (256 bytes). Fixes https://github.com/dolthub/dolthub-issues/issues/489
• 263: Add SECURITY.md.
• 262: Prepared query statement round trip Changes to aliases and json_table to make the AST string formatters round-trip. Biggest change to tests is that useSelectExpressionLiteral seems to behave differently.
• 261: Preserving bind var type for sqltypes.Char params When a caller executes a prepared statement, they send the parameters as well as parameter type information to the sql-server. Vitess populates bindvars type information that go-mysql-server uses to execute the query. Vitess is currently converting many SQL types to VARBINARY, including CHAR, which makes it look like binary data was sent and not a char string, like the caller indicated. This change stops converting sqltypes.Char bind var type info into VARBINARY, which enables go-mysql-server to see that a char string was passed. Without this type information, go-mysql-server doesn't seem able to differentiate between legitimate binary data sent by the client versus a character string. Since these types need to be handled differently, and we can't assume that all VARBINARY types can be converted into strings, it seems like we need to respect the SQL type the caller indicated for the bind var and pass that up to integrators. It seems like this change could be helpful for other SQL types, too, but I wanted to start with just sqltypes.Char to see if this approach causes any other issues. Fixes: https://github.com/dolthub/go-mysql-server/issues/1855 Related go-mysql-server PR: https://github.com/dolthub/go-mysql-server/pull/1919 Related Dolt PR: https://github.com/dolthub/dolt/pull/6441
• 260: Rename MultiAlterDDL to AlterTable and return it for all ALTER TABLE statements This normalization makes it easier to handle building a plan
• 259: Bug fix for round tripping parsed cast function calls When formatting a parsed cast node back into a SQL string, we were using the form CAST(<arg>, <arg>), but that won't roundtrip back to MySQL. It needs to be either CONVERT(<arg>, <arg>) or CAST(<arg> as <arg).
• 258: Adding new keywords to the non-reserved list so they don't need to be… … quoted
• 257: Support for all temporal query forms in SQL 2011 These are the parser changes necessary to support https://github.com/dolthub/dolt/issues/6353 In addition to the 3 forms of temporal query in SQL 2011 (AS OF, FROM .. TO, BETWEEN), this PR also implements support for the 2 extensions added by SQLServer, ALL and CONTAINED IN. https://learn.microsoft.com/en-us/sql/relational-databases/tables/temporal-tables?view=sql-server-ver16 Additionally, VERSION is now a synonym for SYSTEM_TIME, a Dolt pseudo-extension.
• 256: Support ANSI_QUOTES parsing mode The ANSI_QUOTES SQL mode changes the behavior of the double quote character. By default in MySQL, ANSI_QUOTES is not enabled and the double quote character is used to quote string literals. When ANSI_QUOTES is enabled, the double quote character may only quote identifiers. The ANSI_QUOTES mode does not change the behavior for backtick quote chars (they always quote identifiers) or single quote chars (they always quote string literals). MySQL Reference Docs for ANSI_QUOTES Related to: https://github.com/dolthub/dolt/issues/6305 (This is the first step towards supporting ANSI_QUOTES mode in Dolt/GMS) GMS PR https://github.com/dolthub/go-mysql-server/pull/1896 stubs out the new Handler interface function, but actually using ANSI_QUOTES mode in Dolt or GMS won't be possible until a few more changes to Dolt/GMS.
• 254: Fix for using unqouted reserved words ('count' specifically) in the VALUES function
• 253: Automatically concatenate adjacent string literals Fixes https://github.com/dolthub/dolt/issues/5232 This is to match the mysql behavior:

  mysql> select "a" 'b'   "c";
  +-----+
  | a   |
  +-----+
  | abc |
  +-----+
  

  The grammar can't accommodate this so it has to go in the lexer. It doesn't work if the strings are broken up by a mysql special comment, so we still have to special case that. Also removed support for using string literals as table aliases. MySQL has a mode to support using double-quoted strings only as identifiers, but it's not on by default and isn't supported anywhere else in the grammar.
• 252: Fixed keyword usage in primary key clauses Fixes https://github.com/dolthub/dolt/issues/6290
• 251: Fixed missing support for collations as strings Fixes issue https://github.com/dolthub/dolt/issues/6192 We only allowed collations to be declared after the CREATE TABLE portion in their non-string form. This adds support for the string form.
• 250: support LATERAL syntax syntax for: https://github.com/dolthub/dolt/issues/6194

Closed Issues

• 1855: GORM JSONArrayQuery.Contains isn't compatible
• 1902: Valid join query erroneously reported as ambiguous
• 1881: duplicate column names in a table alias should be prevented.
• 1753: "SELECT * FROM tbl" has incorrect schema when table has default values.
• 1745: Allow table names with special characters when MySql allows them