[maharanay22]

Description

Since 26.1.0, native queries to a PostgreSQL integration fail for anything that isn't a plain SELECT. For example, SELECT * FROM pg (UPDATE demo SET status = 'done' WHERE id = 1) returns:

syntax error at or near "UPDATE"
LINE 1: DECLARE "mindsdb_2693970344912" CURSOR FOR UPDATE demo SET s...

The same happens for DELETE, CREATE, ALTER, TRUNCATE, SHOW, EXPLAIN and data-modifying CTEs. Before 26.1.0, native queries didn't go through a server-side cursor

Repro: connect a PostgreSQL database with CREATE DATABASE pg WITH ENGINE = 'postgres', PARAMETERS = {...}, then run any of the queries above through SELECT * FROM pg (...)

Root cause: the data streaming change (#12018 in the old repo) runs every native query through a psycopg server-side cursor, which sends DECLARE ... CURSOR FOR <query>. PostgreSQL only accepts SELECT and VALUES in a cursor. There was a fallback to a client side cursor, but it only kicked in when the error text started with syntax error at or near "insert" or "drop"

The fix falls back to the client side cursor whenever DECLARE fails with SyntaxError or FeatureNotSupported (the error PostgreSQL raises for WITH x AS (UPDATE ...) SELECT ...), and drops the error text matching. DECLARE rejects the statement before running it, so running it again on the client side doesn't apply anything twice. I went with this over checking the first keyword up front because the server is the only reliable judge here: a keyword check gets comments, parentheses, SELECT ... INTO and WITH with UPDATE wrong. Other errors such as a missing table are still returned right away without a retry, and SELECT queries still stream through the server-side cursor

Once the handler ran these statements, they still failed one step later with 'OkResponse' object has no attribute 'data_frame'. This happened because FetchDataframeStepCall expected a table for SELECT * FROM db (...). It now returns an empty result for an OkResponse, which matches how the pre-26.1.0 code handled responses without a table (from reading the code, I didn't run 26.0.1). DROP TABLE hit this error even before the change, since it already went through the old fallback

I used an AI assistant to help investigate this and draft the change and tests, and I reviewed and tested it myself

Fixes: no issue to link, because issues are disabled on this repo. The bug is described above

Type of change

• 🐛 Bug fix (non-breaking change which fixes an issue)

Verification Process

To ensure the changes are working as expected:

• Test Location: tests/unit/handlers/test_postgres.py, tests/unit/executor/test_executor.py, and a local MindsDB (python -m mindsdb --api http) on top of a real PostgreSQL 16.2 server
• Verification Steps: run the native queries below through POST /api/sql/query, once on main and once on this branch

Before (main):

SELECT * FROM pg (DROP TABLE IF EXISTS demo)               -> error: 'OkResponse' object has no attribute 'data_frame'
SELECT * FROM pg (CREATE TABLE demo (id int, status text))  -> error: syntax error at or near "CREATE"
SELECT * FROM pg (UPDATE demo SET status = 'done' WHERE id = 1) -> error: syntax error at or near "UPDATE"
SELECT * FROM pg (DELETE FROM demo WHERE id = 2)            -> error: syntax error at or near "DELETE"
SELECT * FROM pg (SHOW server_version)                      -> error: syntax error at or near "SHOW"
SELECT * FROM pg (EXPLAIN SELECT * FROM demo)               -> error: syntax error at or near "EXPLAIN"

After (this branch):

SELECT * FROM pg (DROP TABLE IF EXISTS demo)                -> {"type": "table", "column_names": [], "data": []}
SELECT * FROM pg (CREATE TABLE demo (id int, status text))  -> {"type": "table", "column_names": [], "data": []}
SELECT * FROM pg (INSERT INTO demo VALUES (1, 'new'), (2, 'new')) -> {"type": "table", "column_names": [], "data": []}
SELECT * FROM pg (UPDATE demo SET status = 'done' WHERE id = 1) -> {"type": "table", "column_names": [], "data": []}
SELECT * FROM pg (DELETE FROM demo WHERE id = 2)            -> {"type": "table", "column_names": [], "data": []}
SELECT * FROM pg (SHOW server_version)                      -> {"type": "table", "column_names": ["server_version"], "data": [["16.2"]]}
SELECT * FROM pg (EXPLAIN SELECT * FROM demo)               -> {"type": "table", "column_names": ["QUERY PLAN"], "data": [["Seq Scan on demo  (cost=0.00..22.70 rows=1270 width=36)"]]}
SELECT * FROM pg (SELECT * FROM demo)                       -> {"type": "table", "column_names": ["id", "status"], "data": [[1, "done"]]}

I also called PostgresHandler.native_query directly against the same server. A data-modifying CTE that increments a counter ran exactly once (max(w) is 1 afterwards), SELECT still streams, and an invalid query still returns its syntax error:

WITH x AS (UPDATE t1 SET w=coalesce(w,0)+1 RETURNING id) SELECT * FROM x -> TABLE, 3 rows
SELECT max(w) AS w FROM t1                                               -> streamed TABLE, 1 rows: ['1']
SELECT * FROM t1 ORDER BY id                                             -> streamed TABLE, 3 rows
SELEC 1                                                                  -> ERROR: syntax error at or near "SELEC"

The new unit tests fail on main (8 subtests in test_native_query_stream_falls_back_to_client_side_when_cursor_is_not_allowed, plus test_integration_native_query_without_result) and pass on this branch

Additional Media:

• I have attached a brief loom video or screenshots showcasing the new functionality or change. (Terminal before and after output is included above in place of a video)

Checklist:

• My code follows the style guidelines(PEP 8) of MindsDB.
• I have appropriately commented on my code, especially in complex areas.
• Necessary documentation updates are either made or tracked in issues.
• Relevant unit and integration tests are updated or added.

[github-actions]

All contributors have signed the CLA ✍️ ✅
_{Posted by the CLA Assistant Lite bot.}

[maharanay22]

I have read the CLA Document and I hereby sign the CLA

[maharanay22]

@lucas-koontz could you take a look at this and #9 to #14 when you have time? They are small connector fixes, all green