pandas-gbq: read_gbq ignores dialect='legacy' when configuration has a query section #18774

Determine this is the right repository

• I determined this is the correct repository in which to report this bug.

Summary of the issue

Context
I call pandas_gbq.read_gbq with dialect="legacy" (or set pandas_gbq.context.dialect = "legacy") and also pass a configuration with a "query" section, for example {"query": {"useQueryCache": False}}, which is the example the read_gbq docstring gives for configuration.

Expected Behavior:
The query runs with legacy SQL, and the extra query options from configuration are applied on top.

Actual Behavior:
The query is sent without useLegacySql, so google-cloud-bigquery runs it as Standard SQL and the dialect is silently ignored. Legacy syntax such as [project:dataset.table] then fails or behaves differently.

GbqConnector.run_query in pandas_gbq/gbq_connector.py builds {"query": {"useLegacySql": self.dialect == "legacy"}} and then calls job_config_dict.update(configuration). A configuration with its own "query" key replaces that whole section, so useLegacySql is dropped. The same happens when the query text itself is passed as configuration={"query": {"query": "..."}}, since read_gbq pops the text and leaves an empty "query" section behind.

API client name and version

pandas-gbq 0.35.2 (main at 4eb0e71), google-cloud-bigquery 3.46.2

Reproduction steps: code

file: main.py

import os

os.environ["BIGQUERY_EMULATOR_HOST"] = "http://127.0.0.1:8765"

import pandas_gbq
from google.auth.credentials import AnonymousCredentials

pandas_gbq.read_gbq(
    "SELECT 1 AS x",
    project_id="p",
    credentials=AnonymousCredentials(),
    progress_bar_type=None,
    dialect="legacy",
    configuration={"query": {"useQueryCache": False}},
)

Reproduction steps: supporting files

file: fake_bq.py (a local stand in for jobs.query that prints the options each query sends)

import json
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer

class H(BaseHTTPRequestHandler):
    def do_POST(self):
        body = json.loads(self.rfile.read(int(self.headers["content-length"])) or b"{}")
        print(f"useLegacySql={body.get('useLegacySql')!r} useQueryCache={body.get('useQueryCache')!r}", flush=True)
        out = {"kind": "bigquery#queryResponse", "jobComplete": True, "totalRows": "1",
               "jobReference": {"projectId": "p", "jobId": "j1", "location": "US"},
               "schema": {"fields": [{"name": "x", "type": "INTEGER", "mode": "NULLABLE"}]},
               "rows": [{"f": [{"v": "1"}]}]}
        data = json.dumps(out).encode()
        self.send_response(200); self.send_header("content-type", "application/json")
        self.send_header("content-length", str(len(data))); self.end_headers(); self.wfile.write(data)
    def log_message(self, *a): pass

ThreadingHTTPServer(("127.0.0.1", 8765), H).serve_forever()

Reproduction steps: actual results

The server prints:

useLegacySql=False useQueryCache=False

Without the configuration argument the same call sends useLegacySql=True.

Reproduction steps: expected results

useLegacySql=True useQueryCache=False

OS & version + platform

Linux 7.0 x86_64

Python environment

Python 3.12.3

Python dependencies

google-api-core          2.42.0
google-auth              2.62.0
google-cloud-bigquery    3.46.2
pandas                   3.0.6
pandas-gbq               0.35.2
pydata-google-auth       1.9.1
requests                 2.34.2

Additional context

The fix is to merge the "query" section of configuration into the default one instead of replacing it, so an explicit useLegacySql in configuration still wins and only the missing key case changes. I have a PR with tests ready and will link it here.