feat(dbapi): use inline begin to eliminate BeginTransaction RPC

[waiho-gumloop]

Fixes googleapis/google-cloud-python#15871

Summary

Connection.transaction_checkout() currently calls Transaction.begin() explicitly before the first query, which sends a standalone BeginTransaction gRPC RPC. This is unnecessary because the Transaction class already supports inline begin — piggybacking BeginTransaction onto the first ExecuteSql/ExecuteBatchDml request via TransactionSelector(begin=...).

This PR removes the explicit begin() call, letting the existing inline begin logic in execute_sql(), execute_update(), and batch_update() handle transaction creation. This saves one gRPC round-trip per transaction (~16ms measured on the emulator).

What changed

google/cloud/spanner_dbapi/connection.py — transaction_checkout():

• Removed self._transaction.begin() on L413
• The transaction is now returned with _transaction_id=None
• Updated docstring to explain the inline begin behavior

tests/unit/spanner_dbapi/test_connection.py:

• Added test_transaction_checkout_does_not_call_begin to assert begin() is not called

tests/mockserver_tests/test_dbapi_inline_begin.py (new):

• 9 mockserver tests verifying inline begin behavior for read-write transactions
• Covers: no BeginTransactionRequest sent, first ExecuteSqlRequest uses TransactionSelector(begin=...), transaction ID reuse on second statement, rollback, read-only unaffected, retry after abort

tests/mockserver_tests/test_tags.py:

• Updated 4 read-write tag tests: removed BeginTransactionRequest from expected RPC sequences, adjusted tag index offsets

tests/mockserver_tests/test_dbapi_isolation_level.py:

• Updated 4 isolation level tests: verify isolation level on ExecuteSqlRequest.transaction.begin.isolation_level instead of BeginTransactionRequest.options.isolation_level

Why this is safe

The inline begin code path already exists and is battle-tested — Session.run_in_transaction() creates a Transaction without calling begin() and relies on the same inline begin logic (session.py L566).

Specific safety analysis:

1. transaction_checkout() callers always execute SQL immediately: It's only called from run_statement() → execute_sql() and batch_dml_executor → batch_update(). Both set _transaction_id via inline begin before any commit/rollback path.

2. execute_sql/execute_update/batch_update handle _transaction_id is None: They acquire a lock, use _make_txn_selector() which returns TransactionSelector(begin=...), and store the returned _transaction_id (transaction.py L612-L623).

3. rollback() handles _transaction_id is None: Skips the RPC — correct when no server-side transaction exists (transaction.py L163).

4. commit() handles _transaction_id is None: Falls back to _begin_mutations_only_transaction() for mutation-only transactions (transaction.py L263-L267).

5. Retry mechanism is compatible: _set_connection_for_retry() resets _spanner_transaction_started=False, so replayed statements go through transaction_checkout() again, create a fresh Transaction, and use inline begin.

PEP 249 conformance

This change is fully conformant with PEP 249 (DB-API 2.0). The spec does not define a begin() method — transactions are implicit. The PEP author clarified on the DB-SIG mailing list that "transactions start implicitly after you connect and after you call .commit() or .rollback()", and the mechanism by which the driver starts the server-side transaction is an implementation detail. Deferring the server-side begin to the first SQL execution (as psycopg2 and other mature DB-API drivers do) is the standard approach. The observable transactional semantics — atomicity between commit()/rollback() calls — are unchanged.

Performance impact

Before (4 RPCs per read-write transaction):

BeginTransaction → ExecuteSql (read) → ExecuteSql (write) → Commit

After (3 RPCs per read-write transaction):

ExecuteSql (read, with inline begin) → ExecuteSql (write) → Commit

Measured ~16ms savings per transaction on the Spanner emulator.

Context

This optimization was identified while profiling SQLAlchemy/DBAPI performance against Spanner. The DBAPI was created ~2 years before inline begin support was added to the Python client library (inline begin landed in PR googleapis/python-spanner#740, Dec 2022). The run_in_transaction path was updated to use inline begin, but transaction_checkout was not.

Test plan

• All 198 DBAPI unit tests pass (tests/unit/spanner_dbapi/)
• New unit test verifies begin() is not called by transaction_checkout()
• 9 new mockserver tests verify inline begin RPC behavior (tests/mockserver_tests/test_dbapi_inline_begin.py)
• 8 existing mockserver tests updated for inline begin expectations (test_tags.py, test_dbapi_isolation_level.py)
• All 53 DBAPI system tests pass against Spanner emulator (tests/system/test_dbapi.py)
• CI system tests (will run automatically)

[google-cla]

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

[gemini-code-assist]

Summary of Changes

Hello @waiho-gumloop, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request optimizes transaction handling in the Spanner DBAPI by leveraging the existing inline begin mechanism. It removes an unnecessary explicit BeginTransaction gRPC call, allowing transactions to be initiated implicitly with the first ExecuteSql or ExecuteBatchDml request. This change streamlines the transaction workflow, reduces network overhead, and improves performance by saving one RPC round-trip per transaction.

Highlights

• Eliminated BeginTransaction RPC: Removed the explicit _transaction.begin() call in Connection.transaction_checkout(), allowing the first SQL execution to initiate the transaction via inline begin.
• Performance Improvement: Reduced one gRPC round-trip per transaction, resulting in approximately 16ms savings on the Spanner emulator.
• Docstring Update: Updated the transaction_checkout() docstring to explain the new inline begin behavior.
• New Unit Test: Added test_transaction_checkout_does_not_call_begin to verify that begin() is no longer explicitly called.

Changelog

• google/cloud/spanner_dbapi/connection.py
  • Removed the explicit call to self._transaction.begin() within the transaction_checkout method.
  • Updated the docstring for transaction_checkout to describe the inline begin behavior and the elimination of a separate BeginTransaction RPC.
• tests/unit/spanner_dbapi/test_connection.py
  • Added a new unit test test_transaction_checkout_does_not_call_begin to assert that Transaction.begin() is not called by transaction_checkout.

Activity

• All 198 existing DBAPI unit tests have passed.
• All 53 existing DBAPI system tests have passed against the Spanner emulator.
• A new unit test was added to specifically verify the absence of the begin() call in `transaction_checkout().
• CI system tests are configured to run automatically.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces a valuable optimization by removing the explicit BeginTransaction RPC call in transaction_checkout. Instead, it leverages the existing inline begin functionality, where the transaction is initiated with the first SQL execution. This change effectively reduces latency by saving one network round-trip per transaction. The modification is well-supported by a new unit test ensuring begin() is not called, and the detailed description provides a thorough safety analysis, confirming the change is sound.

[chalmerlowe]

📢 Migration Notice: 📢 This library is moving to the google-cloud-python monorepo soon.

We kept this PR open due to recent activity. We would like to finalize this PR so it can be merged if it is critical.

If we don't hear from the PR author, we will close this PR in the next few days. The PR can then be re-opened in the monorepo once the migration is complete and work can continue there.

[olavloite]

Do not merge yet: This needs a bit of work, as it does not handle the situation where the first statement in a transaction fails, and thereby also fails to actually start a transaction and return a transaction ID.

[olavloite]

A couple of additional notes on this implementation:

First Statement Fails

The implementation now also handles the case where the first statement in a read/write transaction fails. From the commit message:

Add retries if the first statement in a read/write transaction fails, as the
statement then does not return a transaction ID. In order to ensure that we
get a transaction ID, we first execute an explicit BeginTransaction RPC and
then retry the original statement. We return the response of the retry to
the application, regardless whether the retry fails or succeeds.

The reason that we do a retry with a BeginTransaction AND include the
first statement, is to guarantee transaction consistency. If we were to
leave the first statement out of the transaction, then it will not be
guaranteed that the error condition that cause the failure in the first
place is actually still true when the transaction commits. This would break
the transaction guarantees.

Example (pseudo-code):

-- The following statement fails with ALREADY_EXISTS
insert into some_table (id, value) values (1, 'One');

-- Execute an explicit BeginTransaction RPC.
begin;
-- Retry the initial statement. This ensures that
-- whatever the response is, this response will be
-- valid for the entire transaction.
insert into some_table (id, value) values (1, 'One');

-- This is guaranteed to return a row.
select * from some_table where id=1;

-- ... execute the rest of the transaction ...
commit;

If we had not included the initial insert statement in the retried transaction,
then there is no guarantee that the select statement would actually return any
rows, as other transactions could in theory have deleted it in the meantime.

Performance Gain

The initial description mentions a performance gain of '~16ms measured on the emulator'. The actual gain on real Spanner is likely to be somewhere between 3ms to 5ms, assuming that the applications runs in the same region as the Spanner instance it is talking to.