fix(bq_driver): Fix encoding for SAP HANA by shivamd-gpartner · Pull Request #1506 · googleapis/cpp-bigquery-odbc

shivamd-gpartner · 2026-04-27T07:21:54Z

Fixed the encoding which was causing the issue on SAP HANA and had a different size of SQLWCHAR that the one it was built with . Another issue was the usage of std::regex crashing the driver due conflicting ABI with SAP HANA and moved to use RE2 regex now. PR size is bigger as had to change the usage of SQLWCHAR vector wherever we were creating earlier. Windows and linux checks are run here. Have verified on windows third party tools , seems working fine

sachinpro · 2026-05-07T09:19:05Z

-    std::regex named_pattern(R"([:@]\w+)");
-
-    // Check for positional parameters
-    if (std::regex_search(query, positional_pattern)) {


Let's deep dive into the reason this was crashing.

Let's print query and positional_pattern. Let's write a small c++ application to verify that it crashes independently.

Have verified and independently it works, issue is due to SAP HANA interfering with std::regex ABI , using re2 regex instead

sachinpro · 2026-05-20T14:03:44Z

    for (auto const& procedure_field : bq_procedure.schema.fields) {
-      std::regex column_pattern = BuildRegex(bq_procedure_column, metadata_id);
-      if (std::regex_match(procedure_field.name, column_pattern)) {
+      auto column_pattern = BuildRegex(bq_procedure_column, metadata_id);


We avoid using auto for better readability.

shivamd-gpartner · 2026-05-21T08:43:37Z

@sachinpro please check the PR

sachinpro · 2026-05-26T13:03:11Z

+  return false;
+#else
+  if (sizeof(SQLWCHAR) != 4) return false;
+  return g_utf16le_wire_latched.load(std::memory_order_relaxed);


Please check what is the value returned here and sizeof(SQLWCHAR).

IsRuntimeWireUtf16Le returns true here for SAP HANA and prints sizeof(SQLWCHAR) as 4 as the driver is built with iodbc where sizeof(SQLWCHAR) is 4 , and that's why we can't directly use sizeof(SQLWCHAR) as on SAP HANA system

sachinpro · 2026-05-26T13:12:00Z

+  // later call (including the ambiguous ones from SQLTables(catalog="%"))
+  // honors the latch via IsRuntimeWireUtf16Le().
+  if (sizeof(SQLWCHAR) == 4) {
+    bool use_utf16le = g_utf16le_wire_latched.load(std::memory_order_relaxed);


Let's read DriverManagerEncoding from googlebigqueryodbc.ini to read the encoding and WireWcharSize should use it.

sachinpro · 2026-06-02T13:41:36Z

-    std::memset(outConnectionString, '\0',
-                outConnectionStringBufferLen * sizeof(SQLWCHAR));
-    std::memcpy((SQLWCHAR*)outConnectionString,
-                ToSqlWChar(utf16_out_conn_str->data()),


Lets do the bit manipulation inside ToSqlWChar consitionally for HANA, without changing the signature of that function. We just need to return the 4 bytes of SQLWCHAR in a way that final memcpy writes them correctly in chunks of utf-16 characters.

ToSqlWChar was making the logic overly complicated and not reusable at other places where we were utilising SQLWchar directly to write to the buffer, have refactored and simplified the logic to not do unnecessary copying and reuse WriteWideToWireBuffer function everywhere, Have tested on both SAP Hana and on Tableau, PowerBi to verify things.

shivamd-gpartner temporarily deployed to internal April 28, 2026 11:33 — with GitHub Actions Inactive

shivamd-gpartner force-pushed the test_hana branch from 735d19f to 873b9da Compare April 28, 2026 12:14

shivamd-gpartner temporarily deployed to internal April 28, 2026 12:16 — with GitHub Actions Inactive

shivamd-gpartner temporarily deployed to internal April 28, 2026 14:34 — with GitHub Actions Inactive

shivamd-gpartner force-pushed the test_hana branch 13 times, most recently from 311c71e to 814c1fb Compare May 5, 2026 10:41

shivamd-gpartner force-pushed the test_hana branch from 511f04e to 7b8b154 Compare May 6, 2026 08:26

shivamd-gpartner temporarily deployed to internal May 6, 2026 10:56 — with GitHub Actions Inactive

shivamd-gpartner temporarily deployed to internal May 6, 2026 11:19 — with GitHub Actions Inactive

shivamd-gpartner force-pushed the test_hana branch 3 times, most recently from f9232a2 to 9ce5ad1 Compare May 6, 2026 12:56

shivamd-gpartner temporarily deployed to internal May 6, 2026 13:04 — with GitHub Actions Inactive

shivamd-gpartner force-pushed the test_hana branch 4 times, most recently from 07fe46c to c350b18 Compare May 7, 2026 07:09

sachinpro reviewed May 7, 2026

View reviewed changes

shivamd-gpartner force-pushed the test_hana branch from 3ae14c6 to 79a7205 Compare May 12, 2026 05:38

fixed clang

bbe04b5

shivamd-gpartner temporarily deployed to internal May 20, 2026 13:59 — with GitHub Actions Inactive

sachinpro reviewed May 20, 2026

View reviewed changes

shivamd-gpartner force-pushed the test_hana branch from efcec36 to 83ddd86 Compare May 21, 2026 07:24

shivamd-gpartner temporarily deployed to internal May 21, 2026 07:24 — with GitHub Actions Inactive

udpate comment

337cd07

shivamd-gpartner force-pushed the test_hana branch from 83ddd86 to 337cd07 Compare May 21, 2026 07:35

shivamd-gpartner changed the title ~~fix encoding~~ fix(bq_driver): Fix encoding for SAP HANA May 21, 2026

shivamd-gpartner marked this pull request as ready for review May 21, 2026 08:38

shivamd-gpartner requested a review from a team as a code owner May 21, 2026 08:38

shivamd-gpartner requested a review from sachinpro May 21, 2026 08:39

sachinpro reviewed May 26, 2026

View reviewed changes

shivamd-gpartner temporarily deployed to internal May 27, 2026 17:59 — with GitHub Actions Inactive

shivamd-gpartner temporarily deployed to internal May 29, 2026 12:23 — with GitHub Actions Inactive

sachinpro reviewed Jun 2, 2026

View reviewed changes

shivamd-gpartner added 3 commits June 3, 2026 16:56

test_improvement

d351cd3

removed all references sqlwchar

a59cf33

test

6203a13

shivamd-gpartner force-pushed the test_hana branch from d2d02c2 to 31136af Compare June 5, 2026 05:27

shivamd-gpartner temporarily deployed to internal June 5, 2026 05:27 — with GitHub Actions Inactive

shivamd-gpartner force-pushed the test_hana branch 2 times, most recently from 2054549 to 0a47525 Compare June 5, 2026 07:22

shivamd-gpartner temporarily deployed to internal June 5, 2026 07:22 — with GitHub Actions Inactive

checkers fix

e86861a

shivamd-gpartner force-pushed the test_hana branch from 0a47525 to 2b1c9d0 Compare June 5, 2026 08:20

shivamd-gpartner temporarily deployed to internal June 5, 2026 08:21 — with GitHub Actions Inactive

checkers fix

2c8bd1a

shivamd-gpartner force-pushed the test_hana branch from 2b1c9d0 to 2c8bd1a Compare June 5, 2026 08:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(bq_driver): Fix encoding for SAP HANA#1506

fix(bq_driver): Fix encoding for SAP HANA#1506
shivamd-gpartner wants to merge 34 commits into
mainfrom
test_hana

shivamd-gpartner commented Apr 27, 2026 •

edited

Loading

Uh oh!

sachinpro May 7, 2026

Uh oh!

shivamd-gpartner May 21, 2026

Uh oh!

sachinpro May 20, 2026

Uh oh!

shivamd-gpartner May 21, 2026

Uh oh!

shivamd-gpartner commented May 21, 2026

Uh oh!

sachinpro May 26, 2026

Uh oh!

shivamd-gpartner May 29, 2026

Uh oh!

sachinpro May 26, 2026

Uh oh!

shivamd-gpartner May 29, 2026

Uh oh!

sachinpro Jun 2, 2026

Uh oh!

shivamd-gpartner Jun 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

shivamd-gpartner commented Apr 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

shivamd-gpartner commented May 21, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

shivamd-gpartner commented Apr 27, 2026 •

edited

Loading