[seekdb][change stream] Fix refresh consistency - #1225
Open
hnwyllmm wants to merge 370 commits into
Open
Conversation
…atch Replace periodic timer polling and synchronous refresh with async TG_SCHEDULE dispatch via ObInternalTableChangeNotifier - Notifier: simplified ModuleEntry to single callback (removed array) pure dispatch with no locks or retry logic in the framework - SRS: removed TenantSrsUpdatePeriodicTask/TenantSrsUpdateTask added async RetryTimerTask that self-reschedules on failure and stops on success; removed dead notify_srs_changed - TIMEZONE: moved bootstrap refresh from start to notifier callback path, UpdateTenantTZTask self-reschedules on failure All modules register callbacks in init. Import executor and role-change-driven switch_to_leader both trigger async dispatch.
**Issue:** obperf sampling shows that in seekdb's idle state, KVCache Wash consumes 61.4% of the process CPU, making it the biggest hotspot. Each time the wash timer (200ms) triggers, it executes: 1. `ObKVCacheStore::wash` - `refresh_score`: Iterates through all `mb_handles`, performing a `hazptr protect/release` on each. - Heap construction loop: Iterates through all `mb_handles` again, performing another `hazptr protect/release` on each. → This results in two passes, with each handle undergoing 2 `protect/release` operations. 2. `ObKVCacheMap::clean_garbage_node` - Iterates over 200K buckets, acquiring a write lock (`ObBucketWLockGuard`) for each. - During idle, the vast majority of buckets are empty, leading to locks being acquired and immediately released, which is pure waste. → This accounts for 88% of the total wash CPU (~1280 out of 1407 samples are in `BucketLock::wrlock/unlock`). **Optimization 1: Merged `refresh_score` into the heap construction loop (`ob_kvcache_store.cpp`)** The per-handle score decay logic from `refresh_score` was inlined into the existing heap construction loop. This reduces the number of `hazptr protect/release` operations per handle from 2 to 1. The O(1) global calculation for `base_mb_score_` is preserved. **Optimization 2: Fast skip for empty buckets (`ob_kvcache_map.cpp`)** Before acquiring a lock in `clean_garbage_node` and `replace_fragment_node`, check if `get_bucket_node(i)` is `NULL`. - If `NULL` → skip, do not acquire the lock. - If not `NULL` → acquire the lock, then double-check before processing. Safety: Pointer reads are hardware-atomic on aarch64/x86_64; worst-case scenario falls back to the old behavior. **Verification:** - Before optimization (PID 100137, CPU 10.3%): Wash CPU 61.4%, `clean_garbage` 58.7%. - Intermediate (PID 43062, only optimization 1): Wash CPU 64.0%, `clean_garbage` 61.9% (Optimization 1 did not address the main bottleneck). - After optimization (PID 98980, CPU 10.7%): Wash CPU 11.5%, `clean_garbage` 7.3% (↓81%). - Functional verification: After forcing cache eviction, `clean_node_count=2~5`, and garbage nodes were correctly cleaned. **Flame Graphs:** - Before optimization: http://obperf.oceanbase-dev.com/files/profile_20260517004356__work.flame.svg - After optimization: http://obperf.oceanbase-dev.com/files/profile_20260517100310__work.flame.svg
obperf CPU sampling shows that in idle seekdb instances, ObBKGDSessInActiveGuard accounts for 1.7% of samples (687/41161). This object is constructed/destructed each sleep cycle, each time involving: - thread_local diagnostic info access - is_ash_enabled global variable read - set_sess_inactive: rdtsc + CACHE_ALIGNED bool write - set_sess_active: rdtsc + idle time accumulation + trace_id read + ASH buffer binding A full lifecycle is triggered 100 times per second on the TimeWheel deadlock detector (10ms precision). Two instances of ObBKGDSessInActiveGuard removed: 1. ob_clock_generator.h: ObClockGenerator::usleep - Remove the inactive_guard construction before each nanosleep - Main beneficiary: The TimeWheel stack guard in TimeWheelBase::scan drops from 303 samples to zero 2. utility.h: ob_usleep(v, is_idle_sleep=true) - Remove the guard in the is_idle_sleep branch; both branches are merged into a direct ob_usleep(v) call - The is_idle_sleep parameter is kept but marked as unused for signature compatibility - Beneficiaries: ObMultiTenant::run1, ObTimerService::run1, ObBaseLogWriter::do_flush_log, ObDDLTransController::run1, etc. | Metric | Before Optimization | After Optimization | |-----------------|-------------|-------------| | BKGDSessGuard | 687 (1.7%) | 320 (0.7%) | | TimeWheel+Guard | 303 samples | 0 samples | The remaining Guard samples come from ~35 modules constructing it directly, not via generic sleep functions. ## Risk Assessment - The ASH sampler will record fewer inactive state switches during sleep periods - No functional impact on user-visible diagnostic information - Thread ASH inactive marking can still be used via direct Guard construction ## Next Steps - The remaining 320 Guard samples come from direct construction in various modules (ob_timer_service, ob_base_log_writer, etc.) and can be optimized module-by-module as needed.
… RTTI overhead. On the server side, run_wrapper always points to an ObTenantBase instance, making the type checking from dynamic_cast provably redundant. Changing it to static_cast avoids the virtual table traversal that occurs every time a timer task switches tenants.
… from 1ms to 100ms. ### Background ObClockGenerator is a singleton background thread that calls `gettimeofday` every 1ms to update the atomically cached `cur_ts_`. Readers access it via `ATOMIC_LOAD`. Perf data shows that the thread's wake/sleep cycle incurs significant kernel overhead (`schedule_hrtimeout_range_clock` accounts for ~3.31%, `run1` itself ~0.30%). ### Options Explored #### Option A: Simplify to a direct current_time wrapper (Rejected) Completely remove the background thread; `getClock` directly calls `ObTimeUtility::current_time` (vDSO `gettimeofday`). Release build A/B comparison (oltp_read_only, 4 threads, 30s) | Metric | Original (1ms background thread) | Simplified (direct call) | Difference | |-------------------|--------------------|-------------|---------| | Idle Process CPU | 5.53% | 8.42% | +52% | | Load TPS | 766.99 | 764.22 | -0.36% | | Load Clock CPU | 1.22% | 3.19% | +1.97% | | getClock per call | 1.76ns (ATOMIC) | 35.59ns | 20x | Conclusion: Idle CPU regresses significantly (+2.89%), as the cumulative vDSO overhead from high-frequency internal timer/polling calls to `getClock` exceeds the cost of a single `sleep(1ms)` thread. Rejected. #### Option B: Background advancement interval 1ms → 100ms (Adopted) Minimize scope of change: Keep the background thread + `ATOMIC_LOAD` hot path unchanged, only reduce the wake-up frequency. | Metric | Original (1ms) | 100ms Option | Benefit | |-------------------|------------|-----------|---------------| | run1 Wake Frequency | 1000/s | 10/s | Reduced 100x | | getClock Hot Path | 1.76ns | 1.76ns | Unchanged | | run1 CPU | ~0.30% | ~0.003% | Nearly zero | | Clock Freshness | <1ms | <100ms | Acceptable | | MAX_JUMP_TIME_US | 20ms | 2s | Scales with SLEEP_US | ### Change One-line change: `deps/oblib/src/common/ob_clock_generator.cpp` `SLEEP_US`: 1000 → 100000 `MAX_JUMP_TIME_US` maintains the 20 * SLEEP_US relationship, changing from 20ms to 2s. This parameter only affects the sensitivity of clock jump anomaly detection and has no impact on correctness. ### Risks - The clock can lag by up to 100ms. All callers—such as log timestamps, timeout checks (typically second-level), and thread liveness detection—do not rely on sub-millisecond freshness. - This aligns with the earlier conclusion from reviewing all `getClock` / `getRealClock` callers: no caller strictly depends on monotonicity or sub-millisecond precision.
obperf sampling shows that KVCache Wash accounts for 61.4% of process CPU during seekdb idle state, making it the biggest hotspot. Each time the wash timer (200ms) triggers, it executes: 1. `ObKVCacheStore::wash` - `refresh_score`: Iterates through all mb_handles, performing hazptr protect/release for each one. - Heap construction loop: Iterates through all mb_handles again, performing hazptr protect/release for each one. → This results in two passes, with each handle undergoing protect/release twice. 2. `ObKVCacheMap::clean_garbage_node` - Iterates over 200K buckets, acquiring a write lock (`ObBucketWLockGuard`) for each. - During idle, the vast majority of buckets are empty, so locks are acquired and immediately released, which is pure waste. → This accounts for 88% of the total wash CPU (~1280/1407 samples are in `BucketLock::wrlock/unlock`). ## Optimization 1: Merged `refresh_score` into the heap construction loop (`ob_kvcache_store.cpp`) The per-handle score decay logic from `refresh_score` was inlined into the existing heap construction loop. Each handle now performs hazptr protect/release only once, down from twice. The O(1) global calculation for `base_mb_score_` is preserved. ## Optimization 2: Fast skip for empty buckets (`ob_kvcache_map.cpp`) Before acquiring a lock, `clean_garbage_node` and `replace_fragment_node` now check if `get_bucket_node(i)` is NULL. - NULL → Skip, do not acquire the lock. - Non-NULL → Acquire the lock, then double-check before processing. Safety: Pointer reads are hardware-atomic on aarch64/x86_64; worst-case behavior reverts to the old logic. - Before optimization (PID 100137, CPU 10.3%): Wash CPU 61.4%, `clean_garbage` 58.7%. - Intermediate (PID 43062, only Optimization 1): Wash CPU 64.0%, `clean_garbage` 61.9% (Optimization 1 did not address the main bottleneck). - After optimization (PID 98980, CPU 10.7%): Wash CPU 11.5%, `clean_garbage` 7.3% (↓81%). - Functional verification: After forcing cache eviction, `clean_node_count=2~5`, and garbage nodes were correctly cleaned. ## Flame Graphs - Before optimization: http://obperf.oceanbase-dev.com/files/profile_20260517004356__work.flame.svg - After optimization: http://obperf.oceanbase-dev.com/files/profile_20260517100310__work.flame.svg
Background: obperf CPU sampling shows that in an idle seekdb instance, ObBKGDSessInActiveGuard accounts for 1.7% of samples (687/41161). This object is constructed/destructed each sleep cycle, each time involving: - thread_local diagnostic info access - reading the global variable is_ash_enabled - set_sess_inactive: rdtsc + CACHE_ALIGNED bool write - set_sess_active: rdtsc + idle time accumulation + trace_id read + ASH buffer binding A full lifecycle is triggered 100 times per second on the TimeWheel deadlock detector (10ms precision). Changes: Remove two instances of ObBKGDSessInActiveGuard. 1. ob_clock_generator.h: ObClockGenerator::usleep - Remove the inactive_guard construction before each nanosleep. - Main beneficiary: The TimeWheel stack guard in TimeWheelBase::scan goes from 303 samples to zero. 2. utility.h: ob_usleep(v, is_idle_sleep=true) - Remove the guard in the is_idle_sleep branch, merging both branches into a direct ob_usleep(v). - The is_idle_sleep parameter is kept but commented as unused for signature compatibility. - Beneficiaries: ObMultiTenant::run1, ObTimerService::run1, ObBaseLogWriter::do_flush_log, ObDDLTransController::run1, etc. Results: | Metric | Before Optimization | After Optimization | |-----------------|-------------|-------------| | BKGDSessGuard | 687 (1.7%) | 320 (0.7%) | | TimeWheel+Guard | 303 samples | 0 samples | The remaining Guard samples come from ~35 modules constructing it directly, not via the generic sleep functions. Risk Assessment: - The ASH sampler will record fewer inactive state switches during sleep periods. - No functional impact on user-visible diagnostic information. - The thread's ASH inactive flag can still be used by directly constructing the Guard. Next Steps: - The remaining 320 Guard samples come from direct construction in various modules (ob_timer_service, ob_base_log_writer, etc.) and can be optimized module-by-module as needed.
…e RTTI overhead. On the server side, run_wrapper always points to an ObTenantBase instance, making the type check performed by dynamic_cast provably redundant. Switching to static_cast removes the virtual table traversal that occurs with every timer task during tenant switching.
…tMgr Timer Task. ObPxTargetMgr executes `refresh_statistics` every 500ms. Within this process, it determines whether the current node is a Leader or Follower via the call chain: `get_dummy_leader` -> `check_dummy_location_credible` -> `get_role`. Internally, `get_role` performs `MTL_SWITCH` + `check_palf_exist` + `open_palf` + `get_role` to query the Paxos role of SYS_LS. This is a CPU hotspot per tick (flame graph shows `runTimerTask` at 0.57%, with `lib_mtl_switch` accounting for 10.04% of that). SeekDB operates in observer-lite single-replica mode, where `ElectionImpl::get_role` always returns LEADER, and the Leader never switches. Therefore, this entire Leader/Follower coordination mechanism is effectively dead code. ## Changes Made Removed the following dead code (-237 lines): 1. `get_dummy_leader` — `MTL_SWITCH` + `nonblock_get_leader` to locate the Leader. 2. `check_dummy_location_credible` — Calls `get_role` to verify cache credibility. 3. `get_role` — `MTL_SWITCH` + `open_palf` to obtain the Paxos role. 4. `refresh_dummy_location` — An empty function that only returns `OB_SUCCESS`. 5. `query_statistics` — RPC logic for Followers to report resource increments to the Leader. 6. `reset_follower_statistics` — Logic for resetting statistics on the Follower side. Removed member variables that are no longer needed: - `cluster_id_`, `dummy_cache_leader_`, `rpc_proxy_`, `need_send_refresh_all_` Simplified `refresh_statistics`: - Removed the `get_dummy_leader` call and the Follower branch. - On the first tick, execute `reset_leader_statistics` once; subsequent ticks return directly. ## Scope of Applicability This change applies only to single-replica (observer-lite) deployments. It cannot be applied to multi-machine OceanBase clusters, as they have a genuine need for real Leader/Follower switching. - Compilation passes successfully. - An OBD-deployed single-instance starts normally. - MySQL connections work correctly.
…n-memory atomic CAS
## Problem
The Fetcher main loop (ObCSFetcher::run1) calls try_advance_refresh_scn_ every 200ms
unconditionally, regardless of IDLE or ACTIVE mode. Under IDLE mode (no async vector
index tables), the loop iterates every 10ms, and get_refresh_scn returns GTS each time.
This results in a SQL UPDATE against __all_global_stat every 200ms even when the value
has not changed, causing unnecessary database writes.
Additionally, the Worker path (do_finish_batch_) also writes refresh_scn via SQL
within a transaction, and the virtual table query reads it via SQL — all incurring
database round-trips for a monotonically-advancing counter that only needs to be
queryable within the same process.
## Solution
Migrate refresh_scn from __all_global_stat (SQL-persisted) to in-memory management
in ObCSDispatcher using atomic variables (ATOMIC_LOAD/ATOMIC_STORE/ATOMIC_BCAS)
1. ObCSDispatcher::refresh_scn_ becomes an atomic variable with CAS-based
update_refresh_scn that advances the value only when the new value is larger.
2. Fetcher path (try_advance_refresh_scn_)
- Was: ObGlobalStatProxy::advance_change_stream_refresh_scn → SQL UPDATE
- Now: dispatcher_->update_refresh_scn → in-memory CAS, zero SQL
3. Worker path (do_finish_batch_)
- refresh_scn advancement moves from release_batch to after batch commit
- Uses dispatcher_->update_refresh_scn instead of SQL
4. Read paths
- wait_refresh_scn in ObChangeStreamMgr reads from dispatcher_->get_refresh_scn
- Virtual table ob_all_virtual_change_stream_refresh_stat reads in-memory state
- init_refresh_scn_ still loads from __all_global_stat once as recovery baseline
and allows rollback on reload (recovery semantics differ from runtime advancement)
5. Cleanup
- Removes refresh_scn_inited flag and init_refresh_scn public method
- Refactors release_batch to no longer manage refresh_scn directly
## Files Changed
- ob_change_stream_dispatcher.{h,cpp}: atomic refresh_scn_, update_refresh_scn, cleanup
- ob_change_stream_fetcher.cpp: Fetcher uses dispatcher->update_refresh_scn
- ob_change_stream_worker.cpp: Worker commits refresh_scn after batch success
- ob_change_stream_mgr.cpp: wait_refresh_scn reads from in-memory dispatcher
- ob_all_virtual_change_stream_refresh_stat.cpp: virtual table reads from memory
- wait_cs_sync.inc: mysqltest adjustment
…anual concatenation using MEMCPY. set_ext_tname is called on every timer task execution (a hot path in the handle). The original implementation used databuff_printf to format the thread name as "%s_%s", incurring runtime overhead from format string parsing and va_list variable arguments. Replaced databuff_printf with manual concatenation using STRLEN + MEMCPY. The new approach first gets the length of both strings, checks that the total length does not exceed OB_EXTENED_THREAD_NAME_BUF_LEN (32), and then sequentially writes tname, '_', timer_name_, and '\0' using memcpy. - tname (OB_THREAD_NAME_BUF_LEN=16) and timer_name_ (16) each have ≤15 valid characters. After concatenation, the total is ≤31, which always satisfies the boundary check condition (<32). - The MEMCPY and STRLEN macros are indirectly introduced via ob_define.h, adding no new dependencies. - Full compilation passes with 0 errors. - After starting a new instance, the log output for ext_tname was compared and is completely identical to the old implementation (e.g., TimerWK0_KVCacheWash, TimerWK0_AdvanceCKPT).
The obperf flame graph shows BlockGCTimerTask::runTimerTask consuming 3.31% of CPU. Within that, the two calls to palf_handle_impl_map_.for_each (for get_total_used_disk_space_ and recycle_blocks_) each go through the ObLinkHashMap Iterator/HandleOn/revert path. In this environment, there's only 1 sys LS with a fixed palf_id of ObLSID::SYS_LS_ID (=1), making the for_each traversal entirely wasteful. Changes: 1. get_total_used_disk_space_ — Directly call get_palf_handle_impl(SYS_LS_ID, guard) instead of using for_each with the GetTotalUsedDiskSpace functor. 2. recycle_blocks_ — Directly call get_palf_handle_impl(SYS_LS_ID, guard) and inline the recycling logic (base_lsn check, block GC condition evaluation, delete_block) instead of using for_each with the LogGetRecycableFileCandidate functor. 3. Remove the now-unused GetTotalUsedDiskSpace and LogGetRecycableFileCandidate functors (both declarations and implementations). 4. Add `#include "share/ob_ls_id.h"`.
…00ms polling + simplify for single tenant. The obperf flame graph shows that the DDLTransCtr thread spends 62% of its CPU time on the timed wake-ups from `ObCond::timedwait(100ms)`. The thread wakes up every 100ms, acquires a lock, checks a flag, finds no work, and goes back to sleep—pure idle spinning. 1. `timedwait(100ms)` → `wait`: Blocks indefinitely when idle, only wakes up when `remove_task` calls `signal`, eliminating ~800k unnecessary wake-ups per day. 2. `ObHashSet<uint64_t> tenants_` → `bool need_refresh_` In single-tenant scenarios, the hashset traversal, temporary `ObArray` copy, and per-tenant loop are all unnecessary overhead. Replaced with a single boolean flag read/write. 3. In `run1`, removed the heap allocation of the `ObArray tenant_ids` and the for loop. The critical section under lock is shortened to a single boolean assignment. ## Correctness Justification - The `ObCond`'s internal `bcond_` flag + CAS mechanism ensures no lost wake-up in the signal-before-wait race window. - Reads and writes of `need_refresh_` are protected by the `SpinRWLock lock_`. `remove_task` and `run1` are serialized, eliminating the risk of a "processing + new task submission" gap causing a lost task. - DDLTransCtr is a single consumer (only `run1` is a waiter), which `ObCond` fits perfectly. If changed to multi-consumer in the future, switch to `ObThreadCond` + external predicate mode. - After changing to `wait`, `stop` still exits immediately via `wait_cond_.signal`. None. Functional semantics remain unchanged; only unnecessary CPU usage is eliminated.
… threads Replace tenant-role-based polling with restore-source-aware dynamic intervals. Both threads now use condition variable blocking: 100ms when a restore source exists, 1s (writer) or skips work (service) when idle. Skip ObRemoteLogWriter::do_thread_task_ when no restore source exists When idle, the writer no longer iterates all LSes calling get_next_sorted_task (which always returns OB_NOT_MASTER). Instead it purely blocks on the condition variable. On the transition from active to idle, one final iteration runs for cleanup of residual tasks.
## Background `build.sh rpm --init` links orders of magnitude slower than `build.sh release --init` because `build_package` enables three heavyweight optimization flags | Flag | cmake Option | Link-Time Impact | |------|-------------|-----------------| | ThinLTO | ENABLE_THIN_LTO=ON | -flto=thin, linker performs whole-program optimization | | AutoFDO | ENABLE_AUTO_FDO=ON | -fprofile-sample-use + -finline-functions, aggressive inlining | | HOTFUNC | ENABLE_HOTFUNC=ON | --symbol-ordering-file with ~32K symbols | ThinLTO is the primary culprit: .o files contain LLVM bitcode instead of native code, so the linker must re-optimize and re-generate machine code for the entire binary at link time. ## Change Route the `xrpm)` case branch directly to `do_build` with release-equivalent flags, instead of going through `build_package`. The only rpm-specific flag retained is `-DCMAKE_BUILD_RPM=ON` to satisfy the packaging platform's type detection. Before: rpm → build_package → do_build(..., ENABLE_THIN_LTO=ON, ENABLE_AUTO_FDO=ON, ENABLE_HOTFUNC=ON, ...) After: rpm → do_build(CMAKE_BUILD_TYPE=RelWithDebInfo, OB_USE_LLD=..., CMAKE_BUILD_RPM=ON) ## Scope - `build.sh` line 273-275: `xrpm)` case in `build` function - `build_package` function is untouched (still available for reuse if needed) - Other modes (debug, tgz, deb, etc.) are unaffected
OB_ERR_EMPTY_QUERY means the SRS table has not been fully imported (srs_cnt < 5152), so retrying is pointless — wait for the import notifier to trigger a fresh refresh instead of busy-looping every 1s.
The MemoryDump thread previously triggered a STAT_LABEL scan every 10 seconds, iterating all tenants * ctx_ids * chunks * blocks * objects to generate per-label memory statistics. This periodic full-memory walk caused significant CPU overhead on production machines. This commit sets STAT_LABEL_INTERVAL to INT64_MAX, effectively disabling periodic auto-scanning. The original code path is fully preserved. To manually trigger a memory stat scan ALTER SYSTEM REFRESH MEMORY STAT; Other on-demand dump triggers (kill -62 signal, etc/dump.config) are unaffected and continue to work normally. The STAT_LABEL_INTERVAL constant remains as a single point for future standardization — when a configurable interval parameter is introduced this constant should be replaced with that parameter.
…polling ## Problem obperf flame graph shows ObPluginVectorIndexLoadScheduler::reload_tenant_task consuming significant CPU during idle periods. The timer fires every 1s (VEC_INDEX_SCHEDULAR_BASIC_PERIOD) but all actual task schedules have a 10s interval (schedule_interval), meaning ~90% of timer cycles perform unnecessary work - check_and_load_task_executors unconditionally calls check_and_set_thread_pool clear_old_task_ctx_if_need, load_task_from_inner_table (SQL query to __all_vector_index_task) every 1s - embedding_task_exec_.load_task / start_task called unconditionally every 1s - check_tenant_memory, check_has_vector_index, reload_tenant_task run even when no schedule window is open ## Changes (3 fixes in 1 file) 1. check_and_load_task_executors: add early return when neither HNSW_OPTIMIZE nor IVF_TASK schedule window is open, skipping all inner-table reads and thread pool checks during idle cycles. 2. Gate embedding_task_exec_ on can_schedule(HNSW_OPTIMIZE) - check_and_load_task_executors: embedding_task_exec_.load_task - start_task_executors: embedding_task_exec_.start_task These were the only task executor operations not protected by can_schedule. 3. run_task fast path: when all four schedule types are false, return before check_tenant_memory / check_has_vector_index / reload_tenant_task / check_and_execute_tasks / schedule_finish. Leader switch detection (need_do_for_switch_) is checked before the fast path and remains responsive on every timer tick. ## Verification - Compiled and deployed on x86_64 - mysqltest suite "vector_index": 65 tests run, 0 new failures 5 pre-existing rejects confirmed with original binary (EXPLAIN format width, HNSW approximate search non-determinism, internal table listing) - Logs confirm check_tenant_memory reduced from every 1s to every ~10s; check_and_load_task_executors / reload_tenant_task no longer appear on every cycle
…task` to reduce CPU overhead. **Issue** The obperf flame graph shows that `ObPlanCacheEliminationTask::run_free_cache_obj_task` consumes 3.65% of CPU (1503 out of 41181 samples), making it the single most expensive item within the `runTimerTask` periodic callback. **Root Cause** `run_free_cache_obj_task` is triggered by default every `plan_cache_evict_interval` (with `_ob_plan_cache_gc_strategy` defaulting to `REPORT`). It performs two full traversals of the `alloc_cache_obj_map_` hash table: 1. `dump_deleted_objs<DUMP_ALL>` — Traverses all objects, checking the `should_release` condition for each one (`ref_count > 0 && log_del_time < safe_timestamp`), and collects suspected leaked objects into the `deleted_objs` array. 2. `dump_all_objs` — Traverses all objects again to collect a snapshot and log INFO-level messages. Both traversals serve only diagnostic purposes (summarizing memory usage and logging) and perform no actual release operations. The real cleanup of leaked objects is performed by the DBA-manually-triggered `FLUSH PLAN CACHE` / `LIB CACHE` / `PL CACHE` path (functions like `flush_plan_cache` call `dump_deleted_objs` and subsequently execute `destroy_cache_obj`). **Changes Made** - Removed the `run_free_cache_obj_task` function definition (29 lines). - Removed the `dump_all_objs` function definition (13 lines) — it was only called by `run_free_cache_obj_task`. - Removed the `gc_strategy` check and its call site within `ObPlanCacheEliminationTask::runTimerTask` (4 lines). - Removed the `PlanCacheGCStrategy` enum and `get_plan_cache_gc_strategy` (12 lines). - Kept `dump_deleted_objs<>` / `dump_deleted_objs_by_ns` — these are still used by the `flush` family of functions, which actually perform the cleanup of leaked objects and are low-frequency, manual DBA operations. - Kept `plan_cache_gc_confs` — used by `ObConfigPlanCacheGCChecker` to validate the `_ob_plan_cache_gc_strategy` configuration value. **Impact** - `runTimerTask` now only executes `run_plan_cache_task` (eviction) and the periodic `flush_plan_cache`, and no longer triggers full hash table traversals. - The automatic memory leak diagnostics for the plan cache have been removed: leaked objects can no longer be automatically discovered and logged via the periodic task. Logging for leaked objects still requires a manual `FLUSH PLAN CACHE` to trigger. **Configuration** The `_ob_plan_cache_gc_strategy` configuration (default `REPORT`) no longer affects `runTimerTask` behavior. The configuration item itself is retained for compatibility with existing deployment scripts.
…age/update_queue_size, and relax TIME_SLICE_PERIOD from 10ms to 1s. Flame graph analysis (obperf, 155,396 total samples) of the 10ms scheduling loop in ObMultiTenant::run1 shows two types of unnecessary periodic overhead in ObTenant::timeup: 1. update_token_usage — Every 1 second, it traverses the worker list, atomically clears idle_us_, and calculates token_usage_. However, token_usage_ and worker_us_ are only read by virtual tables (ob_all_virtual_sys_stat / ob_all_virtual_res_mgr_sys_stat) for display and do not affect any scheduling decisions. 2. update_queue_size — Every 10ms, it calls ObServerConfig::get_instance to read the tenant_task_queue_size configuration item. Under normal conditions, this configuration doesn't change, making this an ineffective polling operation. 3. TIME_SLICE_PERIOD = 10000 (10ms) — The flame graph shows ObTenant::timeup itself consumes 3.49% of total CPU, with lock operations (~47%), retry queue processing (32%), and worker recycling (19%). These are triggered every 10ms, which is too frequent. Relaxing this to 1s (1000000us) can significantly reduce lock contention overhead, as the tenant inspection semantics (worker start/stop, retry replay) are not sensitive to millisecond-level delays. - Remove the update_token_usage method body and its call within timeup. - Remove the update_queue_size method body and its call within timeup. - Change get_token_usage and get_worker_time to directly return 0 (preserving interface compatibility for virtual tables). - Remove member variables: token_usage_, token_usage_check_ts_, worker_us_. - Replace update_queue_size with an inline set_queue_limit(int64_t) that directly operates on req_queue_. - Remove the corresponding members from the constructor's initialization list. - TIME_SLICE_PERIOD: 10000 → 1000000 (10ms → 1s) - Add reload_tenant_task_queue_size: acquires a read lock, reads GCONF.tenant_task_queue_size, and calls tenant_->set_queue_limit. - Change the interval for periodic tenant info dump: 10s → 1s (to improve monitoring granularity). - Add a call to reload_tenant_task_queue_size at the end of ObServerReloadConfig::operator, piggybacking on the server-level configuration hot-reload path. - tenant_task_queue_size will no longer be polled every 10ms. Instead, it will be updated passively when triggered by `ALTER SYSTEM SET`. Configuration change path: RPC → config_mgr_->reload_config → ObServerReloadConfig::operator → ObReloadConfig::operator updates GCONF → reload_tenant_task_queue_size → set_queue_limit(GCONF.tenant_task_queue_size). - The output of update_token_usage was only used for display. After removal, virtual table queries will return 0, which does not affect system behavior. - Relaxing TIME_SLICE_PERIOD from 10ms to 1s reduces the timeup call frequency from 100Hz to 1Hz, directly cutting down the cumulative overhead from lock contention, retry drain, and worker traversal.
## Background ObBGThreadMonitor is a thread-level function execution timeout watchdog designed to monitor background thread function execution time. It was fully initialized and running (consuming ~0.52% CPU per flame graph analysis), but **zero** business code was instrumented with MonitorGuard — no threads or functions were ever registered for monitoring. Flame graph data (155,396 total samples) ObBGThreadMonitorTimerTask::runTimerTask: 805 samples (0.52%) — all self-samples, spent scanning 500 empty MonitorEntryStack arrays every second (ObClockGenerator::getClock + 2500 timestamp checks + spinlock acquire/release per cycle). ## Changes ### Deleted - src/share/ob_bg_thread_monitor.h — all classes: MonitorGuard, ObBGThreadMonitor ObBGThreadMonitorTimerTask, MonitorEntryStack, MonitorEntry BGDummyCallback, IBGCallback, MonitorCallbackWrapper ObTSIBGMonitorMemory, ObTSIBGMonitorSlotInfo, macros BG_MONITOR_GUARD(_DEFAULT) BG_NEW_CALLBACK, BG_DELETE_CALLBACK - src/share/ob_bg_thread_monitor.cpp — all implementations ### Modified - src/observer/ob_server.cpp removed #include, init/start/stop/wait/destroy lifecycle calls - src/share/ob_thread_define.h removed TG_DEF(BGThreadMonitor, BGThreadMonitor, TIMER) - src/share/CMakeLists.txt removed ob_bg_thread_monitor.cpp from build ## Risk Assessment - Zero-risk: no business code references MonitorGuard/BG_MONITOR_GUARD. Search across entire src/ confirmed no includes of ob_bg_thread_monitor.h outside the deleted files and ob_server.cpp lifecycle management. - No test files reference the framework. ## Verification - Debug build: passed (make -j80, [100%] Built target observer) - Instance deployment: deployed to ~/ob1, startup clean, SQL connectivity OK
…Mgr Timer Task. ObPxTargetMgr executes `refresh_statistics` every 500ms. Within this, it uses the call chain `get_dummy_leader` -> `check_dummy_location_credible` -> `get_role` to determine if the current node is a Leader or Follower. Internally, `get_role` performs MTL_SWITCH + check_palf_exist + open_palf + get_role to query the Paxos role of SYS_LS. This is a CPU hotspot per tick (flame graph shows runTimerTask at 0.57%, with lib_mtl_switch accounting for 10.04% of that). SeekDB is an observer-lite single-replica mode. `ElectionImpl::get_role` always returns LEADER, and the Leader never switches. This entire Leader/Follower coordination mechanism is effectively dead code. ## Changes Made Removed the following dead code (-237 lines): 1. `get_dummy_leader` — MTL_SWITCH + nonblock_get_leader to locate the Leader. 2. `check_dummy_location_credible` — Calls `get_role` to verify cache credibility. 3. `get_role` — MTL_SWITCH + open_palf to get the Paxos role. 4. `refresh_dummy_location` — Empty function, only returns OB_SUCCESS. 5. `query_statistics` — RPC logic for Followers to report resource increments to the Leader. 6. `reset_follower_statistics` — Resets statistics on the Follower side. Removed member variables that are no longer needed: - `cluster_id_`, `dummy_cache_leader_`, `rpc_proxy_`, `need_send_refresh_all_` Simplified `refresh_statistics`: - Removed the `get_dummy_leader` call and the Follower branch. - On the first tick, executes `reset_leader_statistics` once; subsequent ticks return directly. This change applies only to single-replica (observer-lite) deployments. Multi-machine OceanBase clusters, which have a genuine need for Leader/Follower switching, cannot use this modification. - Compilation passes. - OBD deployment of a single-node instance starts normally. - MySQL connections work normally.
…n-memory atomic CAS
## Problem
The Fetcher main loop (ObCSFetcher::run1) calls try_advance_refresh_scn_ every 200ms
unconditionally, regardless of IDLE or ACTIVE mode. Under IDLE mode (no async vector
index tables), the loop iterates every 10ms, and get_refresh_scn returns GTS each time.
This results in a SQL UPDATE against __all_global_stat every 200ms even when the value
has not changed, causing unnecessary database writes.
Additionally, the Worker path (do_finish_batch_) also writes refresh_scn via SQL
within a transaction, and the virtual table query reads it via SQL — all incurring
database round-trips for a monotonically-advancing counter that only needs to be
queryable within the same process.
## Solution
Migrate refresh_scn from __all_global_stat (SQL-persisted) to in-memory management
in ObCSDispatcher using atomic variables (ATOMIC_LOAD/ATOMIC_STORE/ATOMIC_BCAS)
1. ObCSDispatcher::refresh_scn_ becomes an atomic variable with CAS-based
update_refresh_scn that advances the value only when the new value is larger.
2. Fetcher path (try_advance_refresh_scn_)
- Was: ObGlobalStatProxy::advance_change_stream_refresh_scn → SQL UPDATE
- Now: dispatcher_->update_refresh_scn → in-memory CAS, zero SQL
3. Worker path (do_finish_batch_)
- refresh_scn advancement moves from release_batch to after batch commit
- Uses dispatcher_->update_refresh_scn instead of SQL
4. Read paths
- wait_refresh_scn in ObChangeStreamMgr reads from dispatcher_->get_refresh_scn
- Virtual table ob_all_virtual_change_stream_refresh_stat reads in-memory state
- init_refresh_scn_ still loads from __all_global_stat once as recovery baseline
and allows rollback on reload (recovery semantics differ from runtime advancement)
5. Cleanup
- Removes refresh_scn_inited flag and init_refresh_scn public method
- Refactors release_batch to no longer manage refresh_scn directly
## Files Changed
- ob_change_stream_dispatcher.{h,cpp}: atomic refresh_scn_, update_refresh_scn, cleanup
- ob_change_stream_fetcher.cpp: Fetcher uses dispatcher->update_refresh_scn
- ob_change_stream_worker.cpp: Worker commits refresh_scn after batch success
- ob_change_stream_mgr.cpp: wait_refresh_scn reads from in-memory dispatcher
- ob_all_virtual_change_stream_refresh_stat.cpp: virtual table reads from memory
- wait_cs_sync.inc: mysqltest adjustment
…anual concatenation using MEMCPY. set_ext_tname is called each time a timer task executes (a hot path in handle). The original implementation used databuff_printf to format the thread name as "%s_%s", incurring runtime overhead from format string parsing and va_list variable arguments. Replaced databuff_printf with manual concatenation using STRLEN + MEMCPY: first, get the length of both strings, check that the total length does not exceed OB_EXTENED_THREAD_NAME_BUF_LEN (32), then sequentially memcpy the tname, '_', timer_name_, and '\0'. - tname (OB_THREAD_NAME_BUF_LEN=16) and timer_name_ (16) each have ≤15 valid characters. After concatenation, the total is ≤31, which always satisfies the boundary check condition of <32. - The MEMCPY / STRLEN macros are indirectly introduced via ob_define.h, adding no new dependencies. - Full compilation passes with 0 errors. - After starting a new instance, compared the log output for ext_tname; the format is completely consistent with the old implementation (e.g., TimerWK0_KVCacheWash, TimerWK0_AdvanceCKPT).
The obperf flame graph shows BlockGCTimerTask::runTimerTask consuming 3.31% of CPU, with two calls to palf_handle_impl_map_.for_each (for `get_total_used_disk_space_` and `recycle_blocks_`) each going through the ObLinkHashMap Iterator/HandleOn/revert path. In this environment, there is only 1 sys LS with a fixed palf_id of ObLSID::SYS_LS_ID (=1), making the for_each traversal entirely wasteful. Changes: 1. `get_total_used_disk_space_` — Directly call `get_palf_handle_impl(SYS_LS_ID, guard)` instead of `for_each(GetTotalUsedDiskSpace functor)`. 2. `recycle_blocks_` — Directly call `get_palf_handle_impl(SYS_LS_ID, guard)` and inline the recycling logic (base_lsn check, block GC condition check, delete_block) instead of `for_each(LogGetRecycableFileCandidate functor)`. 3. Remove the now-unused `GetTotalUsedDiskSpace` and `LogGetRecycableFileCandidate` functors (both declarations and implementations). 4. Add `#include "share/ob_ls_id.h"`.
OB_ERR_EMPTY_QUERY means the SRS table has not been fully imported (srs_cnt < 5152), so retrying is pointless — wait for the import notifier to trigger a fresh refresh instead of busy-looping every 1s.
The MemoryDump thread previously triggered a STAT_LABEL scan every 10 seconds, iterating all tenants * ctx_ids * chunks * blocks * objects to generate per-label memory statistics. This periodic full-memory walk caused significant CPU overhead on production machines. This commit sets STAT_LABEL_INTERVAL to INT64_MAX, effectively disabling periodic auto-scanning. The original code path is fully preserved. To manually trigger a memory stat scan ALTER SYSTEM REFRESH MEMORY STAT; Other on-demand dump triggers (kill -62 signal, etc/dump.config) are unaffected and continue to work normally. The STAT_LABEL_INTERVAL constant remains as a single point for future standardization — when a configurable interval parameter is introduced this constant should be replaced with that parameter.
…age/update_queue_size, and relax TIME_SLICE_PERIOD from 10ms to 1s. Flame graph analysis (obperf, 155,396 total samples) of the 10ms scheduling loop in ObMultiTenant::run1 shows two types of unnecessary periodic overhead in ObTenant::timeup. 1. update_token_usage — Every 1 second, it traverses the worker list, atomically clears idle_us_, and calculates token_usage_. However, token_usage_ and worker_us_ are only read by virtual tables (ob_all_virtual_sys_stat / ob_all_virtual_res_mgr_sys_stat) for display and do not affect any scheduling decisions. 2. update_queue_size — Every 10ms, it calls ObServerConfig::get_instance to read the tenant_task_queue_size configuration item. Under normal conditions, this configuration doesn't change, making this an ineffective polling operation. 3. TIME_SLICE_PERIOD = 10000 (10ms) — The flame graph shows ObTenant::timeup itself consumes 3.49% of total CPU, with lock operations (~47%), retry queue processing (32%), and worker recycling (19%). Being triggered every 10ms is too frequent. Relaxing this to 1s (1000000us) can significantly reduce lock contention overhead, and the semantics of tenant inspection (worker start/stop, retry replay) are not sensitive to millisecond-level delays. - Delete the update_token_usage method body and remove its call in timeup. - Delete the update_queue_size method body and remove its call in timeup. - Change get_token_usage and get_worker_time to directly return 0 (preserving interface compatibility for virtual tables). - Remove member variables: token_usage_, token_usage_check_ts_, worker_us_. - Replace update_queue_size with an inline set_queue_limit(int64_t) that directly operates on req_queue_. - Remove the corresponding members from the constructor's initialization list. - TIME_SLICE_PERIOD: 10000 → 1000000 (10ms → 1s). - Add reload_tenant_task_queue_size: acquire a read lock, read GCONF.tenant_task_queue_size, and call tenant_->set_queue_limit. - Timing interval for dumping tenant information: 10s → 1s (improves monitoring granularity). - Add a call to reload_tenant_task_queue_size at the end of ObServerReloadConfig::operator, piggybacking on the server-level configuration hot-reload path. - tenant_task_queue_size will no longer be polled every 10ms. Instead, it will be updated passively when triggered by ALTER SYSTEM SET. Configuration change path: RPC → config_mgr_->reload_config → ObServerReloadConfig::operator → ObReloadConfig::operator updates GCONF → reload_tenant_task_queue_size → set_queue_limit(GCONF.tenant_task_queue_size). - The output of update_token_usage was only used for display. After removal, virtual table queries will return 0, which does not affect system behavior. - Relaxing TIME_SLICE_PERIOD from 10ms to 1s reduces the timeup call frequency from 100Hz to 1Hz, directly cutting down the cumulative overhead from lock contention, retry drain, and worker traversal.
## Background ObBGThreadMonitor is a thread-level function execution timeout watchdog designed to monitor background thread function execution time. It was fully initialized and running (consuming ~0.52% CPU per flame graph analysis), but **zero** business code was instrumented with MonitorGuard — no threads or functions were ever registered for monitoring. Flame graph data (155,396 total samples) ObBGThreadMonitorTimerTask::runTimerTask: 805 samples (0.52%) — all self-samples, spent scanning 500 empty MonitorEntryStack arrays every second (ObClockGenerator::getClock + 2500 timestamp checks + spinlock acquire/release per cycle). ## Changes ### Deleted - src/share/ob_bg_thread_monitor.h — all classes: MonitorGuard, ObBGThreadMonitor ObBGThreadMonitorTimerTask, MonitorEntryStack, MonitorEntry BGDummyCallback, IBGCallback, MonitorCallbackWrapper ObTSIBGMonitorMemory, ObTSIBGMonitorSlotInfo, macros BG_MONITOR_GUARD(_DEFAULT) BG_NEW_CALLBACK, BG_DELETE_CALLBACK - src/share/ob_bg_thread_monitor.cpp — all implementations ### Modified - src/observer/ob_server.cpp removed #include, init/start/stop/wait/destroy lifecycle calls - src/share/ob_thread_define.h removed TG_DEF(BGThreadMonitor, BGThreadMonitor, TIMER) - src/share/CMakeLists.txt removed ob_bg_thread_monitor.cpp from build ## Risk Assessment - Zero-risk: no business code references MonitorGuard/BG_MONITOR_GUARD. Search across entire src/ confirmed no includes of ob_bg_thread_monitor.h outside the deleted files and ob_server.cpp lifecycle management. - No test files reference the framework. ## Verification - Debug build: passed (make -j80, [100%] Built target observer) - Instance deployment: deployed to ~/ob1, startup clean, SQL connectivity OK
Problem ObCSDispatcher::run1 uses dispatch_cond_.wait(100) in its idle loop. Each wakeup triggers futex kernel operations + ObWaitEventGuard + ObDiagnosticInfo::end_wait_event accounting, creating ~10 scheduling events per second even when there is no work. Flame graph (alloc view) shows this accounts for 0.95% of system-wide allocation events. Analysis The condition variable is correctly used with the standard pattern 1. Lock mutex (ObThreadCondGuard) 2. Check condition under lock 3. cond.wait(timeout) Signal is sent under the same mutex in push, so POSIX guarantees that a waiting thread is woken immediately regardless of timeout. The timeout is purely a fallback; longer timeouts do not risk losing signals. Solution 1. Increase dispatch_cond_.wait(100) to dispatch_cond_.wait(10000) idle wakeups drop from 10/sec to 0.1/sec, reducing alloc overhead by ~100x in the idle path. 2. Add dispatch_cond_.signal in stop: prevents the 10s timeout from delaying shutdown when the thread is blocked in wait. Verification - Deployed binary, confirmed idle wakeup intervals are exactly 10s (wait_duration_us = 10000067, 10000155, 10000087, 10000078). - Measured shutdown latency: 168ms (signal in stop works correctly). - Push→signal still wakes dispatcher immediately per condvar semantics (requires change stream activity to exercise directly).
* Reduce small-object memory overhead * Simplify LS map for single log stream * Simplify single-LS iteration paths * Merge remote-tracking branch 'origin/master' into task/2026071000117341494 # Conflicts # src/observer/report/ob_tenant_meta_checker.cpp * Merge remote-tracking branch 'origin/master' into task/2026071000117341494 # Conflicts # src/observer/virtual_table/ob_all_virtual_cs_replica_tablet_stats.cpp # src/observer/virtual_table/ob_all_virtual_cs_replica_tablet_stats.h * Simplify local GTS queues * Simplify longops manager storage * Lazy init SQL plan monitor storage * Lazy create major merge progress maps * Reduce small-memory background overhead * Merge remote-tracking branch 'origin/master' into task/2026071000117341494 # Conflicts # deps/oblib/src/lib/utility/ob_mod_define.h # src/logservice/applyservice/ob_log_apply_service.h # src/logservice/ob_log_service.cpp # src/logservice/palf/log_config_mgr.cpp # src/logservice/palf/log_config_mgr.h # src/logservice/palf/log_sliding_window.cpp # src/logservice/palf/log_sliding_window.h # src/logservice/palf/palf_handle_impl.cpp # src/logservice/palf/palf_handle_impl.h # src/observer/ob_server.cpp # src/observer/ob_server.h # src/observer/ob_service.cpp # src/observer/omt/ob_multi_tenant.cpp # src/observer/report/ob_tenant_meta_checker.cpp # src/observer/scheduler/ob_tenant_dag_scheduler.h # src/observer/virtual_table/ob_all_virtual_checkpoint.cpp # src/observer/virtual_table/ob_all_virtual_checkpoint.h # src/observer/virtual_table/ob_all_virtual_ls_info.cpp # src/observer/virtual_table/ob_all_virtual_ls_info.h # src/observer/virtual_table/ob_all_virtual_memstore_info.cpp # src/observer/virtual_table/ob_all_virtual_memstore_info.h # src/observer/virtual_table/ob_all_virtual_minor_freeze_info.cpp # src/observer/virtual_table/ob_all_virtual_minor_freeze_info.h # src/observer/virtual_table/ob_all_virtual_obj_lock.cpp # src/observer/virtual_table/ob_all_virtual_obj_lock.h # src/observer/virtual_table/ob_all_virtual_tablet_ddl_kv_info.cpp # src/observer/virtual_table/ob_all_virtual_tablet_ddl_kv_info.h # src/observer/virtual_table/ob_all_virtual_tablet_info.cpp # src/observer/virtual_table/ob_all_virtual_tablet_info.h # src/observer/virtual_table/ob_all_virtual_transaction_checkpoint.cpp # src/observer/virtual_table/ob_all_virtual_transaction_checkpoint.h # src/observer/virtual_table/ob_all_virtual_transaction_freeze_checkpoint.cpp # src/observer/virtual_table/ob_all_virtual_transaction_freeze_checkpoint.h # src/observer/virtual_table/ob_all_virtual_tx_data_table.cpp # src/observer/virtual_table/ob_all_virtual_tx_data_table.h # src/observer/virtual_table/ob_all_virtual_tx_lock_stat.cpp # src/observer/virtual_table/ob_all_virtual_tx_lock_stat.h # src/rootserver/freeze/ob_major_merge_progress_checker.cpp # src/share/ob_ls_id.h # src/share/rc/ob_module_provider.h # src/share/rc/ob_tenant_base.h # src/sql/das/ob_das_parallel_handler.h # src/sql/engine/expr/ob_expr_vec_vector.cpp # src/sql/monitor/ob_monitor_node.h # src/sql/monitor/ob_sql_plan_monitor_node_list.cpp # src/sql/session/ob_sql_session_info.cpp # src/sql/session/ob_sql_session_info.h # src/storage/compaction/ob_tenant_compaction_progress.cpp # src/storage/compaction/ob_tenant_freeze_info_mgr.cpp # src/storage/ddl/ob_ddl_inc_redo_log_writer.cpp # src/storage/ls/ob_freezer.cpp # src/storage/meta_store/ob_server_storage_meta_replayer.cpp # src/storage/multi_data_source/runtime_utility/mds_tenant_service.cpp # src/storage/mview/ob_mview_sched_job_utils.cpp # src/storage/slog_ckpt/ob_tenant_storage_checkpoint_writer.cpp # src/storage/tablelock/ob_table_lock_service.cpp # src/storage/tx/ob_gti_source.cpp # src/storage/tx/ob_gti_source.h # src/storage/tx/ob_gts_rpc.h # src/storage/tx/ob_gts_source.cpp # src/storage/tx/ob_gts_source.h # src/storage/tx/ob_gts_task_queue.h # src/storage/tx/ob_tablet_to_ls_cache.cpp # src/storage/tx/ob_tablet_to_ls_cache.h # src/storage/tx/ob_timestamp_service.cpp # src/storage/tx/ob_timestamp_service.h # src/storage/tx/ob_trans_deadlock_adapter.cpp # src/storage/tx/ob_trans_factory.cpp # src/storage/tx/ob_trans_factory.h # src/storage/tx/ob_trans_id_service.cpp # src/storage/tx/ob_trans_id_service.h # src/storage/tx/ob_ts_mgr.cpp # src/storage/tx/ob_ts_mgr.h # src/storage/tx/ob_tx_api.cpp # src/storage/tx/ob_tx_loop_worker.cpp # src/storage/tx_storage/ob_checkpoint_service.cpp # src/storage/tx_storage/ob_empty_shell_task.cpp # src/storage/tx_storage/ob_ls_map.cpp # src/storage/tx_storage/ob_ls_map.h # src/storage/tx_storage/ob_ls_service.cpp # src/storage/tx_storage/ob_ls_service.h # src/storage/tx_storage/ob_tablet_gc_service.cpp # src/storage/tx_storage/ob_tenant_freezer.cpp # src/storage/tx_storage/ob_tenant_freezer_rpc.cpp * Restore transaction context map bucket count * Merge origin/master into small object optimizations * Fix tablet stat test expectations * Merge origin/master into small object optimizations * Merge origin/master into small object optimizations --------- Co-authored-by: hnwyllmm <hnwyllmm@users.noreply.github.com> Co-authored-by: wangyunlai.wyl <wangyunlai.wyl@oceanbase.com>
* make snapshot gc renewal event-driven * make snapshot gc renewal standby-safe * extract snapshot gc renewal from detector * clarify snapshot gc renewal state * simplify snapshot gc renewal scheduling * use last renewed scn for gc coverage * refine snapshot gc renewal targets * merge master into snapshot gc branch * fix freeze detector test after manager refactor * fix freeze detector test after manager refactor --------- Co-authored-by: hnwyllmm <hnwyllmm@users.noreply.github.com> Co-authored-by: wangyunlai.wyl <wangyunlai.wyl@oceanbase.com>
* fix: make floating-point integer casts deterministic * fix overflow default value * fix related case --------- Co-authored-by: hnwyllmm <hnwyllmm@users.noreply.github.com> Co-authored-by: ep-12221 <ep-12221@users.noreply.github.com>
* fix: stabilize seeded distribution results * remove static cast * Update distribution source file references * Rename deterministic distribution identifiers --------- Co-authored-by: hnwyllmm <hnwyllmm@users.noreply.github.com> Co-authored-by: ep-12221 <ep-12221@users.noreply.github.com>
Co-authored-by: hnwyllmm <hnwyllmm@users.noreply.github.com>
* refactor(sql): remove rich vectorization engine 2.0 * Merge branch 'master' into issue/2026072200117650903 * fix farm * fix farm2 * fix farm3 * fix farm4 * fix farm5 * fix(sql): preserve scalar assignment output evaluation * fix(farm): stabilize assignment execution and batch DML drain * Merge branch 'master' into issue/2026072200117650903 * Merge branch 'master' into issue/2026072200117650903 * Merge branch 'master' into issue/2026072200117650903 --------- Co-authored-by: LINxiansheng <LINxiansheng@users.noreply.github.com>
Co-authored-by: hnwyllmm <hnwyllmm@users.noreply.github.com>
* fix: handle stale routine definers in macOS mysqltests * revert: preserve recyclebin routine definer checks * revert: restore product bug and timing-sensitive cases * fix(mysqltest): clean up residual test state * Merge remote-tracking branch 'origin/master' into issue/2026072700117768033 * revert(mysqltest): remove cleanup-related case changes --------- Co-authored-by: hnwyllmm <hnwyllmm@users.noreply.github.com> Co-authored-by: ep-12221 <ep-12221@users.noreply.github.com>
Co-authored-by: footka <footka@users.noreply.github.com>
…nts (oceanbase#1221) * Replace sys task status array with dynamic linked list * Replace fixed-size comment buffer with ObString --------- Co-authored-by: hnwyllmm <hnwyllmm@users.noreply.github.com>
* refactor: remove unsupported enum-driven features * docs: remove internal yuque links from review * fix: align tests after feature removal * chore: remove review artifacts and stale tests --------- Co-authored-by: footka <footka@users.noreply.github.com>
Member
Author
|
The Dima issue is about optimizing change stream refresh. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Task Description
Change Stream previously used a single refresh SCN for both Worker-applied progress and user-visible refresh completion. This could lead to issues where the Fetcher periodically advanced this value without a Worker commit, while the Dispatcher also used it to skip transactions during recovery. Concurrent asynchronous index creation or deletion could therefore cause a refresh to return before the corresponding schema generation and log work were safely completed, or allow log reclamation to advance using a mismatched schema decision.
Solution Description
ready_schema_versionandlast_no_async_index_drained_schema_version.min_dep_lsnadvancement to require an exact match between the runtime schema version and the Fetcher's ready state.ChangeStreamMgr.Passed Regressions
Upgrade Compatibility
No persistent metadata format changes. The existing
change_stream_refresh_scnglobal-state item is retained as the Worker-applied waterline. The user-visible refresh waterline is maintained in memory and is initialized from this persisted value during startup or recovery.Other Information
This MR contains a single commit:
bdd5e6f9865 fix change stream refresh consistency.Release Note
Fixed an issue in Change Stream where concurrent asynchronous index operations could cause a refresh to complete before the associated schema and log work was finished, or lead to log reclamation using an incorrect schema state, ensuring refresh operations are now consistent with the actual completion of background work.